From 2e070380e8d47f7c8b7ace1b2cf7b2cf2c8e8f35 Mon Sep 17 00:00:00 2001 From: David Prevost <77302423+dprevost-LMI@users.noreply.github.com> Date: Thu, 29 May 2025 17:18:24 -0400 Subject: [PATCH] feat(runner+browserstack): Mask sensitive data for Reporters (and more) (#13938) * Draft solution masking on the `onBeforeCommand` Use same default string as appium Use same default token as appium Full working solutions Fix compilation problems Remove it from capabilities from simplicity Fix grammar Use Regex native type Draft mask body & command args in webdriver directly Full working solution Removing old solution and complete the new one Revert "Draft solution masking on the `onBeforeCommand`" This reverts commit 5feb324af2d2a1ded4e81c86cad2670a34150b7b. Code review Update packages/webdriver/src/command.ts Co-authored-by: Christian Bromann Unit test on the regExp converter from string Add missing case of missing starting and ending slash Remove undesired changes Convert string to regExps only once Fix problem after rebase remove occurences of any Add new mask parameter to elementSendKeys to mask text input and more Added also the `/session/:sessionId/appium/settings` endpoints to generate regEx from the sensitive value so that we can mask the value int he appium log on top of WebDriver log and other touchpoint e2e test for masking text value with setValue command Use `updateSettings` only with Appium Review solution to be completely outside of `request.ts` Use env `WDIO_LOG_MASKING_PATTERNS` to pass regex and mask log Full working solutions to mask logs with 0 1 or 2 capturing group Working solution to mask for example BrowserStack key in logs Code review Fix masking not working when output directly in console mask string logging only Better code for wdio-logger masking + more unit tests - Better typing in logger to reflect reality of having only 1 arg of type string of 3 args where the third can be anything. - Move masking utility code into an utils file and add better unit tests - Add `setMaskingPatterns` following `setLogLevelsConfig` for more flexible configuration - Fix a couple fo edge cases - Ensure we mask the output of serializers too Have mask in command optional and fix ut - Have mask optional in send and add command to not interfere with expected output and be less "breaking" if masking is not used - fix ut affected by new logData eventhandler Better masking code for command with masking at true - Move masking code for command into an utility - Use args better with index to fetch mask and text - Apply masking only if the command has the param 'mask', 'text' and mask is true - Share the logger replacer with this package - Fix Logger mock for the `SENSITIVE_DATA_REPLACER` - Replace the sensitive info in both the body and args - Make sure we reuse existing body and args if no masking require for better backward compatibility - Add unit test for commands with masking Review e2e tests to had a input with masking case Remove `.only` on tests Remove fail tentative Tentative fix for failure in `react.test.tsx` - react.test.tsx: SyntaxError: Importing binding name 'SENSITIVE_DATA_REPLACER' is not found * Use the new appium http header masking automatically * fix: used an additional non-typed options to pass the masking property * Add an options object for future needs * Fix masking e2e * Be more verbose on the unmasked vs masked version in the code * Update packages/wdio-logger/src/index.ts Co-authored-by: Christian Bromann * Update packages/wdio-logger/src/index.ts Co-authored-by: Christian Bromann * Code review - Use optional options instead - Add more comment on intentions * Update packages/webdriver/src/types.ts * Update packages/webdriverio/src/types.ts * Update packages/webdriver/tests/command.test.ts * Update packages/webdriverio/src/types.ts * Update packages/webdriver/tests/utils.test.ts * On more UT for emit and log.info cases with masked data * make sure the capture data are reset * Better integration of the appium header * final code review * Fix pipeline * Fix depcheck * Fix this.client header undesired alteration * Ensure more code coverage * Apply suggestions from code review Apply suggestions from code review Apply suggestions from code review * Code reviews * Fix `CommandRuntimeOptions is not defined` * Review to mask all captured group + documentation * Add more verbose e2e * Add `maskingPatterns` capabilities to set easily masking patterns * Code review * Test if this is causing e2e to fails! * Fix typo * Rectify doc since it is not a capability * More robust masking algorithm supporting logging of string array - Apply masking on the entire string after stripping the colors (ansi) for a more robust algorithm - The above allow to support regular expression with keyword like `RESULT` or `COMMAND` or `BIDI COMMAND` - Downside is in the console some colors are lost - Enhance doc for the above and show an example to configure all loggers since using wdio.conf object not always work for all of them * Simplify `setMaskingPatterns` API * Add JSDOC and simplify `mask` following previous changes * Update packages/webdriverio/src/types.ts Co-authored-by: Christian Bromann * Update packages/webdriver/src/types.ts Co-authored-by: Christian Bromann * Use suggested return-early version of mask in used command.ts * Fix grammar + review doc * be more human friendly * Test * Code review * Fix new line sometimes stripped away * Update packages/wdio-logger/README.md Co-authored-by: Christian Bromann * Apply suggestions from code review Co-authored-by: Christian Bromann * Last comment review checking types in setMaskingPatterns * Add doc on `maskingPatterns` in Configuration.md * Add guides on security * Apply suggestions from code review * Add missing import in example --------- Co-authored-by: Christian Bromann --- __mocks__/@wdio/logger.ts | 4 + e2e/wdio/headless/test.e2e.ts | 12 + e2e/wdio/wdio.conf.ts | 1 + packages/wdio-logger/README.md | 48 ++++ packages/wdio-logger/src/browser.ts | 3 + packages/wdio-logger/src/index.ts | 96 ++++++-- packages/wdio-logger/src/utils.ts | 75 ++++++ packages/wdio-logger/tests/node.test.ts | 203 ++++++++++++++- packages/wdio-logger/tests/utils.test.ts | 232 ++++++++++++++++++ packages/wdio-runner/src/index.ts | 5 + packages/wdio-types/src/Options.ts | 10 +- packages/webdriver/src/command.ts | 58 +++-- packages/webdriver/src/constants.ts | 9 + packages/webdriver/src/request/request.ts | 4 +- packages/webdriver/src/request/types.ts | 1 + packages/webdriver/src/types.ts | 13 + packages/webdriver/src/utils.ts | 54 +++- packages/webdriver/tests/command.test.ts | 171 ++++++++++++- .../src/commands/element/addValue.ts | 12 +- .../src/commands/element/setValue.ts | 8 +- packages/webdriverio/src/types.ts | 7 + website/_sidebars.json | 3 +- website/docs/Configuration.md | 17 ++ website/docs/Security.md | 162 ++++++++++++ 24 files changed, 1149 insertions(+), 59 deletions(-) create mode 100644 packages/wdio-logger/src/utils.ts create mode 100644 packages/wdio-logger/tests/utils.test.ts create mode 100644 website/docs/Security.md diff --git a/__mocks__/@wdio/logger.ts b/__mocks__/@wdio/logger.ts index 7c18df102..33d5a0035 100644 --- a/__mocks__/@wdio/logger.ts +++ b/__mocks__/@wdio/logger.ts @@ -1,5 +1,7 @@ import { vi } from 'vitest' +export const SENSITIVE_DATA_REPLACER = '**MASKED**' + export const logMock = { error: vi.fn(), debug: vi.fn(), @@ -11,6 +13,8 @@ export const logMock = { const mock = () => logMock mock.setLevel = vi.fn() mock.setLogLevelsConfig = vi.fn() +mock.setMaskingPatterns = vi.fn() mock.waitForBuffer = vi.fn() mock.clearLogger = vi.fn() + export default mock diff --git a/e2e/wdio/headless/test.e2e.ts b/e2e/wdio/headless/test.e2e.ts index f0a0cfcea..dda31b56c 100644 --- a/e2e/wdio/headless/test.e2e.ts +++ b/e2e/wdio/headless/test.e2e.ts @@ -6,6 +6,7 @@ import path from 'node:path' import { browser, $, expect } from '@wdio/globals' import { imageSize } from 'image-size' +import type { InputOptions } from 'webdriverio' const __dirname = path.dirname(url.fileURLToPath(import.meta.url)) @@ -15,6 +16,17 @@ describe('main suite 1', () => { await expect($('.findme')).toMatchSnapshot() await expect($('.findme')).toMatchInlineSnapshot('"

Test CSS Attributes

"') }) + + it('should support input value with sensitive information', async () => { + await browser.url('https://guinea-pig.webdriver.io/') + + const firstInput = await $('input') + await firstInput.setValue('mySecretPassword', { mask: true } satisfies InputOptions) + + // Note: Doing the below will expose the password in the logs, check to support this command one day! + const inputValue = await firstInput.getValue() + expect(inputValue).toBe('mySecretPassword') + }) it.skip('should allow to check for PWA', async () => { await browser.url('https://webdriver.io') diff --git a/e2e/wdio/wdio.conf.ts b/e2e/wdio/wdio.conf.ts index fc4c59c7f..3d1ae1b5d 100644 --- a/e2e/wdio/wdio.conf.ts +++ b/e2e/wdio/wdio.conf.ts @@ -32,6 +32,7 @@ export const config: WebdriverIO.Config = { * test configurations */ logLevel: 'info', + //maskingPatterns: '/--port=([^ ]*)/', // Uncomment to test masking in logs framework: 'mocha', outputDir: __dirname, diff --git a/packages/wdio-logger/README.md b/packages/wdio-logger/README.md index b6ce9656e..1f41fdd66 100644 --- a/packages/wdio-logger/README.md +++ b/packages/wdio-logger/README.md @@ -70,3 +70,51 @@ const uploadInterval = setInterval(() => { } }, 100); ``` + +## Masking Patterns + +For more secure logging, `setMaskingPatterns`, `WDIO_LOG_MASKING_PATTERNS` or `maskingPatterns` can obfuscate sensitive information from the log. +For example, we can replace `--key=MySecretKey` with `--key=**MASKED**` to hide your cloud service access key or secret key + - The regular expression pattern must be provided as a string similar as a RegEx but as string type, for example, `--key=[^ ]*` + - It support flags and capturing groups like `/--key=([^ ]*)/i` + - Multiple patterns are separated by a comma, like `--key=([^ ]*),secrets=([^ ]*)` + - If no capturing group is provided, the entire matching string of the pattern is masked + - If one or more capturing groups are provided, we replace all the matching groups with `**MASKED**` + - If there are multiple matches for a single group, we replace them all, too + - Support both masking in a file and the console + - Note: In the console, when masking, some colors get stripped, which is a known limitation + +`setMaskingPatterns` example + ```javascript +import logger from '@wdio/logger'; + +// Default for all loggers +logger.setMaskingPatterns('/--key=([^ ]*)/i,/--secrets=([^ ]*)/i') + +// For a specific logger +logger.setMaskingPatterns({'internal' : '/--key=([^ ]*)/i,/--secrets=([^ ]*)/i'}) +const log = logger('internal'); +``` + +Using wdio config from a `conf.ts` file, we can also configure masking patterns +```javascript +export const config: WebdriverIO.Config = { + /** + * test configurations + */ + logLevel: 'debug', + maskingPatterns: '/--key=([^ ]*)/i,/--secrets=([^ ]*)/i', +} +``` + +Below are examples with the environment variable `WDIO_LOG_MASKING_PATTERNS` in the code directly: + +```javascript +// Using environment variable in code +process.env.WDIO_LOG_MASKING_PATTERNS = '/--key=([^ ]*)/i,/--secrets=([^ ]*)/i' +``` + +Or before your command line +```shell +WDIO_LOG_MASKING_PATTERNS='RESULT ([^ ]*)' npx wdio run ./wdio/wdio.conf.ts +``` diff --git a/packages/wdio-logger/src/browser.ts b/packages/wdio-logger/src/browser.ts index 9f900573b..9c07ee19a 100644 --- a/packages/wdio-logger/src/browser.ts +++ b/packages/wdio-logger/src/browser.ts @@ -4,6 +4,8 @@ const LOG_METHODS = ['error', 'warn', 'info', 'debug', 'trace', 'silent'] export type { Logger } from './index.js' +export const SENSITIVE_DATA_REPLACER = '**MASKED**' + /** * This implementation of the Logger package is a simple adptation to run it within * a browser environment. @@ -25,5 +27,6 @@ export default function getLogger (component: string) { // logging interface expects a 'setLevel' method getLogger.setLevel = () => {} getLogger.setLogLevelsConfig = () => {} +getLogger.setMaskingPatterns = () => {} getLogger.waitForBuffer = () => {} getLogger.clearLogger = () => {} diff --git a/packages/wdio-logger/src/index.ts b/packages/wdio-logger/src/index.ts index ba5d14a18..54691fd06 100644 --- a/packages/wdio-logger/src/index.ts +++ b/packages/wdio-logger/src/index.ts @@ -1,3 +1,5 @@ +export { SENSITIVE_DATA_REPLACER } from './utils.js' + import fs from 'node:fs' import util from 'node:util' @@ -6,6 +8,7 @@ import type { ColorName } from 'chalk' import chalk from 'chalk' import prefix from 'loglevel-plugin-prefix' import ansiStrip from 'strip-ansi' +import { mask, parseMaskingPatterns } from './utils.js' prefix.reg(log) @@ -33,29 +36,30 @@ const SERIALIZERS = [{ /** * display error stack */ - matches: (err: unknown) => err instanceof Error, + matches: (err: unknown): err is Error => err instanceof Error, serialize: (err: Error) => err.stack }, { /** * color commands blue */ - matches: (log: string) => log === matches.COMMAND || log === matches.BIDICOMMAND, + matches: (log: unknown): log is string => log === matches.COMMAND || log === matches.BIDICOMMAND, serialize: (log: string) => chalk.magenta(log) }, { /** * color data yellow */ - matches: (log: string) => log === matches.DATA, + matches: (log: unknown): log is string => log === matches.DATA, serialize: (log: string) => chalk.yellow(log) }, { /** * color result cyan */ - matches: (log: string) => log === matches.RESULT || log === matches.BIDIRESULT, + matches: (log: unknown): log is string => log === matches.RESULT || log === matches.BIDIRESULT, serialize: (log: string) => chalk.cyan(log) }] interface LoggerInterface extends log.Logger { + maskingPatterns: RegExp[] | undefined progress(...msg: string[]): void; } @@ -65,13 +69,14 @@ interface Loggers { const loggers: Loggers = log.getLoggers() as Loggers let logLevelsConfig: Record = {} +let maskingPatternsConfig: Record = {} const logCache = new Set() let logFile: fs.WriteStream | null const originalFactory = log.methodFactory -const wdioLoggerMethodFactory = function (this: log.Logger, methodName: log.LogLevelNames, logLevel: log.LogLevelNumbers, loggerName: string) { +const wdioLoggerMethodFactory = (wdioLogger: LoggerInterface) => function (this: log.Logger, methodName: log.LogLevelNames, logLevel: log.LogLevelNumbers, loggerName: string) { const rawMethod = originalFactory(methodName, logLevel, loggerName) - return (...args: string[]) => { + return (...args: [string, string?, ...unknown[]]) => { /** * create logFile lazily */ @@ -80,25 +85,27 @@ const wdioLoggerMethodFactory = function (this: log.Logger, methodName: log.LogL } /** - * split `prefixer: value` sting to `prefixer: ` and `value` + * split `prefixer: value` string to `prefixer: ` and `value` * so that SERIALIZERS can match certain string */ const match = Object.values(matches).filter(x => args[0].endsWith(`: ${x}`))[0] if (match) { - const prefixStr = args.shift()!.slice(0, -match.length - 1) + const prefixStr = (args.shift() as string[]).slice(0, -match.length - 1) args.unshift(prefixStr, match) } args = args.map((arg) => { for (const s of SERIALIZERS) { if (s.matches(arg)) { - return s.serialize(arg as Error & string) as string + return s.serialize(arg as Error & string) } } return arg - }) + }) as [string, string?, ...unknown[]] + + const unmaskedLogText = ansiStrip(`${util.format.apply(this, args as [format: string, ...params: string[]])}\n`) + const maskedLogText = mask(unmaskedLogText, wdioLogger.maskingPatterns) - const logText = ansiStrip(`${util.format.apply(this, args as [format: string, ...params: string[]])}\n`) if (logFile && logFile.writable) { /** * empty logging cache if stuff got logged before @@ -112,15 +119,21 @@ const wdioLoggerMethodFactory = function (this: log.Logger, methodName: log.LogL logCache.clear() } - if (!logsContainInitPackageError(logText)) { - return logFile.write(logText) + if (!logsContainInitPackageError(unmaskedLogText)) { + return logFile.write(maskedLogText) } // If we get Error during init of integration packages, write logs to both "outputDir" and the terminal - logFile.write(logText) + logFile.write(maskedLogText) } - logCache.add(logText) - rawMethod(...args) + logCache.add(maskedLogText) + + // To not break console color formatting, we use the masked log text only when needed + if (maskedLogText === unmaskedLogText) { + rawMethod(...args) + } else { + rawMethod(maskedLogText.replace(/\n$/, '')) + } } } @@ -151,16 +164,21 @@ export default function getLogger (name: string) { } loggers[name] = log.getLogger(name) as LoggerInterface - loggers[name].setLevel(logLevel) - loggers[name].methodFactory = wdioLoggerMethodFactory - loggers[name].progress = progress - prefix.apply(loggers[name], { + const logger = loggers[name] + + logger.setLevel(logLevel) + logger.maskingPatterns = maskingPatternsConfig[name] ?? parseMaskingPatterns(process.env.WDIO_LOG_MASKING_PATTERNS) + logger.progress = progress + + logger.methodFactory = wdioLoggerMethodFactory(logger) + + prefix.apply(logger, { template: '%t %l %n:', timestampFormatter: (date) => chalk.gray(date.toISOString()), levelFormatter: (level: string) => chalk[COLORS[level]](level.toUpperCase()), nameFormatter: (name) => chalk.whiteBright(name) }) - return loggers[name] + return logger } /** * Wait for writable stream to be flushed. @@ -215,6 +233,42 @@ getLogger.setLogLevelsConfig = (logLevels: Record = {} loggers[logName].setLevel(logLevel) }) } + +type MaskPattern = string +type MaskPatternForSpecificLogger = Record +getLogger.setMaskingPatterns = (pattern: MaskPattern | MaskPatternForSpecificLogger) => { + + if ( typeof pattern === 'string') { + /** + * set the environment variable for masking patterns + */ + if (process.env.WDIO_LOG_MASKING_PATTERNS === undefined) { + process.env.WDIO_LOG_MASKING_PATTERNS = pattern + } + } else if ( typeof pattern === 'object') { + /** + * build maskingPatternsConfig object + */ + maskingPatternsConfig = Object.entries(pattern).reduce((acc, [logName, maskingPatternsString]) => { + acc[logName] = parseMaskingPatterns(maskingPatternsString) + return acc + }, maskingPatternsConfig) + } else { + throw new Error(`Invalid pattern property, expected \`string\` or \`Record\` but received \`${typeof pattern}\``) + } + + /** + * set masking patterns for each logger + */ + Object.keys(loggers).forEach(logName => { + /** + * either apply maskingPatterns or use the global one + */ + const maskingPatterns = maskingPatternsConfig[logName] ?? parseMaskingPatterns(process.env.WDIO_LOG_MASKING_PATTERNS) + + loggers[logName].maskingPatterns = maskingPatterns + }) +} const getLogLevelName = (logName: string) => logName.split(':').shift() as log.LogLevelDesc export type Logger = LoggerInterface diff --git a/packages/wdio-logger/src/utils.ts b/packages/wdio-logger/src/utils.ts new file mode 100644 index 000000000..e4f01a8bf --- /dev/null +++ b/packages/wdio-logger/src/utils.ts @@ -0,0 +1,75 @@ +export const SENSITIVE_DATA_REPLACER = '**MASKED**' + +const skipError = (aFunction: Function) => { + try { + return aFunction() + } catch { + return undefined + } +} + +/** + * Parses a comma-separated string of regular expressions into an array of RegExp objects. + * Supports both `/pattern/flags` and plain pattern formats. + * + * @param {string | undefined} maskingRegexString - The string containing regex patterns. + * @returns {(RegExp[] | undefined)} Array of RegExp objects or undefined if input is invalid. + */ +export const parseMaskingPatterns = (maskingRegexString: string | undefined) => { + if (typeof maskingRegexString !== 'string') { return undefined } + const regexStrings = maskingRegexString?.split(/,\s*/).filter((regexStr) => regexStr.trim() !== '') + + return regexStrings?.map((regexStr) => { + const regexParts = regexStr.match(/^\/(.*?)\/([gimsuy]*)$/) + if (!regexParts) { + // When passing only a simple string without `/` or flags, aka `(--key=)([^ ]*)` + return skipError(() => new RegExp(regexStr)) + } else if (regexParts?.[1]) { + // Case with flag `/(--key=)([^ ]*)/i` or without flag `/(--key=)([^ ]*)/` + return skipError(() => regexParts[2] ? new RegExp(regexParts[1], regexParts[2]) : new RegExp(regexParts[1])) + } + return undefined + + }).filter((regex) => regex !== undefined) +} + +/** + * Masks sensitive data in a string using the provided masking patterns. + * + * - If a pattern has no capturing groups, the whole match is replaced with the mask. + * - If a pattern has capturing groups, each group is replaced with the mask, preserving the rest of the match. + * + * @param {string} text - The text to mask. + * @param {RegExp[] | undefined} maskingPatterns - Array of RegExp patterns to use for masking. + * @returns {string} The masked text, or the original value if not a string or if no patterns are provided. + */ +export const mask = (text: string, maskingPatterns: RegExp[] | undefined) => { + if (!maskingPatterns || typeof text !== 'string') { return text } + + const endsWithNewline = text.endsWith('\n') + let maskedText = text + maskingPatterns.forEach((maskingRegex) => { + + // Replace the whole match when no capturing groups or each capturing group + maskedText = maskedText.replace(maskingRegex, (fullMatch, ...capturedGroupsAndMore) => { + const capturedGroups = capturedGroupsAndMore.slice(0, capturedGroupsAndMore.length - 2) + + if (capturedGroups.length === 0) { + return SENSITIVE_DATA_REPLACER + } + + let matchedMaskedText = fullMatch + capturedGroups.forEach((group) => { + matchedMaskedText = matchedMaskedText.replace(group, SENSITIVE_DATA_REPLACER ) + }) + return matchedMaskedText + }) + }) + + // Ensure a trailing newline is preserved if present in the original text + if (endsWithNewline && !maskedText.endsWith('\n')) { + maskedText += '\n' + } + + return maskedText +} diff --git a/packages/wdio-logger/tests/node.test.ts b/packages/wdio-logger/tests/node.test.ts index 60cdec139..7ee173c78 100644 --- a/packages/wdio-logger/tests/node.test.ts +++ b/packages/wdio-logger/tests/node.test.ts @@ -1,5 +1,5 @@ import fs from 'node:fs' -import { it, describe, expect, vi, afterEach, beforeEach, afterAll } from 'vitest' +import { it, describe, expect, vi, afterEach, beforeAll, beforeEach, afterAll } from 'vitest' import nodeLogger from '../src/index.js' import nodeLogger2 from '../build/index.js' @@ -221,6 +221,141 @@ describe('wdio-logger node', () => { expect(write.mock.results[3].value).toContain('test-logFile4: Error: bar') }) + describe('making patterns', () => { + afterEach(() => { + delete process.env.WDIO_LOG_MASKING_PATTERNS + }) + + describe('given a WDIO_LOG_MASKING_PATTERNS value', () => { + it('masked sensitive information with one pattern', () => { + process.env.WDIO_LOG_PATH = 'wdio.test.log' + process.env.WDIO_LOG_MASKING_PATTERNS = '--key=[^ ]*' + + const log = nodeLogger('test-logFile-maskedLogPatternNoFlag') + log.info('wdio.conf.ts --user= --key=mySecretKey --spec template.test.ts') + + expect(write.mock.results[0].value).toContain('wdio.conf.ts --user= **MASKED** --spec template.test.ts') + }) + + it('masked sensitive information and keep ending new line even when capturing the whole line', () => { + process.env.WDIO_LOG_PATH = 'wdio.test.log' + process.env.WDIO_LOG_MASKING_PATTERNS = '/RESULT ([^ ]*)/' + + const log = nodeLogger('test-logFile-maskedEverythingButKeepNewLine') + log.info('RESULT test') + + expect(write.mock.results[0].value).toContain('**MASKED**\n') + }) + + it('masked sensitive information with one pattern having 0 group and global flag', () => { + process.env.WDIO_LOG_PATH = 'wdio.test.log' + process.env.WDIO_LOG_MASKING_PATTERNS = '/--key=[^ ]*/gi' + + const log = nodeLogger('test-logFile-maskedLogFile0Group') + log.info('wdio.conf.ts --user= --KEY=mySecretKey --key=mySecretKey --spec template.test.ts') + + expect(write.mock.results[0].value).toContain('wdio.conf.ts --user= **MASKED** **MASKED** --spec template.test.ts') + }) + + it('masked sensitive information with one pattern having 1 group', () => { + process.env.WDIO_LOG_PATH = 'wdio.test.log' + process.env.WDIO_LOG_MASKING_PATTERNS = '--key=([^ ]*)' + + const log = nodeLogger('test-logFile-maskedLogFile2Group') + log.info('wdio.conf.ts --user= --key=mySecretKey --spec template.test.ts') + + expect(write.mock.results[0].value).toContain('wdio.conf.ts --user= --key=**MASKED** --spec template.test.ts') + }) + + it('masked sensitive information with two patterns having 1 groups each', () => { + process.env.WDIO_LOG_PATH = 'wdio.test.log' + process.env.WDIO_LOG_MASKING_PATTERNS = '--key=([^ ]*),/TOKEN=([^ ]*)/i' + + const log = nodeLogger('test-logFile-masked2RegExHaving2Group') + log.info('TOKEN=mySecretToken wdio.conf.ts --user=myUser --key=mySecretKey --spec template.test.ts') + + expect(write.mock.results[0].value).toContain('TOKEN=**MASKED** wdio.conf.ts --user=myUser --key=**MASKED** --spec template.test.ts') + }) + + it('masked sensitive information when logging arrays of string', () => { + process.env.WDIO_LOG_PATH = 'wdio.test.log' + process.env.WDIO_LOG_MASKING_PATTERNS = 'COMMAND ([^ ]*)' + + const log = nodeLogger('test-logFile-maskedLoggingOfStringArrays') + log.info('COMMAND', 'mySecretKey') + + expect(write.mock.results[0].value).toContain('COMMAND **MASKED**') + }) + + describe('given call to getLogger.setMaskingPatterns', () => { + const pattern = '(--key=)([^ ]*)' + const expectedPatterns = [/(--key=)([^ ]*)/] + + const scenarios: { + name: string + logger: string + config: Record + expectedPatterns: RegExp[] | undefined + }[] = [{ + name: 'should be possible to set masking pattern in config', + logger: 'scenarios-setMaskingPatterns-3', + config: { 'scenarios-setMaskingPatterns-3': pattern }, + expectedPatterns + }, { + name: 'should be possible to override WDIO_LOG_MASKING_PATTERNS in config', + logger: 'scenarios-setMaskingPatterns-5', + get config() { + process.env.WDIO_LOG_MASKING_PATTERNS = 'info' + return { 'scenarios-setMaskingPatterns-5': pattern } + }, + expectedPatterns + }, { + name: 'should be possible to set multiple patterns in config for one logger', + logger: 'scenarios-setMaskingPatterns-6', + get config() { + return { 'scenarios-setMaskingPatterns-6': `${pattern},info` } + }, + expectedPatterns: [expectedPatterns[0], /info/] + }] + + scenarios.forEach((scenario) => { + it(scenario.name, () => { + nodeLogger.setMaskingPatterns(scenario.config) + + const log = nodeLogger(scenario.logger) + expect(log.maskingPatterns).toEqual(scenario.expectedPatterns) + }) + }) + + it('should be possible to set patterns in config for multiple loggers', () => { + const config = { 'test-logFile-setMaskingPatterns-1': pattern, 'test-logFile-setMaskingPatterns-2': 'info' } + + nodeLogger.setMaskingPatterns(config) + + expect(nodeLogger('test-logFile-setMaskingPatterns-1').maskingPatterns).toEqual(expectedPatterns) + expect(nodeLogger('test-logFile-setMaskingPatterns-2').maskingPatterns).toEqual([/info/]) + }) + + it('should be possible to pass a default using `setMaskingPatterns`', () => { + nodeLogger.setMaskingPatterns(pattern) + + expect(nodeLogger('test-logFile-setMaskingPatterns-3').maskingPatterns).toEqual(expectedPatterns) + }) + it('should be possible to pass a default using `WDIO_LOG_MASKING_PATTERNS` env', () => { + process.env.WDIO_LOG_MASKING_PATTERNS = 'myEnvPattern' + + expect(nodeLogger('test-logFile-setMaskingPatterns-4').maskingPatterns).toEqual([/myEnvPattern/]) + }) + it('should throw when wrong type', () => { + process.env.WDIO_LOG_MASKING_PATTERNS = 'myEnvPattern' + + expect(() => nodeLogger.setMaskingPatterns(undefined as unknown as string)).throw('Invalid pattern property, expected `string` or `Record` but received `undefined`') + + }) + }) + }) + }) + it('is not confused by multiple copies of source code', () => { process.env.WDIO_LOG_PATH = 'wdio.test.log' @@ -298,9 +433,75 @@ describe('wdio-logger node', () => { writableBuffer = undefined }) afterAll(() => { + delete process.env.WDIO_LOG_PATH logInfoSpy.mockRestore() + nodeLogger.clearLogger() + nodeLogger2.clearLogger() }) }) + + describe('given logs outputted in the console', () => { + + describe('given WDIO_LOG_MASKING_PATTERNS', () => { + const consoleInfoSpy = vi.spyOn(console, 'info') + beforeAll(() => { + delete process.env.WDIO_LOG_MASKING_PATTERNS + consoleInfoSpy.mockClear() + }) + afterEach(() => { + delete process.env.WDIO_LOG_MASKING_PATTERNS + consoleInfoSpy.mockClear() + }) + + describe('given WDIO_LOG_MASKING_PATTERNS is undefined', () => { + it('does not mask sensitive information', () => { + const log = nodeLogger('test-console-noMasking') + log.info('wdio.conf.ts --key=mySecretKey --spec template.test.ts') + + expect(consoleInfoSpy).toHaveBeenCalledWith(expect.stringContaining('cyanBright INFO whiteBright test-console-noMasking: wdio.conf.ts --key=mySecretKey --spec template.test.ts')) + }) + + it('logs info with array or arguments for RESULT matching string with cyan color', () => { + const log = nodeLogger('test-console-noMasking-MatchingStringWithColor') + log.info('RESULT', 'mySecretKey') + + expect(consoleInfoSpy).toHaveBeenCalledWith(expect.stringContaining('cyanBright INFO whiteBright test-console-noMasking-MatchingStringWithColor:'), 'cyan RESULT', 'mySecretKey') + }) + }) + + describe('given WDIO_LOG_MASKING_PATTERNS is defined', () => { + it('masked sensitive information with one pattern', () => { + process.env.WDIO_LOG_MASKING_PATTERNS = '--key=([^ ]*)' + + const log = nodeLogger('test-console-masked1Pattern') + log.info('wdio.conf.ts --key=mySecretKey --spec template.test.ts') + + expect(consoleInfoSpy).toHaveBeenCalledWith(expect.stringContaining('wdio.conf.ts --key=**MASKED** --spec template.test.ts')) + }) + + it('masked sensitive information and keep date, formatting but colors does not work and ensure no trailing new line', () => { + process.env.WDIO_LOG_MASKING_PATTERNS = '--key=([^ ]*)' + + const log = nodeLogger('test-console-masked1Pattern') + log.info('--key=mySecretKey') + + expect.stringMatching( + /^gray \d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}\.\d{3}Z cyanBright INFO whiteBright test-console-masked1Pattern: --key=\*\*MASKED\*\*$/ + ) + }) + + it('logs masked sensitive information without color as an entire string instead of an args array with color', () => { + process.env.WDIO_LOG_MASKING_PATTERNS = 'RESULT ([^ ]*)' + + const log = nodeLogger('test-console-maskedLoggingOfStringArrays') + log.info('RESULT', 'mySecretKey') + + expect(consoleInfoSpy).toHaveBeenCalledWith(expect.stringContaining('cyan RESULT **MASKED**')) + }) + }) + }) + }) + describe('waitForBuffer with no logFile', () => { it('should be ok if logFile is undefined', async () => { expect(await nodeLogger.waitForBuffer()).toBe(undefined) diff --git a/packages/wdio-logger/tests/utils.test.ts b/packages/wdio-logger/tests/utils.test.ts new file mode 100644 index 000000000..2745d8b72 --- /dev/null +++ b/packages/wdio-logger/tests/utils.test.ts @@ -0,0 +1,232 @@ +import { it, describe, expect } from 'vitest' +import { mask, parseMaskingPatterns } from '../src/utils.js' + +describe('wdio-logger utils', () => { + describe('parseMaskingPatterns', () => { + + it('should works when undefined', () => { + const patterns = parseMaskingPatterns(undefined) + + expect(patterns).toEqual(undefined) + }) + + it('should skip when pattern is empty', () => { + const patterns = parseMaskingPatterns('') + + expect(patterns).toEqual([]) + }) + + it('should works when pattern has space', () => { + const patterns = parseMaskingPatterns(' ') + + expect(patterns).toEqual([]) + }) + + it('should parse one pattern with no flag', () => { + const stringPatterns = '/--key=[^ ]*/' + + const patterns = parseMaskingPatterns(stringPatterns) + + expect(patterns).toEqual([/--key=[^ ]*/]) + }) + + it('should parse one pattern no flag and no slash', () => { + const stringPatterns = '--key=[^ ]*' + + const patterns = parseMaskingPatterns(stringPatterns) + + expect(patterns).toEqual([/--key=[^ ]*/]) + }) + + it('should parse one pattern having 0 group and case insensitive', () => { + const stringPatterns = '/--key=[^ ]*/i' + + const patterns = parseMaskingPatterns(stringPatterns) + + expect(patterns).toEqual([/--key=[^ ]*/i]) + }) + + it('should parse one pattern having 0 group and global flag', () => { + const stringPatterns = '/--key=[^ ]*/g' + + const patterns = parseMaskingPatterns(stringPatterns) + + expect(patterns).toEqual([/--key=[^ ]*/g]) + }) + + it('should parse one pattern ex having 1 group', () => { + const stringPatterns = '/--key=([^ ]*)/g' + + const patterns = parseMaskingPatterns(stringPatterns) + + expect(patterns).toEqual([/--key=([^ ]*)/g]) + }) + + it('should parse one pattern having 2 group', () => { + const stringPatterns = '(--key=)([^ ]*)' + + const patterns = parseMaskingPatterns(stringPatterns) + + expect(patterns).toEqual([/(--key=)([^ ]*)/]) + }) + + it('should "parse" one pattern having more than 2 group', () => { + const stringPatterns = '(--key)(=)([^ ]*)' + + const patterns = parseMaskingPatterns(stringPatterns) + + expect(patterns).toEqual([/(--key)(=)([^ ]*)/]) + }) + + it('should parse two patterns having 2 groups each', () => { + const stringPatterns = '(--key=)([^ ]*),/(TOKEN=)([^ ]*)/i' + + const patterns = parseMaskingPatterns(stringPatterns) + + expect(patterns).toEqual([/(--key=)([^ ]*)/, /(TOKEN=)([^ ]*)/i]) + }) + + it('skipped regex when it is invalid', () => { + const stringPatterns = '(--key=)([^ ]*,/(TOKEN=)([^ ]*)/i' + + const patterns = parseMaskingPatterns(stringPatterns) + + expect(patterns).toEqual([/(TOKEN=)([^ ]*)/i]) + }) + + it('should support human error with comma-space separation', () => { + const stringPatterns = '(--key=)([^ ]*), /(TOKEN=)([^ ]*)/i' + + const patterns = parseMaskingPatterns(stringPatterns) + + expect(patterns).toEqual([/(--key=)([^ ]*)/, /(TOKEN=)([^ ]*)/i]) + }) + + it('empty when all invalid', () => { + const stringPatterns = '(--key=)([^ ]*' + + const patterns = parseMaskingPatterns(stringPatterns) + + expect(patterns).toEqual([]) + }) + + it('undefined when wrong type', () => { + const stringPatterns = [] as unknown as string + + const patterns = parseMaskingPatterns(stringPatterns) + + expect(patterns).toEqual(undefined) + }) + + it('empty when invalid empty pattern string ', () => { + const stringPatterns = '//' + + const patterns = parseMaskingPatterns(stringPatterns) + + expect(patterns).toEqual([]) + }) + + }) + + describe('mask', () => { + + it('should return the arg as it is when not a string', () => { + const arg = 123 + const patterns = [/--key=[^ ]*/] + + const maskedValue = mask(arg as unknown as string, patterns) + + expect(maskedValue).toEqual(arg) + }) + + it('should return the arg when undefined', () => { + const arg = undefined + const patterns = [/--key=[^ ]*/] + + const maskedValue = mask(arg as unknown as string, patterns) + + expect(maskedValue).toEqual(arg) + }) + + it('should not mask when nothing to mask', () => { + const arg = 'mask nothing' + const patterns = [/--key=[^ ]*/] + + const maskedValue = mask(arg, patterns) + + expect(maskedValue).toEqual(arg) + }) + + it('should mask when there is something to mask', () => { + const arg = '--key=mySecretKey' + const patterns = [/--key=[^ ]*/] + + const maskedValue = mask(arg, patterns) + + expect(maskedValue).toEqual('**MASKED**') + }) + + it('should mask and keep the rest of the string', () => { + const arg = 'before --key=mySecretKey after' + const patterns = [/--key=[^ ]*/] + + const maskedValue = mask(arg, patterns) + + expect(maskedValue).toEqual('before **MASKED** after') + }) + + it('should mask the captured group only when having one capturing group', () => { + const arg = 'before --key=mySecretKey after' + const patterns = [/--key=([^ ]*)/] + + const maskedValue = mask(arg, patterns) + + expect(maskedValue).toEqual('before --key=**MASKED** after') + }) + + it('should mask all captured groups when having 2 groups', () => { + const arg = 'before --key=mySecretKey after' + const patterns = [/(--key)=([^ ]*)/] + + const maskedValue = mask(arg, patterns) + + expect(maskedValue).toEqual('before **MASKED**=**MASKED** after') + }) + + it('should also work with global flag', () => { + const arg = 'before --key=mySecretKey after' + const patterns = [/--key=([^ ]*)/g] + + const maskedValue = mask(arg, patterns) + + expect(maskedValue).toEqual('before --key=**MASKED** after') + }) + + it('should masked multiple occurrences with the global flag', () => { + const arg = 'before --key=mySecretKey1 --key=mySecretKey2 after' + const patterns = [/--key=([^ ]*)/g] + + const maskedValue = mask(arg, patterns) + + expect(maskedValue).toEqual('before --key=**MASKED** --key=**MASKED** after') + }) + + it('should masked properly when having case insensitive flag', () => { + const arg = 'before --KEY=mySecretKey1 --key=mySecretKey2 after' + const patterns = [/--key=([^ ]*)/ig] + + const maskedValue = mask(arg, patterns) + + expect(maskedValue).toEqual('before --KEY=**MASKED** --key=**MASKED** after') + }) + + it('should keep trailing new line if one existed', () => { + const arg = 'RESULT test\n' + const patterns = [/RESULT ([^ ]*)/] + + const maskedValue = mask(arg, patterns) + + expect(maskedValue).toEqual('RESULT **MASKED**\n') + }) + }) +}) \ No newline at end of file diff --git a/packages/wdio-runner/src/index.ts b/packages/wdio-runner/src/index.ts index bd8d5399e..a3b15c5ba 100644 --- a/packages/wdio-runner/src/index.ts +++ b/packages/wdio-runner/src/index.ts @@ -62,7 +62,12 @@ export default class Runner extends EventEmitter { this._config = this._configParser.getConfig() this._specFileRetryAttempts = (this._config.specFileRetries || 0) - (retries || 0) + logger.setLogLevelsConfig(this._config.logLevels, this._config.logLevel) + if (this._config.maskingPatterns) { + logger.setMaskingPatterns(this._config.maskingPatterns) + } + const capabilities = this._configParser.getCapabilities() const isMultiremote = this._isMultiremote = !Array.isArray(capabilities) || (Object.values(caps).length > 0 && Object.values(caps).every(c => typeof c === 'object' && c.capabilities)) diff --git a/packages/wdio-types/src/Options.ts b/packages/wdio-types/src/Options.ts index f24c62d7e..1b2a18ad2 100644 --- a/packages/wdio-types/src/Options.ts +++ b/packages/wdio-types/src/Options.ts @@ -49,7 +49,7 @@ export interface Connection { */ path?: string /** - * Query paramaters that are propagated to the driver server. + * Query parameters that are propagated to the driver server. */ queryParams?: { [name: string]: string @@ -139,6 +139,14 @@ export interface WebDriver extends Connection { * when attempting to start a session. */ cacheDir?: string + + /** + * Mask sensitive data in logs by replacing matching string or all captured groups for the provided regular expressions as string + * It replaces the matched string or the capture groups with `**MASKED**` + * Useful for masking sensitive data like cloud provider credentials for example with '/--key=([^ ]*)/' + * Use comma separated strings to use multiple patterns. + */ + maskingPatterns?: string } export type SauceRegions = 'us' | 'eu' | 'us-west-1' | 'us-east-4' | 'eu-central-1' | 'staging' diff --git a/packages/webdriver/src/command.ts b/packages/webdriver/src/command.ts index 0c0f24e6b..9ae4e142e 100644 --- a/packages/webdriver/src/command.ts +++ b/packages/webdriver/src/command.ts @@ -1,11 +1,14 @@ import logger from '@wdio/logger' -import { commandCallStructure, isValidParameter, getArgumentType } from '@wdio/utils' +import { commandCallStructure, isValidParameter, getArgumentType, transformCommandLogResult } from '@wdio/utils' import { WebDriverBidiProtocol, type CommandEndpoint } from '@wdio/protocols' import { environment } from './environment.js' import type { BidiHandler } from './bidi/handler.js' import type { WebDriverResponse } from './request/types.js' import type { BaseClient, BidiCommands, BidiResponses, WebDriverResultEvent } from './types.js' +import { CommandRuntimeOptions } from './types.js' +import { mask } from './utils.js' +import { APPIUM_MASKING_HEADER } from './constants.js' const log = logger('webdriver') const BIDI_COMMANDS: BidiCommands[] = Object.values(WebDriverBidiProtocol).map((def) => def.socket.command) @@ -19,7 +22,14 @@ export default function ( ) { const { command, deprecated, ref, parameters, variables = [], isHubCommand = false } = commandInfo - return async function protocolCommand (this: BaseClient, ...args: unknown[]): Promise { + return async function protocolCommand (this: BaseClient, ...unmaskedArgs: unknown[]): Promise { + + let runtimeOptions = {} + if (unmaskedArgs.length > 0 && unmaskedArgs[unmaskedArgs.length - 1] instanceof CommandRuntimeOptions ) { + // Popping the additional options to not have `Wrong parameters applied` thrown + runtimeOptions = unmaskedArgs.pop() as CommandRuntimeOptions + } + const isBidiCommand = BIDI_COMMANDS.includes(command as BidiCommands) let endpoint = endpointUri // clone endpointUri in case we change it const commandParams = [...variables.map((v) => Object.assign(v, { @@ -32,7 +42,6 @@ export default function ( const commandUsage = `${command}(${commandParams.map((p) => p.name).join(', ')})` const moreInfo = `\n\nFor more info see ${ref}\n` - const body: Record = {} /** * log deprecation warning if command is deprecated @@ -64,7 +73,7 @@ export default function ( * parameter check */ const minAllowedParams = commandParams.filter((param) => param.required).length - if (args.length < minAllowedParams || args.length > commandParams.length) { + if (unmaskedArgs.length < minAllowedParams || unmaskedArgs.length > commandParams.length) { const parameterDescription = commandParams.length ? `\n\nProperty Description:\n${commandParams.map((p) => ` "${p.name}" (${p.type}): ${p.description}`).join('\n')}` : '' @@ -80,7 +89,8 @@ export default function ( /** * parameter type check */ - for (const [it, arg] of Object.entries(args)) { + const unmaskedBody: Record = {} + for (const [it, arg] of Object.entries(unmaskedArgs)) { if (isBidiCommand) { break } @@ -118,9 +128,14 @@ export default function ( /** * rest of args are part of body payload */ - body[commandParams[i].name] = arg + unmaskedBody[commandParams[i].name] = arg } + /** + * Until this point, the body and args should not be logged or emitted in any way. Used the masked version to do so. + */ + const { maskedBody, maskedArgs, isMasked } = mask(commandInfo, runtimeOptions, unmaskedBody, unmaskedArgs) + /** * Make sure we pass along an abort signal to the request class so we * can abort the request as well as any retries in case the session is @@ -133,34 +148,41 @@ export default function ( const { isAborted, abortSignal, cleanup } = manageSessionAbortions.call(this) const requiresSession = endpointUri.includes('/:sessionId/') if (isAborted && command !== 'deleteSession' && requiresSession) { - throw new Error(`Trying to run command "${commandCallStructure(command, args)}" after session has been deleted, aborting request without executing it`) + throw new Error(`Trying to run command "${commandCallStructure(command, maskedArgs)}" after session has been deleted, aborting request without executing it`) } - const request = new environment.value.Request(method, endpoint, body, abortSignal, isHubCommand, { - onPerformance: (data) => this.emit('request.performance', data), - onRequest: (data) => this.emit('request.start', data), + const request = new environment.value.Request(method, endpoint, unmaskedBody, abortSignal, isHubCommand, { + onPerformance: (data) => this.emit('request.performance', { ...data, request: { + ...data.request, + body: isMasked ? maskedBody : data.request.body + } }), + onRequest: (data) => this.emit('request.start', { ...data, body: isMasked ? maskedBody : data.body }), onResponse: (data) => this.emit('request.end', data), - onRetry: (data) => this.emit('request.retry', data) + onRetry: (data) => this.emit('request.retry', data), + onLogData: (data) => log.info('DATA', transformCommandLogResult((isMasked ? maskedBody : data))) }) - this.emit('command', { command, method, endpoint, body }) - log.info('COMMAND', commandCallStructure(command, args)) + this.emit('command', { command, method, endpoint, body: maskedBody }) + log.info('COMMAND', commandCallStructure(command, maskedArgs)) + + const options = isMasked ? { ...this.options, headers: { ...this.options.headers, ...APPIUM_MASKING_HEADER } } : this.options + /** * use then here so we can better unit test what happens before and after the request */ - return request.makeRequest(this.options, this.sessionId).then((result) => { + return request.makeRequest(options, this.sessionId).then((result) => { if (typeof result.value !== 'undefined') { let resultLog = result.value if (/screenshot|recording/i.test(command) && typeof result.value === 'string' && result.value.length > 64) { resultLog = `${result.value.slice(0, 61)}...` - } else if (command === 'executeScript' && typeof body.script === 'string' && body.script.includes('(() => window.__wdioEvents__)')) { + } else if (command === 'executeScript' && typeof maskedBody.script === 'string' && maskedBody.script.includes('(() => window.__wdioEvents__)')) { resultLog = `[${(result.value as unknown[]).length} framework events captured]` } log.info('RESULT', resultLog) } - this.emit('result', { command, method, endpoint, body, result }) + this.emit('result', { command, method, endpoint, body: maskedBody, result }) if (command === 'deleteSession') { /** @@ -169,7 +191,7 @@ export default function ( const browser = this as { _bidiHandler?: BidiHandler } browser._bidiHandler?.close() - const shutdownDriver = (body.deleteSessionOpts as { shutdownDriver?: boolean })?.shutdownDriver !== false + const shutdownDriver = (maskedBody.deleteSessionOpts as { shutdownDriver?: boolean })?.shutdownDriver !== false /** * kill driver process if there is one */ @@ -207,7 +229,7 @@ export default function ( return result.value as WebDriverResponse | BidiResponses }).catch((error) => { - this.emit('result', { command, method, endpoint, body, result: { error } }) + this.emit('result', { command, method, endpoint, body: maskedBody, result: { error } }) throw error }).finally(() => { cleanup() diff --git a/packages/webdriver/src/constants.ts b/packages/webdriver/src/constants.ts index 126da88f8..589ddfdb0 100644 --- a/packages/webdriver/src/constants.ts +++ b/packages/webdriver/src/constants.ts @@ -141,6 +141,13 @@ export const DEFAULTS: Options.Definition> = { cacheDir: { type: 'string', default: environment.value.variables.WEBDRIVER_CACHE_DIR + }, + /** + * Mask sensitive data in logs by replacing matching string or all captured groups for the provided regular expressions as string + */ + maskingPatterns: { + type: 'string', + default : undefined, } } @@ -149,3 +156,5 @@ export const SHADOW_ELEMENT_KEY = 'shadow-6066-11e4-a52e-4f735466cecf' export const BASE_64_REGEX = /^([A-Za-z0-9+/]{4})*([A-Za-z0-9+/]{3}=|[A-Za-z0-9+/]{2}==)?$/ export const BASE_64_SAFE_STRING_TO_PROCESS_LENGTH = 200_000 + +export const APPIUM_MASKING_HEADER = { 'x-appium-is-sensitive': 'true' } diff --git a/packages/webdriver/src/request/request.ts b/packages/webdriver/src/request/request.ts index f6629b087..d5e996bd4 100644 --- a/packages/webdriver/src/request/request.ts +++ b/packages/webdriver/src/request/request.ts @@ -1,5 +1,5 @@ import logger from '@wdio/logger' -import { transformCommandLogResult, sleep } from '@wdio/utils' +import { sleep } from '@wdio/utils' import type { Options } from '@wdio/types' import { type RequestInit as UndiciRequestInit } from 'undici' @@ -165,7 +165,7 @@ export abstract class WebDriverRequest { log.info(`[${fullRequestOptions.method}] ${(url as URL).href}`) if (fullRequestOptions.body && Object.keys(fullRequestOptions.body).length) { - log.info('DATA', transformCommandLogResult(fullRequestOptions.body)) + this.eventHandler.onLogData?.(fullRequestOptions.body) } const { ...requestLibOptions } = fullRequestOptions diff --git a/packages/webdriver/src/request/types.ts b/packages/webdriver/src/request/types.ts index 61a984fbc..4e9d383a4 100644 --- a/packages/webdriver/src/request/types.ts +++ b/packages/webdriver/src/request/types.ts @@ -25,6 +25,7 @@ export interface RequestEventHandler { onResponse?: (ev: RequestEndEvent) => void onRetry?: (ev: RequestRetryEvent) => void onPerformance?: (ev: RequestPerformanceEvent) => void + onLogData?: (ev: BodyInit) => void } export type RequestStartEvent = RequestInit diff --git a/packages/webdriver/src/types.ts b/packages/webdriver/src/types.ts index 4476505d9..62cac8719 100644 --- a/packages/webdriver/src/types.ts +++ b/packages/webdriver/src/types.ts @@ -96,3 +96,16 @@ export interface AttachOptions extends Partial, Partial} body - The request body object to potentially mask. + * @param {unknown[]} args - The arguments array to potentially mask. + * @returns {{ + * maskedBody: Record, + * maskedArgs: unknown[], + * isMasked: boolean + * }} An object containing the (possibly) masked body and args, and a flag indicating if masking was applied. + */ +export function mask(commandInfo: CommandEndpoint, options: CommandRuntimeOptions, body: Record, args: unknown[]) { + const unmaskedResult = { maskedBody: body, maskedArgs: args, isMasked: false } + if (!options.mask) { + return unmaskedResult + } + + const textValueParamIndex = commandInfo.parameters.findIndex((param) => param.name === 'text') + if (textValueParamIndex === -1 ) { + return unmaskedResult + } + + const textValueIndexInArgs = (commandInfo.variables?.length ?? 0) + textValueParamIndex + const text = args[textValueIndexInArgs] + if (typeof text !== 'string' || !text) { + return unmaskedResult + } + + const maskedBody = { + ...body, + text: SENSITIVE_DATA_REPLACER + } satisfies Record as Record + + const textValueArgsIndex = textValueParamIndex + (commandInfo.variables?.length ?? 0) + const maskedArgs = args.slice(0, textValueArgsIndex).concat(SENSITIVE_DATA_REPLACER).concat(args.slice(textValueArgsIndex + 1)) + + return { + maskedBody, + maskedArgs, + isMasked: true, + } +} \ No newline at end of file diff --git a/packages/webdriver/tests/command.test.ts b/packages/webdriver/tests/command.test.ts index 08975777c..f71cf1889 100644 --- a/packages/webdriver/tests/command.test.ts +++ b/packages/webdriver/tests/command.test.ts @@ -1,5 +1,5 @@ import path from 'node:path' -import { EventEmitter, on } from 'node:events' +import { EventEmitter } from 'node:events' import { describe, it, expect, vi, beforeEach } from 'vitest' import logger from '@wdio/logger' @@ -8,9 +8,12 @@ import type { Options } from '@wdio/types' import '../src/browser.js' // @ts-expect-error mock feature -import { WebDriverRequest as RequestMock, thenMock, catchMock } from '../src/request/request.js' +import { WebDriverRequest as RequestMock, thenMock, catchMock, makeRequestMock, getCapturedRequestHandler } from '../src/request/request.js' import commandWrapper from '../src/command.js' import type { BaseClient } from '../src/types.js' +import { CommandRuntimeOptions } from '../src/types.js' +import type { RequestEventHandler } from '../build/request/types.js' +import type { RequestOptions } from '../src/request/types.js' vi.mock('@wdio/logger', () => import(path.join(process.cwd(), '__mocks__', '@wdio/logger'))) vi.mock('fetch') @@ -53,7 +56,8 @@ const requestHandler = { onPerformance: expect.any(Function), onRequest: expect.any(Function), onResponse: expect.any(Function), - onRetry: expect.any(Function) + onRetry: expect.any(Function), + onLogData: expect.any(Function) } vi.mock('../src/request/request', () => { @@ -62,17 +66,27 @@ vi.mock('../src/request/request', () => { const catchMock = vi.fn().mockReturnValue({ finally: finallyMock }) const promise = { then: thenMock, catch: catchMock, finally: finallyMock } - const WebDriverRequest = vi.fn().mockReturnValue({ - makeRequest: () => (promise), - on: vi.fn() + + const makeRequestMock = vi.fn().mockReturnValue(promise) + thenMock.mockReturnValue(promise) + + const capturedRequestEventHandler = vi.fn<() => RequestEventHandler>() + + // Mock the constructor + const WebDriverRequest = vi.fn().mockImplementation(( + _method, _endpoint, _body, _abortSignal, _isHubCommand, requestEventHandler: RequestEventHandler + ) => { + capturedRequestEventHandler.mockReturnValue(requestEventHandler) + return { makeRequest: makeRequestMock } }) - thenMock.mockReturnValue(promise) return { thenMock, catchMock, finallyMock, WebDriverRequest, + makeRequestMock, + getCapturedRequestHandler: capturedRequestEventHandler } }) @@ -94,8 +108,11 @@ class FakeClient extends EventEmitter { capabilities = {} requestedCapabilities = {} options = { - logLevel: 'warn' as Options.WebDriverLogTypes - } as any + logLevel: 'warn' as Options.WebDriverLogTypes, + headers: { + 'custom-header': 'custom-value' + }, + } as Partial emit = vi.fn() on = vi.fn() } @@ -106,10 +123,12 @@ type mockResponse = (...args: any[]) => any describe('command wrapper', () => { beforeEach(() => { vi.mocked(log.warn).mockClear() + vi.mocked(log.info).mockClear() vi.mocked(scope.emit).mockClear() vi.mocked(scope.on).mockClear() vi.mocked(thenMock).mockClear() vi.mocked(catchMock).mockClear() + vi.mocked(getCapturedRequestHandler).mockClear() }) it('should fail if wrong arguments are passed in', async () => { @@ -252,6 +271,140 @@ describe('command wrapper', () => { result: { error } }) }) + + describe('masking command', () => { + const maskCommandEndpoint: CommandEndpoint = { + command: 'elementSendKeys', + ref: 'https://w3c.github.io/webdriver/#dfn-element-send-keys', + description: '', + variables: [{ + name: 'elementId', + description: 'the id of an element returned in a previous call to Find Element(s)' + }], + parameters: [{ + name: 'text', + type: 'string', + description: 'the text to be sent to the element', + required: true + }] + } + const elementId = '123' + const text = 'mySecretPassword' + const runtimeOptionsWithMasking = new CommandRuntimeOptions({ mask: true }) + + it('should do the http request with the unmasked text when masking is requested', async () => { + const commandFn = commandWrapper(commandMethod, commandPath, maskCommandEndpoint) + + await commandFn.call(scope, elementId, text, runtimeOptionsWithMasking) + + expect(RequestMock).toHaveBeenCalledWith( + expect.any(String), + expect.any(String), + { + text: 'mySecretPassword', + }, + expect.any(AbortSignal), + false, + requestHandler + ) + }) + + it('should emit "result" with masked text on success', async () => { + const protocolCommandFunction = commandWrapper(commandMethod, commandPath, maskCommandEndpoint) + + const test = await protocolCommandFunction.call(scope, elementId, text, runtimeOptionsWithMasking) + vi.mocked(scope.emit).mockClear() + const thenCallback = thenMock.mock.calls[0][0] + + thenCallback({}) + + expect(scope.emit).toBeCalledWith('result', expect.objectContaining({ + body: { text: '**MASKED**' }, + })) + }) + + it('should emit "result" with masked text on error', async () => { + const protocolCommandFunction = commandWrapper(commandMethod, commandPath, maskCommandEndpoint) + + const test = await protocolCommandFunction.call(scope, elementId, text, runtimeOptionsWithMasking) + vi.mocked(scope.emit).mockClear() + const errorCallback = catchMock.mock.calls[0][0] + + const error = new Error('Request failed') + expect(() => errorCallback(error)).toThrow(error) + expect(scope.emit).toBeCalledWith('result', expect.objectContaining({ + body: { text: '**MASKED**' }, + })) + }) + + it('should emit "command" with masked text', async () => { + const protocolCommandFunction = commandWrapper(commandMethod, commandPath, maskCommandEndpoint) + + await protocolCommandFunction.call(scope, elementId, text, runtimeOptionsWithMasking) + + expect(scope.emit).toBeCalledWith('command', expect.objectContaining({ + body: { text: '**MASKED**' }, + })) + }) + + it('should emit "request.performance" with masked text when using the RequestEventHandler.onPerformance', async () => { + const protocolCommandFunction = commandWrapper(commandMethod, commandPath, maskCommandEndpoint) + + await protocolCommandFunction.call(scope, elementId, text, runtimeOptionsWithMasking) + vi.mocked(scope.emit).mockClear() + getCapturedRequestHandler().onPerformance({ data: 'data', request: { body: { text: 'mySecretPassword' } } }) + + expect(scope.emit).toBeCalledWith('request.performance', expect.objectContaining({ + request: { body: { text: '**MASKED**' } }, + })) + }) + + it('should emit "request.start" with masked text when using the RequestEventHandler.onRequest', async () => { + const protocolCommandFunction = commandWrapper(commandMethod, commandPath, maskCommandEndpoint) + + await protocolCommandFunction.call(scope, elementId, text, runtimeOptionsWithMasking) + vi.mocked(scope.emit).mockClear() + getCapturedRequestHandler().onRequest({ data: 'data', body: { text: 'mySecretPassword' } }) + + expect(scope.emit).toBeCalledWith('request.start', expect.objectContaining( + { body: { text: '**MASKED**' } } + )) + }) + + it('should log "DATA" with masked text when using the RequestEventHandle.onLogData', async () => { + const protocolCommandFunction = commandWrapper(commandMethod, commandPath, maskCommandEndpoint) + + await protocolCommandFunction.call(scope, elementId, text, runtimeOptionsWithMasking) + vi.mocked(scope.emit).mockClear() + getCapturedRequestHandler().onLogData({ text: 'mySecretPassword' }) + + expect(log.info).toBeCalledWith('DATA', { text: '**MASKED**' }) + }) + + it('should log "COMMAND" with masked text', async () => { + const protocolCommandFunction = commandWrapper(commandMethod, commandPath, maskCommandEndpoint) + + await protocolCommandFunction.call(scope, elementId, text, runtimeOptionsWithMasking) + + expect(log.info).toBeCalledWith('COMMAND', 'elementSendKeys("123", "**MASKED**")') + }) + + it('should makeRequest with the appium header for sensitive data', async () => { + const protocolCommandFunction = commandWrapper(commandMethod, commandPath, maskCommandEndpoint) + + await protocolCommandFunction.call(scope, elementId, text, runtimeOptionsWithMasking) + expect(makeRequestMock).toHaveBeenCalledWith(expect.objectContaining( + { headers: { + 'custom-header': 'custom-value', + 'x-appium-is-sensitive': 'true' + } } + ), expect.any(String)) + expect(scope.options.headers).toEqual({ + 'custom-header': 'custom-value', + }) + }) + + }) }) describe('command wrapper result log', () => { diff --git a/packages/webdriverio/src/commands/element/addValue.ts b/packages/webdriverio/src/commands/element/addValue.ts index 5ce26966b..a6c278938 100644 --- a/packages/webdriverio/src/commands/element/addValue.ts +++ b/packages/webdriverio/src/commands/element/addValue.ts @@ -1,3 +1,6 @@ +import type { InputOptions } from '../../types.js' +import { CommandRuntimeOptions } from 'webdriver' + const VALID_TYPES = ['string', 'number'] /** @@ -25,11 +28,13 @@ const VALID_TYPES = ['string', 'number'] * * @alias element.addValue * @param {string|number} value value to be added + * @param {InputOptions} additional options, exclusive to Webdriverio * */ export function addValue ( this: WebdriverIO.Element, - value: string | number + value: string | number, + options?: InputOptions ) { /** * The JSONWireProtocol allowed array values and use of special characters when adding a value to an input. @@ -43,5 +48,10 @@ export function addValue ( ) } + if (options) { + // @ts-ignore bypassing typing to pass the third parameter until we can find a better solution + return this.elementSendKeys(this.elementId, value.toString(), new CommandRuntimeOptions(options)) + } + // Have this call separated so at least one of the two code lines is typed checked return this.elementSendKeys(this.elementId, value.toString()) } diff --git a/packages/webdriverio/src/commands/element/setValue.ts b/packages/webdriverio/src/commands/element/setValue.ts index 7d43e00aa..bf9612236 100644 --- a/packages/webdriverio/src/commands/element/setValue.ts +++ b/packages/webdriverio/src/commands/element/setValue.ts @@ -1,3 +1,5 @@ +import type { InputOptions } from '../../types.js' + /** * Send a sequence of key strokes to an element after the input has been cleared before. If the element doesn't need * to be cleared first then use [`addValue`](/docs/api/element/addValue). @@ -22,12 +24,14 @@ * * @alias element.setValue * @param {string|number} value value to be added + * @param {InputOptions} additional options, exclusive to Webdriverio * */ export async function setValue ( this: WebdriverIO.Element, - value: string | number + value: string | number, + options?: InputOptions, ) { await this.clearValue() - return this.addValue(value) + return this.addValue(value, options) } diff --git a/packages/webdriverio/src/types.ts b/packages/webdriverio/src/types.ts index 7e8d6e6f4..a114ba74e 100644 --- a/packages/webdriverio/src/types.ts +++ b/packages/webdriverio/src/types.ts @@ -693,6 +693,13 @@ export type TransformReturn = T extends Array ? Array> : T +/** + * Additional options outside of the WebDriver spec, exclusively for WebdriverIO, only for runtime, and not sent to Appium + */ +export interface InputOptions { + mask?: boolean +} + declare global { namespace WebdriverIO { /** diff --git a/website/_sidebars.json b/website/_sidebars.json index caff3b4ae..65add7e8c 100644 --- a/website/_sidebars.json +++ b/website/_sidebars.json @@ -63,7 +63,8 @@ "pageobjects", "debugging", "repl", - "record" + "record", + "security" ] }, { diff --git a/website/docs/Configuration.md b/website/docs/Configuration.md index d2abb77a8..e4373509a 100644 --- a/website/docs/Configuration.md +++ b/website/docs/Configuration.md @@ -197,6 +197,23 @@ The path to the root of the cache directory. This directory is used to store all Type: `String`
Default: `process.env.WEBDRIVER_CACHE_DIR || os.tmpdir()` +### maskingPatterns + +For more secure logging, regular expressions set with `maskingPatterns` can obfuscate sensitive information from the log. + - The string format is a regular expression with or without flags (e.g. `/.../i`) and comma-separated for multiple regular expressions. + - For more details on masking patterns, see the [Masking Patterns section in the WDIO Logger README](https://github.com/webdriverio/webdriverio/blob/main/packages/wdio-logger/README.md#masking-patterns). + +Type: `String`
+Default: `undefined` + +**Example:** + +```js +{ + maskingPatterns: '/--key=([^ ]*)/i,/RESULT (.*)/' +} +``` + --- ## WebdriverIO diff --git a/website/docs/Security.md b/website/docs/Security.md new file mode 100644 index 000000000..4988beff0 --- /dev/null +++ b/website/docs/Security.md @@ -0,0 +1,162 @@ +--- +id: security +title: Security +--- + +WebdriverIO has the security aspect in mind when providing solutions. Below are some ways to better secure your test. + +# Masking Sensitive Data + +If you are using sensitive data during your test, it is essential to ensure that they are not visible to everyone, such as in logs. Also, when using a cloud provider, private keys are often involved. This information must be masked from logs, reporters, and other touchpoints. The following provides some masking solutions to run tests without exposing those values. + +## WebDriverIO + +### Mask Commands' Text Value + +The commands `addValue` and `setValue` support a boolean mask value to mask in WDIO and Appium logs, as well as reporters. Moreover, other tools, such as performance tools and third-party tools, will also receive the mask version, enhancing security. + +For example, if you are using a real production user and need to enter a password that you want to mask, then it is now possible with the following: + +```ts + async enterPassword(userPassword) { + const passwordInputElement = $('Password'); + + // Get focus + await passwordInputElement.click(); + + await passwordInputElement.setValue(userPassword, { mask: true }); + } +``` + +The above will hide the text value from WDIO logs and also from Appium logs. + +Logs example: +```text +2025-05-25T23:02:42. 336Z INFO webdriver: DATA { text: "**MASKED**" } +``` + +Limitations: + - In Appium, additional plugins could leak even though we ask to mask the information. + - Cloud providers could use a proxy for HTTP logging, which bypasses the mask mechanism put in place. + +:::info + +Minimum required version: + - WDIO v9.15.0 + - Appium v2.19.0 + +### Mask in WDIO Logs + +Using the `maskingPatterns` configuration, we can mask sensitive information from WDIO logs. However, Appium logs are not covered. + +For example, if you are using a Cloud provider and use the info level, then most certainly you will "leak" the user's key as shown below: + +```text +2025-05-29T19:09:11.309Z INFO @wdio/local-runner: Start worker 0-0 with arg: ./wdio.conf.ts --user=cloud_user --key=myCloudSecretExposedKey --spec myTest.test.ts +``` + +To counter that we can pass the the regular expression `'--key=([^ ]*)'` and now in the logs you will see + +```text +2025-05-29T19:09:11.309Z INFO @wdio/local-runner: Start worker 0-0 with arg: ./wdio.conf.ts --user=cloud_user --key=**MASKED** --spec myTest.test.ts +``` + +You can achieve the above by providing the regular expression to the `maskingPatterns` field of the configuration. + - For multiple regular expressions, use a single string but with a comma-separated value. + - For more details on masking patterns, see the [Masking Patterns section in the WDIO Logger README](https://github.com/webdriverio/webdriverio/blob/main/packages/wdio-logger/README.md#masking-patterns). + +```ts +export const config: WebdriverIO.Config = { + specs: [...], + capabilities: [{...}], + services: ['lighthouse'], + + /** + * test configurations + */ + logLevel: 'info', + maskingPatterns: '/--key=([^ ]*)/', + framework: 'mocha', + outputDir: __dirname, + + reporters: ['spec'], + + mochaOpts: { + ui: 'bdd', + timeout: 60000 + } +} +``` + +:::info + +Minimum required version: + - WDIO v9.15.0 + +### Disable WDIO Loggers + +Another way to block the logging of sensitive data is to lower or silence the log level or disable the logger. +It can achieved as follow: + +```ts +import logger from '@wdio/logger'; + +/** + * Set the logger level of the WDIO logger to 'silent' before *running a promise, which helps hide sensitive information in the logs. + */ +export const withSilentLogger = async (promise: () => Promise): Promise => { + const webdriverLogLevel = driver.options.logLevel ?? 'error'; + + try { + logger.setLevel('webdriver', 'silent'); + return await promise(); + } finally { + logger.setLevel('webdriver', webdriverLogLevel); + } +}; +``` + +## Thirds Party Solutions + +### Appium +Appium offers its masking solution; see [Log filter](https://appium.io/docs/en/2.0/guides/log-filters/) + - It can be tricky to use their solution. One way if possible is to pass a token in your string like `@mask@` and use it as a regular expression + - In some Appium versions, the values are also logged with each character comma-separated, so we need to be careful. + - Unfortunately, BrowserStack does not support this solution, but it is still useful locally + +Using the `@mask@` example previously mentioned, we can use the following JSON file named `appiumMaskLogFilters.json` +```json +[ + { + "pattern": "@mask@(.*)", + "flags": "s", + "replacer": "**MASKED**" + }, + { + "pattern": "\\[(\\\"@\\\",\\\"m\\\",\\\"a\\\",\\\"s\\\",\\\"k\\\",\\\"@\\\",\\S+)\\]", + "flags": "s", + "replacer": "[*,*,M,A,S,K,E,D,*,*]" + } +] +``` + +Then pass the JSON file name to the `logFilters` field into the appium service config: +```ts +import { AppiumServerArguments, AppiumServiceConfig } from '@wdio/appium-service'; +import { ServiceEntry } from '@wdio/types/build/Services'; + +const appium = [ + 'appium', + { + args: { + log: './logs/appium.log', + logFilters: './appiumMaskLogFilters.json', + } satisfies AppiumServerArguments, + } satisfies AppiumServiceConfig, +] satisfies ServiceEntry; +``` + +### BrowserStack + +BrowserStack also offer some level of masking to hide some data; see [hide sensitive data](https://www.browserstack.com/docs/automate/selenium/hide-sensitive-data) + - Unfortunately, the solution is all-or-nothing, so all text values of provided commands will be masked.