Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: do not ignore incorrect patterns with braces of single value #992

Merged
merged 23 commits into from Aug 6, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
e093494
refactor: use arrow function
iiroj Jul 13, 2021
e6c62de
fix: correctly handle incorrect patterns with braces of only single v…
iiroj Jul 13, 2021
cf6acdc
test: add test for matching pattern with bracket sequence
iiroj Jul 13, 2021
799f55a
refactor: less nesting in validateConfig
iiroj Jul 14, 2021
80fb012
refactor: move pattern fixing to validateConfig
iiroj Jul 15, 2021
40682e9
refactor: move error to messages
iiroj Jul 15, 2021
f527a25
refactor: throw early for empty configuration
iiroj Jul 15, 2021
dac0151
fix: improve configuration error output
iiroj Jul 15, 2021
6d381a4
refactor: move formatConfig into validateConfig
iiroj Jul 15, 2021
995913b
test: update snapshot serializer to remove absolute file path
iiroj Jul 15, 2021
a167697
fix: do not pass unused logger argument
iiroj Jul 15, 2021
7dbaa29
refactor: move config validation logging to validateConfig
iiroj Jul 22, 2021
f538361
fix: do not match escaped curly braces (`\{` or `\}`)
iiroj Jul 22, 2021
23ef46b
test: better error message in snapshot
iiroj Jul 24, 2021
665ac50
test: add missing assertions
iiroj Jul 24, 2021
640e58e
fix: match braces with escaped comma inside
iiroj Jul 24, 2021
51a63ea
fix: do not match braces when they start with the dollar sign (`$`)
iiroj Jul 24, 2021
e195b6a
docs: update BRACES_REGEXP comment
iiroj Jul 24, 2021
87219a5
refactor: move brace pattern validation to separate file
iiroj Jul 25, 2021
089e683
fix: correctly handle nested braces
iiroj Jul 25, 2021
2bd598f
refactor: clean up code
iiroj Jul 25, 2021
4ae8143
test: add some more tests
iiroj Jul 25, 2021
c9a24ce
Merge branch 'master' into braces-single-value
iiroj Aug 5, 2021
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
7 changes: 0 additions & 7 deletions lib/formatConfig.js

This file was deleted.

50 changes: 24 additions & 26 deletions lib/generateTasks.js
Expand Up @@ -16,43 +16,41 @@ const debug = require('debug')('lint-staged:gen-tasks')
* @param {boolean} [options.files] - Staged filepaths
* @param {boolean} [options.relative] - Whether filepaths to should be relative to gitDir
*/
module.exports = function generateTasks({
config,
cwd = process.cwd(),
gitDir,
files,
relative = false,
}) {
const generateTasks = ({ config, cwd = process.cwd(), gitDir, files, relative = false }) => {
debug('Generating linter tasks')

const absoluteFiles = files.map((file) => normalize(path.resolve(gitDir, file)))
const relativeFiles = absoluteFiles.map((file) => normalize(path.relative(cwd, file)))

return Object.entries(config).map(([pattern, commands]) => {
return Object.entries(config).map(([rawPattern, commands]) => {
let pattern = rawPattern

const isParentDirPattern = pattern.startsWith('../')

const fileList = micromatch(
relativeFiles
// Only worry about children of the CWD unless the pattern explicitly
// specifies that it concerns a parent directory.
.filter((file) => {
if (isParentDirPattern) return true
return !file.startsWith('..') && !path.isAbsolute(file)
}),
pattern,
{
cwd,
dot: true,
// If pattern doesn't look like a path, enable `matchBase` to
// match against filenames in every directory. This makes `*.js`
// match both `test.js` and `subdirectory/test.js`.
matchBase: !pattern.includes('/'),
}
).map((file) => normalize(relative ? file : path.resolve(cwd, file)))
// Only worry about children of the CWD unless the pattern explicitly
// specifies that it concerns a parent directory.
const filteredFiles = relativeFiles.filter((file) => {

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not a real issue, but I have a habit to always provide a bit of advice if I can:
generateTask.js is more about "generate" and "task", and this moment is more about clearing the user input and some sort of validation.
Wondering - is it possible to move this logic into validateConfig, where it gravitates more remove connectivity with messages and logger. However, it will also create new connectivity between generateTasks and validateConfig which does not exist currently.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I actually had the same thought in #992 (comment) after making these changes.

It makes sense to move it there, however currently validateConfig mostly just throws errors, and in this case it would simply warn. I guess that's ok though.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should now be moved to validateConfig

if (isParentDirPattern) return true
return !file.startsWith('..') && !path.isAbsolute(file)
})

const matches = micromatch(filteredFiles, pattern, {
cwd,
dot: true,
// If the pattern doesn't look like a path, enable `matchBase` to
// match against filenames in every directory. This makes `*.js`
// match both `test.js` and `subdirectory/test.js`.
matchBase: !pattern.includes('/'),
strictBrackets: true,
})

const fileList = matches.map((file) => normalize(relative ? file : path.resolve(cwd, file)))

const task = { pattern, commands, fileList }
debug('Generated task: \n%O', task)

return task
})
}

module.exports = generateTasks
154 changes: 62 additions & 92 deletions lib/index.js
@@ -1,6 +1,5 @@
'use strict'

const dedent = require('dedent')
const { cosmiconfig } = require('cosmiconfig')
const debugLog = require('debug')('lint-staged')
const stringifyObject = require('stringify-object')
Expand All @@ -13,9 +12,7 @@ const {
ConfigNotFoundError,
GetBackupStashError,
GitError,
InvalidOptionsError,
} = require('./symbols')
const formatConfig = require('./formatConfig')
const validateConfig = require('./validateConfig')
const validateOptions = require('./validateOptions')

Expand Down Expand Up @@ -85,105 +82,78 @@ const lintStaged = async (
} = {},
logger = console
) => {
try {
await validateOptions({ shell }, logger)
await validateOptions({ shell }, logger)

debugLog('Loading config using `cosmiconfig`')
debugLog('Loading config using `cosmiconfig`')

const resolved = configObject
? { config: configObject, filepath: '(input)' }
: await loadConfig(configPath)
const resolved = configObject
? { config: configObject, filepath: '(input)' }
: await loadConfig(configPath)

if (resolved == null) {
throw ConfigNotFoundError
}
if (resolved == null) {
logger.error(`${ConfigNotFoundError.message}.`)
throw ConfigNotFoundError
}

debugLog('Successfully loaded config from `%s`:\n%O', resolved.filepath, resolved.config)

// resolved.config is the parsed configuration object
// resolved.filepath is the path to the config file that was found
const formattedConfig = formatConfig(resolved.config)
const config = validateConfig(formattedConfig)

if (debug) {
// Log using logger to be able to test through `consolemock`.
logger.log('Running lint-staged with the following config:')
logger.log(stringifyObject(config, { indent: ' ' }))
} else {
// We might not be in debug mode but `DEBUG=lint-staged*` could have
// been set.
debugLog('lint-staged config:\n%O', config)
}
debugLog('Successfully loaded config from `%s`:\n%O', resolved.filepath, resolved.config)

// Unset GIT_LITERAL_PATHSPECS to not mess with path interpretation
debugLog('Unset GIT_LITERAL_PATHSPECS (was `%s`)', process.env.GIT_LITERAL_PATHSPECS)
delete process.env.GIT_LITERAL_PATHSPECS

try {
const ctx = await runAll(
{
allowEmpty,
concurrent,
config,
cwd,
debug,
maxArgLength,
quiet,
relative,
shell,
stash,
verbose,
},
logger
)
debugLog('Tasks were executed successfully!')
printTaskOutput(ctx, logger)
return true
} catch (runAllError) {
if (runAllError && runAllError.ctx && runAllError.ctx.errors) {
const { ctx } = runAllError
if (ctx.errors.has(ApplyEmptyCommitError)) {
logger.warn(PREVENTED_EMPTY_COMMIT)
} else if (ctx.errors.has(GitError) && !ctx.errors.has(GetBackupStashError)) {
logger.error(GIT_ERROR)
if (ctx.shouldBackup) {
// No sense to show this if the backup stash itself is missing.
logger.error(RESTORE_STASH_EXAMPLE)
}
}
// resolved.config is the parsed configuration object
// resolved.filepath is the path to the config file that was found
const config = validateConfig(resolved.config, logger)

printTaskOutput(ctx, logger)
return false
}
if (debug) {
// Log using logger to be able to test through `consolemock`.
logger.log('Running lint-staged with the following config:')
logger.log(stringifyObject(config, { indent: ' ' }))
} else {
// We might not be in debug mode but `DEBUG=lint-staged*` could have
// been set.
debugLog('lint-staged config:\n%O', config)
}

// Probably a compilation error in the config js file. Pass it up to the outer error handler for logging.
throw runAllError
}
} catch (lintStagedError) {
/** throw early because `validateOptions` options contains own logging */
if (lintStagedError === InvalidOptionsError) {
throw InvalidOptionsError
}
// Unset GIT_LITERAL_PATHSPECS to not mess with path interpretation
debugLog('Unset GIT_LITERAL_PATHSPECS (was `%s`)', process.env.GIT_LITERAL_PATHSPECS)
delete process.env.GIT_LITERAL_PATHSPECS

/** @todo move logging to `validateConfig` and remove this try/catch block */
if (lintStagedError === ConfigNotFoundError) {
logger.error(`${lintStagedError.message}.`)
} else {
// It was probably a parsing error
logger.error(dedent`
Could not parse lint-staged config.
try {
const ctx = await runAll(
{
allowEmpty,
concurrent,
config,
cwd,
debug,
maxArgLength,
quiet,
relative,
shell,
stash,
verbose,
},
logger
)
debugLog('Tasks were executed successfully!')
printTaskOutput(ctx, logger)
return true
} catch (runAllError) {
if (runAllError && runAllError.ctx && runAllError.ctx.errors) {
const { ctx } = runAllError
if (ctx.errors.has(ApplyEmptyCommitError)) {
logger.warn(PREVENTED_EMPTY_COMMIT)
} else if (ctx.errors.has(GitError) && !ctx.errors.has(GetBackupStashError)) {
logger.error(GIT_ERROR)
if (ctx.shouldBackup) {
// No sense to show this if the backup stash itself is missing.
logger.error(RESTORE_STASH_EXAMPLE)
}
}

${lintStagedError}
`)
printTaskOutput(ctx, logger)
return false
}
logger.error() // empty line
// Print helpful message for all errors
logger.error(dedent`
Please make sure you have created it correctly.
See https://github.com/okonet/lint-staged#configuration.
`)

throw lintStagedError

// Probably a compilation error in the config js file. Pass it up to the outer error handler for logging.
throw runAllError
}
}

Expand Down
4 changes: 2 additions & 2 deletions lib/makeCmdTasks.js
Expand Up @@ -3,8 +3,8 @@
const cliTruncate = require('cli-truncate')
const debug = require('debug')('lint-staged:make-cmd-tasks')

const { configurationError } = require('./messages')
const resolveTaskFn = require('./resolveTaskFn')
const { createError } = require('./validateConfig')

const STDOUT_COLUMNS_DEFAULT = 80

Expand Down Expand Up @@ -51,7 +51,7 @@ const makeCmdTasks = async ({ commands, files, gitDir, renderer, shell, verbose
// Do the validation here instead of `validateConfig` to skip evaluating the function multiple times
if (isFn && typeof command !== 'string') {
throw new Error(
createError(
configurationError(
'[Function]',
'Function task should return a string or an array of strings',
resolved
Expand Down
19 changes: 18 additions & 1 deletion lib/messages.js
Expand Up @@ -2,11 +2,26 @@

const chalk = require('chalk')
const { error, info, warning } = require('log-symbols')
const format = require('stringify-object')

const configurationError = (opt, helpMsg, value) =>
`${chalk.redBright(`${error} Validation Error:`)}

Invalid value for '${chalk.bold(opt)}': ${chalk.bold(
format(value, { inlineCharacterLimit: Number.POSITIVE_INFINITY })
)}

${helpMsg}`

const NOT_GIT_REPO = chalk.redBright(`${error} Current directory is not a git directory!`)

const FAILED_GET_STAGED_FILES = chalk.redBright(`${error} Failed to get staged files!`)

const incorrectBraces = (before, after) => `${warning} ${chalk.yellow(
`Detected incorrect braces with only single value: \`${before}\`. Reformatted as: \`${after}\``
)}
`

const NO_STAGED_FILES = `${info} No staged files found.`

const NO_TASKS = `${info} No staged files match any configured task.`
Expand All @@ -29,7 +44,7 @@ const GIT_ERROR = `\n ${error} ${chalk.red(`lint-staged failed due to a git err

const invalidOption = (name, value, message) => `${chalk.redBright(`${error} Validation Error:`)}

Invalid value for option ${chalk.bold(name)}: ${chalk.bold(value)}
Invalid value for option '${chalk.bold(name)}': ${chalk.bold(value)}

${message}

Expand All @@ -51,9 +66,11 @@ const CONFIG_STDIN_ERROR = 'Error: Could not read config from stdin.'

module.exports = {
CONFIG_STDIN_ERROR,
configurationError,
DEPRECATED_GIT_ADD,
FAILED_GET_STAGED_FILES,
GIT_ERROR,
incorrectBraces,
invalidOption,
NO_STAGED_FILES,
NO_TASKS,
Expand Down
71 changes: 71 additions & 0 deletions lib/validateBraces.js
@@ -0,0 +1,71 @@
const { incorrectBraces } = require('./messages')

/**
* A correctly-formed brace expansion must contain unquoted opening and closing braces,
* and at least one unquoted comma or a valid sequence expression.
* Any incorrectly formed brace expansion is left unchanged.
*
* @see https://www.gnu.org/software/bash/manual/html_node/Brace-Expansion.html
*
* Lint-staged uses `micromatch` for brace expansion, and its behavior is to treat
* invalid brace expansions as literal strings, which means they (typically) do not match
* anything.
*
* This RegExp tries to match most cases of invalid brace expansions, so that they can be
* detected, warned about, and re-formatted by removing the braces and thus hopefully
* matching the files as intended by the user. The only real fix is to remove the incorrect
* braces from user configuration, but this is left to the user (after seeing the warning).
*
* @example <caption>Globs with brace expansions</caption>
* - *.{js,tx} // expanded as *.js, *.ts
* - *.{{j,t}s,css} // expanded as *.js, *.ts, *.css
* - file_{1..10}.css // expanded as file_1.css, file_2.css, …, file_10.css
*
* @example <caption>Globs with incorrect brace expansions</caption>
* - *.{js} // should just be *.js
* - *.{js,{ts}} // should just be *.{js,ts}
* - *.\{js\} // escaped braces, so they're treated literally
* - *.${js} // dollar-sign inhibits expansion, so treated literally
* - *.{js\,ts} // the comma is escaped, so treated literally
*/
const BRACES_REGEXP = /(?<![\\$])({)(?:(?!(?<!\\),|\.\.|\{|\}).)*?(?<!\\)(})/g

/**
* @param {string} pattern
* @returns {string}
*/
const withoutIncorrectBraces = (pattern) => {
let output = `${pattern}`
let match = null

while ((match = BRACES_REGEXP.exec(pattern))) {
const fullMatch = match[0]
const withoutBraces = fullMatch.replace(/{/, '').replace(/}/, '')
output = output.replace(fullMatch, withoutBraces)
}

return output
}

/**
* Validate and remove incorrect brace expansions from glob pattern.
* For example `*.{js}` is incorrect because it doesn't contain a `,` or `..`,
* and will be reformatted as `*.js`.
*
* @param {string} pattern the glob pattern
* @param {*} logger
* @returns {string}
*/
const validateBraces = (pattern, logger) => {
const fixedPattern = withoutIncorrectBraces(pattern)

if (fixedPattern !== pattern) {
logger.warn(incorrectBraces(pattern, fixedPattern))
}

return fixedPattern
}

module.exports = validateBraces

module.exports.BRACES_REGEXP = BRACES_REGEXP