Make src_paths behave as expected when using --resolve-all-configs and improve performance #2142
Conversation
When using `--resolve-all-configs`, there is unexpected behavior in that `src_paths` ends up resolving relative to the project root, which defaults to the current working directory. This results in first-party modules being marked as third-party modules in the default case. Under the previous implementation, one possible workaround would be to specify the relative path to config directory (e.g. `relative/path/to/configdir/src`). However, assuming that the most common use of `--resolve-all-configs` is to support multiple sub-projects in the same repository/overall directory, this workaround would now require each sub-project to understand where it lives in the filesystem. This change proposes a fix that sets `directory` on the `config_data` to be the directory containing the used configuration file if not already set. Downstream, this directory is then used to resolve the absolute paths specified by `src_paths`. Fixes PyCQA#2045
|
One thing of note: This would be backwards incompatible for anyone depending on the behavior of resolving first-party This change also makes the behavior consistent with configuring a settings path. |
Avoid recursing into default skip files like .venv. Also avoid doing an iteration per config file type. Lastly, use scandir to improve file system walk performance. The previous implementation would walk the entire tree, and iterate over each config source type to test if the file exists. Instead, walk over existing files and do a fast filter to remove candidates.
sudowork
changed the title
| potential_config_file.path, CONFIG_SECTIONS[potential_config_file.name] | ||
| ) | ||
| if "directory" not in config_data: | ||
| config_data["directory"] = os.path.dirname(potential_config_file.path) |
There was a problem hiding this comment.
This is the main logic to fix the src_paths resolution.
When using
--resolve-all-configs, there is unexpected behavior in thatsrc_pathsends up resolving relative to the project root, which defaults to the current working directory. This results in first-party modules being marked as third-party modules in the default case.Under the previous implementation, one possible workaround would be to specify the relative path to config directory (e.g.
relative/path/to/configdir/src). However, assuming that the most common use of--resolve-all-configsis to support multiple sub-projects in the same repository/overall directory, this workaround would now require each sub-project to understand where it lives in the filesystem.This change proposes a fix that sets
directoryon theconfig_datato be the directory containing the used configuration file if not already set. Downstream, this directory is then used to resolve the absolute paths specified bysrc_paths.This change also introduces performance improvements to
find_all_configsby pruning as we walk the filesystem and other smaller performance enhancements.Fixes #2045