When using `--resolve-all-configs`, there is unexpected behavior in that
`src_paths` ends up resolving relative to the project root, which
defaults to the current working directory. This results in first-party
modules being marked as third-party modules in the default case.

Under the previous implementation, one possible workaround would be to
specify the relative path to config directory (e.g.
`relative/path/to/configdir/src`). However, assuming that the most
common use of `--resolve-all-configs` is to support multiple
sub-projects in the same repository/overall directory, this workaround
would now require each sub-project to understand where it lives in the
filesystem.

This change proposes a fix that sets `directory` on the `config_data` to
be the directory containing the used configuration file if not already
set. Downstream, this directory is then used to resolve the absolute
paths specified by `src_paths`.

Fixes PyCQA#2045

Avoid recursing into default skip files like .venv. Also avoid doing an
iteration per config file type. Lastly, use scandir to improve file system walk
performance.

The previous implementation would walk the entire tree, and iterate over
each config source type to test if the file exists. Instead, walk over
existing files and do a fast filter to remove candidates.