Allow recursive globs for package_data

[nullableVoidPtr]

Summary of changes

Calls to glob within find_data_files now have recursive=True. This allows for the use of the ** globs in package_data.

Closes #1806

Pull Request Checklist

• Changes have tests
• News fragment added in changelog.d/.

[abravalheri]

Thank you very much for the contribution @nullableVoidPtr.

[abravalheri]

Thank you very much @nullableVoidPtr,

I also manually tested your branch with the following commands and everything seems to work fine:

rm -rf /tmp/myproj
mkdir /tmp/myproj
cd /tmp/myproj

cat <<EOS > /tmp/myproj/pyproject.toml
[build-system]
requires = ['setuptools @ git+https://github.com/nullableVoidPtr/setuptools@feature/recursive-glob-pkg-data']
build-backend = 'setuptools.build_meta'

[project]
name = "myproj"
version = "0.42"

[tool.setuptools]
package-data = {"*" = ["**/*.json"]}
EOS

mkdir -p src/mypkg/subpkg/subsubpkg
touch src/mypkg/a.json
touch src/mypkg/b.json
touch src/mypkg/subpkg/c.json
touch src/mypkg/subpkg/d.json
touch src/mypkg/subpkg/subsubpkg/e.json

virtualenv -p python3.10 .venv
.venv/bin/python -m pip install -U pip build
.venv/bin/python -m build
unzip -l dist/*.whl

Successfully built myproj-0.42.tar.gz and myproj-0.42-py3-none-any.whl
Archive:  dist/myproj-0.42-py3-none-any.whl
  Length      Date    Time    Name
---------  ---------- -----   ----
        0  2022-05-16 12:12   mypkg/a.json
        0  2022-05-16 12:12   mypkg/b.json
        0  2022-05-16 12:12   mypkg/subpkg/c.json
        0  2022-05-16 12:12   mypkg/subpkg/d.json
        0  2022-05-16 12:12   mypkg/subpkg/subsubpkg/e.json
      111  2022-05-16 12:14   myproj-0.42.dist-info/METADATA
       92  2022-05-16 12:14   myproj-0.42.dist-info/WHEEL
        6  2022-05-16 12:14   myproj-0.42.dist-info/top_level.txt
      649  2022-05-16 12:14   myproj-0.42.dist-info/RECORD

Please have in mind that in the (near) future we might include some warnings to discourage people to consider entire directory as "data" without listing the sub-packages first or using auto-discovery. See the discussions in #3308, #3260.

[EwoutH]

This is an awesome feature! With only 3 lines of code I could include two directories recursively for a project!

[tool.setuptools.package-data]
"ema_workbench.examples.data" = ["**"]
"ema_workbench.examples.models" = ["**"]

One request though: Could this feature be included in the Data Files Support section of the tutorial? I think this feature is currently largely undocumented.