Drop dependency on Setuptools

[jakirkham]

Instead of relying on setuptools to get functionality from pkg_resources (which is slowly being refactored away), leverage packaging for the functionality used by Conda-Build.

Edit: Looked to see whether Conda needed a similar change. Appears it only indirectly uses pkg_resources via a vendored copy of tqdm. Updating the vendored copy of tqdm would fix that issue ( conda/conda#11432 ).

Fixes #4733

[jakirkham]

Updated with latest master. Please let me know if there's anything else needed here 🙂

[jezdez]

One thing to note is that pkg_resources.parse_version falls back to the LegacyVersion which is a pre-PEP440 compatible version, and as such could still be used in one or more packages out there. In other words, this has some potential to be a breaking change for a few projects and needs to be communicated as such.

FWIW I'm in favor of this move since it's way over time to standardize on PEP440 and following.

@chenghlee Have you heard anything about how compatible we are with PEP 440 nowadays?

[jakirkham]

Open to other solutions if there is a preferred alternative

[jezdez]

I've looked into more of the actual failing tests, and I think we shouldn't rush this for 3.22.0.

It's worth considering all the implications of the old LegacyVersion being used for the various non-PEP 440 conforming versions though, which happen to exist for conda packages that ship non-Python software, for example.
I do think we should support and use PEP 440 as the only standard for Python packages, but at the same time use a more relaxed version specifier for non-Python packages.

[jezdez]

I've backported LegacyVersion to get this unblocked and get rid of the dependency on setuptools. When we'll send skeleton into the retirement, we can remove it again.

[jezdez]

@jakirkham I took the liberty to amend your PR, could you take a look if that's along the lines what you had in mind?

[jakirkham]

Thanks Jannis! Doing a bit of vendoring to protect ourselves makes sense to me 🙂

[jakirkham]

Am seeing this test failure in some jobs (for example). Not sure what the expect behavior was before, but that version number doesn't look ok.

It is worth noting that SymPy is on GitHub these days. So a test that was presumably for Bitbucket/Mercurial may not be as relevant (particularly since Bitbucket dropped Mercurial support).

Maybe something using Heptapod would be better (if Mercurial testing is desired).

___________________________ test_sympy[with version] ___________________________
[gw0] linux -- Python 3.8.16 /usr/share/miniconda/envs/test/bin/python
Traceback (most recent call last):
  File "/home/runner/work/conda-build/conda-build/src/tests/test_api_skeleton.py", line 189, in test_sympy
    api.skeletonize(
  File "/home/runner/work/conda-build/conda-build/src/conda_build/api.py", line 270, in skeletonize
    skeleton_return = module.skeletonize(packages, output_dir=output_dir, version=version,
  File "/home/runner/work/conda-build/conda-build/src/conda_build/skeletons/pypi.py", line 280, in skeletonize
    versions = sorted(pypi_data["releases"].keys(), key=Version)
  File "/usr/share/miniconda/envs/test/lib/python3.8/site-packages/packaging/version.py", line 197, in __init__
    raise InvalidVersion(f"Invalid version: '{version}'")
packaging.version.InvalidVersion: Invalid version: '0.5.13-hg'

[jakirkham]

Any thoughts on what we should do here?

[jezdez]

@jakirkham @jaimergp I've updated it to use the legacy version parser everywhere, I think that should be okay, assuming the tests pass

[jakirkham]

Thanks Jannis! 🙏

Seeing another failure just on Unix Python 3.7 CI jobs (for example). Though not sure why it is coming up here.

_________________ test_render_with_python_arg_reduces_subspace _________________
[gw0] linux -- Python 3.7.16 /usr/share/miniconda/envs/test/bin/python
Traceback (most recent call last):
  File "/usr/share/miniconda/envs/test/lib/python3.7/shutil.py", line 566, in move
    os.rename(src, real_dst)
FileNotFoundError: [Errno 2] No such file or directory: '/usr/share/miniconda/envs/test/conda-bld/work' -> '/usr/share/miniconda/envs/test/conda-bld/test_subspace_selection_cli_1678300329405/work'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/runner/work/conda-build/conda-build/tests/cli/test_main_render.py", line 135, in test_render_with_python_arg_reduces_subspace
    main_render.execute(args)
  File "/home/runner/work/conda-build/conda-build/conda_build/cli/main_render.py", line 194, in execute
    variants=args.variants)
  File "/home/runner/work/conda-build/conda-build/conda_build/api.py", line 41, in render
    permit_unsatisfiable_variants=permit_unsatisfiable_variants)
  File "/home/runner/work/conda-build/conda-build/conda_build/render.py", line 834, in render_recipe
    m.config.compute_build_id(m.name(), m.version(), reset=reset_build_id)
  File "/home/runner/work/conda-build/conda-build/conda_build/config.py", line 615, in compute_build_id
    shutil.move(old_dir, work_dir)
  File "/usr/share/miniconda/envs/test/lib/python3.7/shutil.py", line 580, in move
    copy_function(src, real_dst)
  File "/usr/share/miniconda/envs/test/lib/python3.7/shutil.py", line 266, in copy2
    copyfile(src, dst, follow_symlinks=follow_symlinks)
  File "/usr/share/miniconda/envs/test/lib/python3.7/shutil.py", line 120, in copyfile
    with open(src, 'rb') as fsrc:
FileNotFoundError: [Errno 2] No such file or directory: '/usr/share/miniconda/envs/test/conda-bld/work'

[jezdez]

Given that 3.7 isn't maintained by conda-forge and defaults anymore, I'm ready to take the chance and remove it from the test matrix. Do you have concerns about this?

[jezdez]

I've filed #4796 and #4797 to unblock us here.

[jakirkham]

None. Have also been dropping Python 3.7 from other projects I work on

[jakirkham]

Thanks Jannis! 🙏

[beckermr]

xref: conda-forge/conda-build-feedstock#201

[dholth]

It looks like the new file is https://github.com/pypa/packaging/blob/main/src/packaging/version.py with underscores in front of each function. (It would be cleaner to import a non-underscored name from that file)

[jakirkham]

Maybe this should be raised as a new issue?

AIUI the public APIs we looked at weren't sufficient for reproducing the behavior we needed. Do you know a better public API to replace these with?

[dholth]

I'm surprised that parse from packaging's version.py has been renamed to _parse in our version, but it seems like there are other differences including us having _LegacyVersion.

[jakirkham]

IIUC _LegacyVersion was just a vendored copy of this class (since it is being phased out and we still need it atm), but maybe I'm missing something