Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update setup.py, buildscripts, CI and docs to require setuptools<60. #8356

Merged
merged 1 commit into from
Aug 16, 2022

Conversation

stuartarchibald
Copy link
Contributor

As title. setuptools>=65 removes the msvc compilers that
distutils/extension building relies on. NumPy 1.23 deprecates
numpy.distutils and recommends using setuptools<60. This patch
moves the code base to match these requirements to buy some time to
migrate away from numpy.distutils/distutils in general.

For context see #8355.

As title. setuptools>=65 removes the msvc compilers that
distutils/extension building relies on. NumPy 1.23 deprecates
`numpy.distutils` and recommends using `setuptools<60`. This patch
moves the code base to match these requirements to buy some time to
migrate away from `numpy.distutils`/`distutils` in general.

For context see numba#8355.
@@ -43,7 +43,7 @@ conda list
if [[ "$CONDA_SUBDIR" == "linux-32" || "$BITS32" == "yes" ]]; then
conda create -n $CONDA_ENV -q -y ${EXTRA_CHANNELS} python=$PYTHON pip gitpython pyyaml
else
conda create -n $CONDA_ENV -q -y ${EXTRA_CHANNELS} python=$PYTHON numpy=$NUMPY pip gitpython pyyaml
conda create -n $CONDA_ENV -q -y ${EXTRA_CHANNELS} python=$PYTHON numpy=$NUMPY pip gitpython pyyaml "setuptools<60"
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Didn't also add this to the linux-32 line above, it's implicit by there being no new packages created for linux-32 and it's scheduled for removal anyway.

I also checked that installing the other optional packages below didn't move the setuptools version through some transitive dependency, however this might have to be programmatically enforced at some point.

@stuartarchibald
Copy link
Contributor Author

I think this needs to be a candidate for 0.56.1 to ensure the build works and also that in production there's suitable restrictions in place against setuptools.

@esc
Copy link
Member

esc commented Aug 16, 2022

Build numba_smoketest_cpu_yaml_121 has started

@esc
Copy link
Member

esc commented Aug 16, 2022

Build numba_smoketest_cpu_yaml_121 has started

This passed.

Copy link
Member

@esc esc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I inspected the patch manually and ran git grep setuptools locally to produce:

.readthedocs.yml:  - method: setuptools
CHANGE_LOG:* James Bourbeau fixed some issues with tooling, #4794 add ``setuptools`` as a
CHANGE_LOG:* PR #4794: Add setuptools as a dependency
CHANGE_LOG:This is a bug-fix release to resolve packaging issues with setuptools
buildscripts/condarecipe.local/meta.yaml:  # build and run dependencies are duplicated to avoid setuptools issues
buildscripts/condarecipe.local/meta.yaml:    - setuptools <60
buildscripts/condarecipe.local/meta.yaml:    - setuptools <60
buildscripts/condarecipe.local/meta.yaml:    - setuptools <60
buildscripts/gpuci/build.sh:                  "setuptools<60"
buildscripts/incremental/setup_conda_environment.cmd:conda create -n %CONDA_ENV% -q -y python=%PYTHON% numpy=%NUMPY% cffi pip scipy jinja2 ipython gitpython pyyaml "setuptools<60"
buildscripts/incremental/setup_conda_environment.sh:    conda create -n $CONDA_ENV -q -y ${EXTRA_CHANNELS} python=$PYTHON numpy=$NUMPY pip gitpython pyyaml "setuptools<60"
docs/environment.yml:  - setuptools<60
docs/source/developer/repomap.rst:- :ghfile:`setup.py` - Standard Python distutils/setuptools script
docs/source/extending/entrypoints.rst:<https://setuptools.readthedocs.io/en/latest/setuptools.html#dynamic-discovery-of-services-and-plugins>`_
docs/source/extending/entrypoints.rst:feature of ``setuptools``.  This allows a Python package to register an
docs/source/user/installing.rst:  * ``setuptools<60``
docs/source/user/installing.rst:  * ``setuptools<60``
docs/source/user/pycc.rst:in your ``setup.py`` script, using distutils or setuptools::
numba/pycc/platform.py:        # Need to import it here since setuptools may monkeypatch it
numba/tests/pycc_distutils_usecase/setup_setuptools.py:from setuptools import setup
numba/tests/pycc_distutils_usecase/setup_setuptools_nested.py:from setuptools import setup
numba/tests/test_import.py:        # bug in setuptools/packaging causing a deprecation warning
numba/tests/test_pycc.py:    import setuptools
numba/tests/test_pycc.py:    setuptools = None
numba/tests/test_pycc.py:    @unittest.skipIf(setuptools is None, "test needs setuptools")
numba/tests/test_pycc.py:    def test_setup_py_setuptools(self):
numba/tests/test_pycc.py:        self.check_setup_py("setup_setuptools.py")
numba/tests/test_pycc.py:    @unittest.skipIf(setuptools is None, "test needs setuptools")
numba/tests/test_pycc.py:    def test_setup_py_setuptools_nested(self):
numba/tests/test_pycc.py:        self.check_setup_nested_py("setup_setuptools_nested.py")
setup.py:from setuptools import Extension, find_packages, setup
setup.py:        from setuptools._vendor.packaging import version as _version_module
setup.py:        # User forgot to give an argument probably, let setuptools handle that.
setup.py:    'setuptools <60',

AFAICT all the needed places have been patched.

@esc esc added 5 - Ready to merge Review and testing done, is ready to merge and removed 3 - Ready for Review labels Aug 16, 2022
@esc esc merged commit a06693d into numba:main Aug 16, 2022
@esc esc added the BuildFarm Passed For PRs that have been through the buildfarm and passed label Aug 16, 2022
@henryiii
Copy link
Contributor

See conda-forge/numba-feedstock#94 (comment) - this is a terrible limit that puts unnecessary constraints on all users of numba. Someone using numba can no longer use PEP 621 builds, because support was added around setuptools 61. They can not use modern edible installs (64). They can't use auto-discovery from 61 either.

The breakages introduced in v65.0.0 were very quickly reverted so that users wouldn't do exactly what was done here. NumPy itself works with any current version of setuptools with an environment variable. And this is a small optional feature of numba that many users do not need.

@esc
Copy link
Member

esc commented Sep 29, 2022

See conda-forge/numba-feedstock#94 (comment) - this is a terrible limit that puts unnecessary constraints on all users of numba. Someone using numba can no longer use PEP 621 builds, because support was added around setuptools 61. They can not use modern edible installs (64). They can't use auto-discovery from 61 either.

The breakages introduced in v65.0.0 were very quickly reverted so that users wouldn't do exactly what was done here. NumPy itself works with any current version of setuptools with an environment variable. And this is a small optional feature of numba that many users do not need.

@henryiii thank you for bringing this to our attention. We are "on the case" already, see also: #8366 (comment)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
5 - Ready to merge Review and testing done, is ready to merge BuildFarm Passed For PRs that have been through the buildfarm and passed Effort - short Short size effort needed
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

None yet

4 participants