Make coverage packaging be PEP-517 compliant

[ssbarnea]

Describe the bug
I observed some warnings when installing coverage related to the packaging of coverage, likely caused by lack of PEP-517 compliance.

Using legacy 'setup.py install' for coverage, since package 'wheel' is not installed.

To Reproduce

pip install coverage

Expected behavior
Full PEP-517 compliance (aka removal of setup.py and adoption of pyproject.toml).

Additional context
Add any other context about the problem here.

[nedbat]

I'm not opposed to this, but could use help changing the tooling.

[pfmoore]

There's not actually a problem here with PEP 517 compliance:

❯ pip install --no-binary coverage coverage
Collecting coverage
  Using cached coverage-6.4.1.tar.gz (719 kB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Building wheels for collected packages: coverage
  Building wheel for coverage (pyproject.toml) ... done
  Created wheel for coverage: filename=coverage-6.4.1-cp310-cp310-win_amd64.whl size=186291 sha256=3a568a74b8c95e42043eb4c118e09b6eb89a2d0f5b2b0617cfa7bef5cac793a5
  Stored in directory: c:\users\gustav\appdata\local\pip\cache\wheels\a7\02\73\c1f7efbef9c538f8893948fa77a61364777e01e861fa54fcec
Successfully built coverage
Installing collected packages: coverage
Successfully installed coverage-6.4.1

Certainly, coverage could change its build process to reflect recent standards, but the existing code is recognised and handled correctly, at least by a modern version of pip.

Maybe the OP can give a better explanation of how to reproduce the issue, as it doesn't seem to reproduce for me.

I'll note that the error reported, "Using legacy 'setup.py install' for coverage, since package 'wheel' is not installed" would not normally be given by pip if run with no special options, as wheel is installed into the build environment automatically. Was the problem that the OP was using --no-build-isolation (or some other non-standard option) when running pip?

[ssbarnea]

I am sure that I did not add any argument to pip when doing so and I seen this warning being displayed multiple times (different repos using coverage). Maybe in some cases pip ends up creating a build environment for one package without wheel package, one that requires coverage.

It might be related to pip or tox but I was still unable to find any place where this flag would have being set.

Sadly I forgot to post full log and now I do not know on which project I seen it. I will be back with more details, i bet it will not take long to see it again as I use coverage in most projects.

[ssbarnea]

Reproduced it. It happens all the time if you do not have wheel installed. In my case macos, python 3.11-dev from pyenv, no virtualenv active and pip 22.1.1:

$ pip install -v --no-cache-dir coverage
Using pip 22.1.2 from /Users/ssbarnea/.pyenv/versions/3.11-dev/lib/python3.11/site-packages/pip (python 3.11)
Collecting coverage
  Downloading coverage-6.4.1.tar.gz (719 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 720.0/720.0 kB 7.2 MB/s eta 0:00:00
  Running command python setup.py egg_info
  running egg_info
  creating /private/var/folders/32/1xrphgzd4xv777syxjtkpdw80000gn/T/pip-pip-egg-info-n54j1ri1/coverage.egg-info
  writing /private/var/folders/32/1xrphgzd4xv777syxjtkpdw80000gn/T/pip-pip-egg-info-n54j1ri1/coverage.egg-info/PKG-INFO
  writing dependency_links to /private/var/folders/32/1xrphgzd4xv777syxjtkpdw80000gn/T/pip-pip-egg-info-n54j1ri1/coverage.egg-info/dependency_links.txt
  writing entry points to /private/var/folders/32/1xrphgzd4xv777syxjtkpdw80000gn/T/pip-pip-egg-info-n54j1ri1/coverage.egg-info/entry_points.txt
  writing requirements to /private/var/folders/32/1xrphgzd4xv777syxjtkpdw80000gn/T/pip-pip-egg-info-n54j1ri1/coverage.egg-info/requires.txt
  writing top-level names to /private/var/folders/32/1xrphgzd4xv777syxjtkpdw80000gn/T/pip-pip-egg-info-n54j1ri1/coverage.egg-info/top_level.txt
  writing manifest file '/private/var/folders/32/1xrphgzd4xv777syxjtkpdw80000gn/T/pip-pip-egg-info-n54j1ri1/coverage.egg-info/SOURCES.txt'
  reading manifest file '/private/var/folders/32/1xrphgzd4xv777syxjtkpdw80000gn/T/pip-pip-egg-info-n54j1ri1/coverage.egg-info/SOURCES.txt'
  reading manifest template 'MANIFEST.in'
  no previously-included directories found matching 'doc/_build'
  no previously-included directories found matching 'doc/_spell'
  no previously-included directories found matching 'tests/eggsrc/build'
  adding license file 'LICENSE.txt'
  writing manifest file '/private/var/folders/32/1xrphgzd4xv777syxjtkpdw80000gn/T/pip-pip-egg-info-n54j1ri1/coverage.egg-info/SOURCES.txt'
  Preparing metadata (setup.py) ... done
Using legacy 'setup.py install' for coverage, since package 'wheel' is not installed.
Installing collected packages: coverage
  Running command Running setup.py install for coverage
  running install
  running build
  running build_py
  creating build
  creating build/lib.macosx-12.4-arm64-3.11
  creating build/lib.macosx-12.4-arm64-3.11/coverage
...
  running install_egg_info
  running egg_info
  writing coverage.egg-info/PKG-INFO
  writing dependency_links to coverage.egg-info/dependency_links.txt
  writing entry points to coverage.egg-info/entry_points.txt
  writing requirements to coverage.egg-info/requires.txt
  writing top-level names to coverage.egg-info/top_level.txt
  reading manifest file 'coverage.egg-info/SOURCES.txt'
  reading manifest template 'MANIFEST.in'
  no previously-included directories found matching 'doc/_build'
  no previously-included directories found matching 'doc/_spell'
  no previously-included directories found matching 'tests/eggsrc/build'
  adding license file 'LICENSE.txt'
  writing manifest file 'coverage.egg-info/SOURCES.txt'
  Copying coverage.egg-info to /Users/ssbarnea/.pyenv/versions/3.11-dev/lib/python3.11/site-packages/coverage-6.4.1-py3.11.egg-info
  running install_scripts
  Installing coverage script to /Users/ssbarnea/.pyenv/versions/3.11-dev/bin
  Installing coverage-3.11 script to /Users/ssbarnea/.pyenv/versions/3.11-dev/bin
  Installing coverage3 script to /Users/ssbarnea/.pyenv/versions/3.11-dev/bin
  writing list of installed files to '/private/var/folders/32/1xrphgzd4xv777syxjtkpdw80000gn/T/pip-record-fivujvk0/install-record.txt'
  Running setup.py install for coverage ... done
Successfully installed coverage-6.4.1

@pfmoore WTDY, does it count as a pip bug? Shouldn't pip install wheel when missing instead of falling back to setup.py? As wheel is not magically installed by default.

PS. Either way, it is still true that removing setup.py would avoid getting this warning and make pip use the pyproject.toml file.

[pfmoore]

@pfmoore WTDY, does it count as a pip bug? Shouldn't pip install wheel when missing instead of falling back to setup.py?

As my previous post shows, it's not obvious that it's a pip bug, as it works on Windows. So my expectation is that it's something in your environment. Pip does install wheel when missing, unless there's command line settings (or configuration) to make it behave differently.

What does pip config list show?

PS. Either way, it is still true that removing setup.py would avoid getting this warning and make pip use the pyproject.toml file.

Not necessarily. If your environment is blocking build isolation somehow, removing setup.py would simply cause pip to fail saying it can't build without the build dependencies.

[ssbarnea]

There is no custom pip config present.

$ pip config list
global.timeout='10'

I tried to use other python versions (like 3.9 from system/brew) and I was not able to reproduce the issue, mainly because coverage already has binary wheels for most platforms and pythons.

The reason why I seen this is because there is no binary wheel for 3.11 and will likely reproduce with any py311. I still think we should do something about it to avoid it, like installing wheel.

[pfmoore]

As I showed, --no-binary coverage allows you to test on other versions of Python. But I think I see why I was confused. You (presumably) have setuptools installed, but not wheel. I had neither setuptools nor wheel installed. This is messy heuristic code in pip that decides when to trigger the legacy install route. I wouldn't worry too much about it, as it is being gradually phased out anyway (PEP 517 builds will be used all the time in the not too distant future anyway).

Sorry for the wasted time, and thanks for your patience.

Coverage works fine under PEP 517 as it stands, but adding a pyproject.toml is a reasonable thing to do (it's a couple of lines):

[build-system]
requires = ["setuptools"]
build-backend = "setuptools.build_meta"

Note that you don't need to include wheel in requires because setuptools will add the extra build dependency itself.

Removing setup.py is not necessary for "full PEP 517 compliance" and is probably more work than necessary. I'm not objecting to the suggestion, just noting that it needs a better justification than "PEP 517 requires it"...

[ssbarnea]

Interestingly adding the pyproject.toml did uncover other unrelated problems, failure to even install due to greenlet not compiling with python 3.11, found python-greenlet/greenlet#305

PS. I do really appreciate your patience on this, btw feel free to rename or propose a different issue title to make it more appropriate as it is not fully PEP-517. Ditching the setup.py may be long shot, but still nice.

[nedbat]

I've made a pull request: #1447. It didn't have an problem with greenlet, should I be concerned?

[nedbat]

Hmm, I was using greenlet==1.1.2 and it worked, but in the meantime they've released 1.1.3 as requested in python-greenlet/greenlet#305, so all is well I guess.

[nedbat]

This is in commit 447334c.

[nedbat]

Now that I've done this, I can no longer run my test suite while offline. Is there way to run "make clean; tox" while not connected to the internet?

[ssbarnea]

Pip can be used offline, but you probably forgot something. Something that I almost always add inside tox file, is a PIP_DISABLE_VERSION_CHECK=1 to setenv as pip is too aggressive about that.

[layday]

I assume the issue is that pip attempts to install wheel which is not provisioned by venv. You could try downloading wheel ahead of time and setting PIP_NO_INDEX=1 and PIP_FIND_LINKS=/path/to/wheel/dir in your environment.

[nedbat]

This was what I tried: run the test suite once while online, turn off the network, run tox exactly the same again. No cleaning, etc. The second run fails. What am I doing wrong, or what do I have to do differently?

% tox -e py39
py39 develop-inst-noop: /Users/nedbatchelder/coverage/trunk
py39 installed: atomicwrites==1.4.1,attrs==22.1.0,backports.functools-lru-cache==1.6.4,cffi==1.15.1,colorama==0.4.5,-e git+ssh://git@github.com/nedbat/coveragepy.git@5908ca17f844edf8def46ee28fdf5aeeec4257fe#egg=coverage,decorator==5.1.1,distlib==0.3.6,dnspython==2.2.1,eventlet==0.33.1,exceptiongroup==1.0.0rc9,execnet==1.9.0,filelock==3.8.0,flaky==3.7.0,future==0.18.2,gevent==21.12.0,greenlet==1.1.3,hypothesis==6.54.5,importlib-metadata==4.12.0,iniconfig==1.1.1,packaging==21.3,platformdirs==2.5.2,pluggy==1.0.0,py==1.11.0,PyContracts @ https://github.com/slorg1/contracts/archive/c5a6da27d4dc9985f68e574d20d86000880919c3.zip,pycparser==2.21,pyparsing==3.0.9,pytest==7.1.3,pytest-forked==1.4.0,pytest-xdist==2.5.0,qualname==0.1.0,six==1.16.0,sortedcontainers==2.4.0,tomli==2.0.1,typing_extensions==4.3.0,virtualenv==20.16.5,zipp==3.8.1,zope.event==4.5.0,zope.interface==5.4.0
py39 run-test-pre: PYTHONHASHSEED='2791579876'
py39 run-test: commands[0] | python igor.py zip_mods
py39 run-test: commands[1] | python setup.py --quiet build_ext --inplace
py39 run-test: commands[2] | python -m pip install -q -e .
py39 run-test: commands[3] | python igor.py test_with_tracer c
=== CPython 3.9.14 with C tracer (.tox/py39/bin/python) ===
bringing up nodes...
.......................................................................................................x.....x............s...s...................x........ [ 13%]
.........x....................................................s..s...........................................x............................................ [ 26%]
...........................................................................................s.............................................................. [ 39%]
.......................................................................................................................................................... [ 52%]
.................................................................................s........................................................................ [ 65%]
........................................................................................s..................s.......................s...................... [ 79%]
...........................................s.............................................................................................................. [ 92%]
..........................................................s................................                                                                [100%]
1154 passed, 11 skipped, 5 xfailed in 55.63s
py39 run-test: commands[4] | python igor.py remove_extension
py39 run-test: commands[5] | python igor.py test_with_tracer py
Skipping tests with Python tracer: Only one tracer: no Python tracer for CPython
____________________________________________________________________________ summary _____________________________________________________________________________
  py39: commands succeeded
  congratulations :)

Turned off the wifi, then:

% tox -e py39
py39 develop-inst-noop: /Users/nedbatchelder/coverage/trunk
py39 installed: atomicwrites==1.4.1,attrs==22.1.0,backports.functools-lru-cache==1.6.4,cffi==1.15.1,colorama==0.4.5,-e git+ssh://git@github.com/nedbat/coveragepy.git@5908ca17f844edf8def46ee28fdf5aeeec4257fe#egg=coverage,decorator==5.1.1,distlib==0.3.6,dnspython==2.2.1,eventlet==0.33.1,exceptiongroup==1.0.0rc9,execnet==1.9.0,filelock==3.8.0,flaky==3.7.0,future==0.18.2,gevent==21.12.0,greenlet==1.1.3,hypothesis==6.54.5,importlib-metadata==4.12.0,iniconfig==1.1.1,packaging==21.3,platformdirs==2.5.2,pluggy==1.0.0,py==1.11.0,PyContracts @ https://github.com/slorg1/contracts/archive/c5a6da27d4dc9985f68e574d20d86000880919c3.zip,pycparser==2.21,pyparsing==3.0.9,pytest==7.1.3,pytest-forked==1.4.0,pytest-xdist==2.5.0,qualname==0.1.0,six==1.16.0,sortedcontainers==2.4.0,tomli==2.0.1,typing_extensions==4.3.0,virtualenv==20.16.5,zipp==3.8.1,zope.event==4.5.0,zope.interface==5.4.0
py39 run-test-pre: PYTHONHASHSEED='400962076'
py39 run-test: commands[0] | python igor.py zip_mods
py39 run-test: commands[1] | python setup.py --quiet build_ext --inplace
py39 run-test: commands[2] | python -m pip install -q -e .
  error: subprocess-exited-with-error

  × pip subprocess to install build dependencies did not run successfully.
  │ exit code: 1
  ╰─> [7 lines of output]
      WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.HTTPSConnection object at 0x1055c0f40>: Failed to establish a new connection: [Errno 8] nodename nor servname provided, or not known')': /simple/setuptools/
      WARNING: Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.HTTPSConnection object at 0x105a4a190>: Failed to establish a new connection: [Errno 8] nodename nor servname provided, or not known')': /simple/setuptools/
      WARNING: Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.HTTPSConnection object at 0x105a4aeb0>: Failed to establish a new connection: [Errno 8] nodename nor servname provided, or not known')': /simple/setuptools/
      WARNING: Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.HTTPSConnection object at 0x105d68ee0>: Failed to establish a new connection: [Errno 8] nodename nor servname provided, or not known')': /simple/setuptools/
      WARNING: Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.HTTPSConnection object at 0x105d68a30>: Failed to establish a new connection: [Errno 8] nodename nor servname provided, or not known')': /simple/setuptools/
      ERROR: Could not find a version that satisfies the requirement setuptools (from versions: none)
      ERROR: No matching distribution found for setuptools
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× pip subprocess to install build dependencies did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: InvocationError for command /Users/nedbatchelder/coverage/trunk/.tox/py39/bin/python -m pip install -q -e . (exited with code 1)
____________________________________________________________________________ summary _____________________________________________________________________________
ERROR:   py39: commands failed

[ssbarnea]

I suspect that this might be caused by either virtualenv or venv.

For example virtualenv has some optional flags --no-download and --never-download, which I am unable to see any for venv, but at the same time, AFAIK venv itself with not go online.

Try venv(s) first, to rule them out.

[pfmoore]

The problem is that pip is building the project (the python -m pip install -q -e . command) in an isolated environment. That's a newly created environment where everything gets installed from scratch. You are offline, so pip can't get setuptools and wheel from PyPI.

What you need to do is download wheels for setuptools and wheel from PyPI, and store them somewhere local. Then, when you're offline, set the envrionment variable PIP_FIND_LINKS to the path of the directory you stored those links in. You may also need to set PIP_NO_INDEX to 1, to stop pip tring PyPI. Those environment variables will be inherited by the build process, so the build will use your local wheels, and not go looking for PyPI to get them.

You may need to tell tox to pass the PIP_* environment variables through to the test command, I think it clears out the environment by default.

[nedbat]

This is now released as part of coverage 6.5.0.