Emergent test failure on pypy3: no module named zipp

[jaraco]

Even in maint/44.x, tests are failing to run on pypy3 with this error:

$ ( # Run testsuite.
  if [ -z "$DISABLE_COVERAGE" ]
  then
    tox -- --cov
  else
    tox
  fi
)

.tox create: /home/travis/build/pypa/setuptools/.tox/.tox
.tox installdeps: tox-pip-version >= 0.0.6, virtualenv < 20, tox >= 3.2
Traceback (most recent call last):
  File "/opt/python/pypy3.6-7.1.1/lib-python/3/runpy.py", line 183, in _run_module_as_main
    mod_name, mod_spec, code = _get_module_details(mod_name, _Error)
  File "/opt/python/pypy3.6-7.1.1/lib-python/3/runpy.py", line 142, in _get_module_details
    return _get_module_details(pkg_main_name, error)
  File "/opt/python/pypy3.6-7.1.1/lib-python/3/runpy.py", line 109, in _get_module_details
    __import__(pkg_name)
  File "/home/travis/build/pypa/setuptools/.tox/.tox/site-packages/tox/__init__.py", line 9, in <module>
    import pluggy
  File "/home/travis/build/pypa/setuptools/.tox/.tox/site-packages/pluggy/__init__.py", line 16, in <module>
    from .manager import PluginManager, PluginValidationError
  File "/home/travis/build/pypa/setuptools/.tox/.tox/site-packages/pluggy/manager.py", line 11, in <module>
    import importlib_metadata
  File "/home/travis/build/pypa/setuptools/.tox/.tox/site-packages/importlib_metadata/__init__.py", line 9, in <module>
    import zipp
ModuleNotFoundError: No module named 'zipp'
The command "( # Run testsuite.
  if [ -z "$DISABLE_COVERAGE" ]
  then
    tox -- --cov
  else
    tox
  fi
)
" exited with 1.

[jaraco]

Running the build locally, I don't encounter the issue.

[jaraco]

I was able to replicate the issue with this Dockerfile:

FROM ubuntu:xenial

RUN apt update
RUN apt install -y curl libexpat1 bzip2
RUN curl -sSf --retry 5 https://storage.googleapis.com/travis-ci-language-archives/python/binaries/ubuntu/16.04/x86_64/pypy3.tar.bz2 -o - | tar xj --directory /
ENV PATH="/opt/python/pypy3/bin:${PATH}"

RUN python -m pip install -U pip setuptools wheel
RUN python -m pip install -U tox tox-venv

RUN echo '[tox]\n\
skipsdist = true\n\
requires = \n\
  tox-pip-version' > tox.ini

RUN tox

[jaraco]

Debugging deeper, I see that zipp isn't being installed as unknown:

draft $ docker run -it @($(docker build -q .).strip())                                                                                                                    
root@7a8498610f01:/# tox -v
using tox.ini: /tox.ini (pid 11)
could not satisfy requires PackageNotFoundError('tox-pip-version',)
using tox-3.14.4 from /opt/python/pypy3/site-packages/tox/__init__.py (pid 11)
.tox cannot reuse: no previous config //.tox/.tox/.tox-config1
.tox create: //.tox/.tox
[15] //.tox$ /opt/python/pypy3/bin/python -m venv .tox >.tox/log/.tox-0.log
.tox installdeps: tox-pip-version, tox >= 3.14.4
[21] /$ //.tox/.tox/bin/python -m pip install tox-pip-version 'tox >= 3.14.4' >//.tox/.tox/log/.tox-1.log
write config to //.tox/.tox/.tox-config1 as '5a2d9f9072906827c8580e9d9c506bdbae5c7678d946b83ac23ed3627e56149c /opt/python/pypy3/bin/python\n3.14.4 0 0 0\n00000000000000000000000000000000 tox-pip-version\n00000000000000000000000000000000 tox >= 3.14.4'
[45] /$ //.tox/.tox/bin/python -m tox -v
Traceback (most recent call last):
  File "/opt/python/pypy3/lib-python/3/runpy.py", line 183, in _run_module_as_main
    mod_name, mod_spec, code = _get_module_details(mod_name, _Error)
  File "/opt/python/pypy3/lib-python/3/runpy.py", line 142, in _get_module_details
    return _get_module_details(pkg_main_name, error)
  File "/opt/python/pypy3/lib-python/3/runpy.py", line 109, in _get_module_details
    __import__(pkg_name)
  File "//.tox/.tox/site-packages/tox/__init__.py", line 9, in <module>
    import pluggy
  File "//.tox/.tox/site-packages/pluggy/__init__.py", line 16, in <module>
    from .manager import PluginManager, PluginValidationError
  File "//.tox/.tox/site-packages/pluggy/manager.py", line 11, in <module>
    import importlib_metadata
  File "//.tox/.tox/site-packages/importlib_metadata/__init__.py", line 9, in <module>
    import zipp
ModuleNotFoundError: No module named 'zipp'
root@7a8498610f01:/# cat .tox/.tox/log/.tox-1.log
action: .tox, msg: getenv
cwd: /
cmd: //.tox/.tox/bin/python -m pip install tox-pip-version 'tox >= 3.14.4'
Collecting tox-pip-version
  Downloading https://files.pythonhosted.org/packages/94/f3/3e68654797b8c42717ba6ae0e4d163f93995d648b493c8699638e4d8f22c/tox-pip-version-0.0.6.tar.gz
Collecting tox>=3.14.4
  Cache entry deserialization failed, entry ignored
  Cache entry deserialization failed, entry ignored
  Downloading https://files.pythonhosted.org/packages/b5/d8/f88f4c09a33995601b3103dde6220d46d7c60cfbf5df961690322248b495/tox-3.14.4-py2.py3-none-any.whl (81kB)
Collecting packaging>=14 (from tox>=3.14.4)
  Cache entry deserialization failed, entry ignored
  Cache entry deserialization failed, entry ignored
  Downloading https://files.pythonhosted.org/packages/98/42/87c585dd3b113c775e65fd6b8d9d0a43abe1819c471d7af702d4e01e9b20/packaging-20.1-py2.py3-none-any.whl
Collecting pluggy<1,>=0.12.0 (from tox>=3.14.4)
  Cache entry deserialization failed, entry ignored
  Cache entry deserialization failed, entry ignored
  Downloading https://files.pythonhosted.org/packages/a0/28/85c7aa31b80d150b772fbe4a229487bc6644da9ccb7e427dd8cc60cb8a62/pluggy-0.13.1-py2.py3-none-any.whl
Collecting py<2,>=1.4.17 (from tox>=3.14.4)
  Cache entry deserialization failed, entry ignored
  Cache entry deserialization failed, entry ignored
  Downloading https://files.pythonhosted.org/packages/99/8d/21e1767c009211a62a8e3067280bfce76e89c9f876180308515942304d2d/py-1.8.1-py2.py3-none-any.whl (83kB)
Collecting six<2,>=1.14.0 (from tox>=3.14.4)
  Cache entry deserialization failed, entry ignored
  Cache entry deserialization failed, entry ignored
  Downloading https://files.pythonhosted.org/packages/65/eb/1f97cb97bfc2390a276969c6fae16075da282f5058082d4cb10c6c5c1dba/six-1.14.0-py2.py3-none-any.whl
Collecting virtualenv>=16.0.0 (from tox>=3.14.4)
  Cache entry deserialization failed, entry ignored
  Cache entry deserialization failed, entry ignored
  Downloading https://files.pythonhosted.org/packages/a3/dd/e43866ac0d25cd1ecbde0814f6cd7372ea257fbdf443dc9f0a01740da93d/virtualenv-20.0.4-py2.py3-none-any.whl (4.6MB)
Collecting toml>=0.9.4 (from tox>=3.14.4)
  Cache entry deserialization failed, entry ignored
  Cache entry deserialization failed, entry ignored
  Downloading https://files.pythonhosted.org/packages/a2/12/ced7105d2de62fa7c8fb5fce92cc4ce66b57c95fb875e9318dba7f8c5db0/toml-0.10.0-py2.py3-none-any.whl
Collecting filelock<4,>=3.0.0 (from tox>=3.14.4)
  Cache entry deserialization failed, entry ignored
  Cache entry deserialization failed, entry ignored
  Downloading https://files.pythonhosted.org/packages/93/83/71a2ee6158bb9f39a90c0dea1637f81d5eef866e188e1971a1b1ab01a35a/filelock-3.0.12-py3-none-any.whl
Collecting importlib-metadata<2,>=0.12; python_version < "3.8" (from tox>=3.14.4)
  Cache entry deserialization failed, entry ignored
  Cache entry deserialization failed, entry ignored
  Downloading https://files.pythonhosted.org/packages/8b/03/a00d504808808912751e64ccf414be53c29cad620e3de2421135fcae3025/importlib_metadata-1.5.0-py2.py3-none-any.whl
Collecting pyparsing>=2.0.2 (from packaging>=14->tox>=3.14.4)
  Cache entry deserialization failed, entry ignored
  Cache entry deserialization failed, entry ignored
  Downloading https://files.pythonhosted.org/packages/5d/bc/1e58593167fade7b544bfe9502a26dc860940a79ab306e651e7f13be68c2/pyparsing-2.4.6-py2.py3-none-any.whl (67kB)
Collecting appdirs<2,>=1.4.3 (from virtualenv>=16.0.0->tox>=3.14.4)
  Cache entry deserialization failed, entry ignored
  Cache entry deserialization failed, entry ignored
  Downloading https://files.pythonhosted.org/packages/56/eb/810e700ed1349edde4cbdc1b2a21e28cdf115f9faf263f6bbf8447c1abf3/appdirs-1.4.3-py2.py3-none-any.whl
Collecting distlib<1,>=0.3.0 (from virtualenv>=16.0.0->tox>=3.14.4)
  Cache entry deserialization failed, entry ignored
  Cache entry deserialization failed, entry ignored
  Downloading https://files.pythonhosted.org/packages/7d/29/694a3a4d7c0e1aef76092e9167fbe372e0f7da055f5dcf4e1313ec21d96a/distlib-0.3.0.zip (571kB)
Collecting importlib-resources<2,>=1.0; python_version < "3.7" (from virtualenv>=16.0.0->tox>=3.14.4)
  Cache entry deserialization failed, entry ignored
  Cache entry deserialization failed, entry ignored
  Downloading https://files.pythonhosted.org/packages/2f/f7/b4aa02cdd3ee7ebba375969d77c00826aa15c5db84247d23c89522dccbfa/importlib_resources-1.0.2-py2.py3-none-any.whl
Collecting zipp>=0.5 (from importlib-metadata<2,>=0.12; python_version < "3.8"->tox>=3.14.4)
  Cache entry deserialization failed, entry ignored
  Downloading https://files.pythonhosted.org/packages/60/85/668bca4a9ef474ca634c993e768f12bd99af1f06bb90bb2655bc538a967e/zipp-2.2.0.tar.gz
  Running setup.py (path:/tmp/pip-build-f09wken_/zipp/setup.py) egg_info for package zipp produced metadata for project name unknown. Fix your #egg=zipp fragments.
Installing collected packages: pyparsing, six, packaging, unknown, importlib-metadata, pluggy, py, appdirs, distlib, filelock, importlib-resources, virtualenv, toml, tox, tox-pip-version
  Running setup.py install for unknown: started
    Running setup.py install for unknown: finished with status 'done'
  Running setup.py install for distlib: started
    Running setup.py install for distlib: finished with status 'done'
  Running setup.py install for tox-pip-version: started
    Running setup.py install for tox-pip-version: finished with status 'done'
Successfully installed appdirs-1.4.3 distlib-0.3.0 filelock-3.0.12 importlib-metadata-1.5.0 importlib-resources-1.0.2 packaging-20.1 pluggy-0.13.1 py-1.8.1 pyparsing-2.4.6 six-1.14.0 toml-0.10.0 tox-3.14.4 tox-pip-version-0.0.6 unknown-0.0.0 virtualenv-20.0.4
You are using pip version 9.0.1, however version 20.0.2 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.

[jaraco]

I was able to replicate the issue without tox using this Dockerfile:

FROM ubuntu:xenial

RUN apt update
RUN apt install -y curl libexpat1 bzip2
RUN curl -sSf --retry 5 https://storage.googleapis.com/travis-ci-language-archives/python/binaries/ubuntu/16.04/x86_64/pypy3.tar.bz2 -o - | tar xj --directory /
ENV PATH="/opt/python/pypy3/bin:${PATH}"

RUN python -m venv env
RUN env/bin/python -m pip -V

CMD env/bin/python -m pip install zipp

Doing some more troubleshooting, I've identified the proximate cause:

• The version PyPy3 as installed by Travis includes a venv that installs pip 9.0.1.
• On Pypy3, pip 9 doesn't download wheels (even if universal), so builds from sdist.
• The sdist requires pip 10 or later to build or at least setuptools with declarative config, neither of which is present.
• The project builds as if it has no metadata (even a name).

[pganssle]

@jaraco Assuming it's not as simple as "upgrade pip in .travis.yml", maybe migrating the pypy3 build to Azure Pipelines would help?

[jaraco]

I've tried to address this issue with tox-dev/tox-venv#25, but there was too much pushback and I gave up. Now the issue rises again where the workaround tox-pip-version isn't yet relevant.

[gaborbernat]

With virtualenv now creating venv style environments by switching back to virtualenv you can guarantee newer pip out of box 🤷‍♂️(tox-venv is kinda absolute with virtualenv 20, not?)

[jaraco]

Assuming it's not as simple as "upgrade pip in .travis.yml"

It isn't. There's no opportunity to upgrade pip as tox is managing the environment creation and because it's the bootstrap environment, so doesn't get the benefit of pip_version support.

by switching back to virtualenv you can guarantee newer pip out of box

Yes, perhaps. I was really hoping to be able to rely on the stdlib implementation. My understanding was that tox was going to integrate tox-venv as the default implementation, so I wanted to adopt that. It will take a lot of work to unravel the work it took to get venv working. I'm going to defer this effort until I have confidence that venv/tox-venv is superseded.

Plus, setuptools is currently pinned against virtualenv 20 because of pypa/virtualenv#1638, so that's a non-starter for now.

Given the limited scope of this issue, I'm just going to mark the tests as expected failures in Travis until someone has the time to develop a more robust solution.

[gaborbernat]

My understanding was that tox was going to integrate tox-venv as the default implementation

I think you misunderstood this. The plan here was to create venv style environments instead of old virtualenv styles. The solution here was and is to use virtualenv 20, not to provide support for venv. venv is a standard library module, and out of our control; so relying on it besides gives you some kind of venv is asking for trouble (as you found out here). This has been achieved and there are no future changes planned on the environment creation side. tox-venv was and will be a community-supported plugin, and is not planned to be supported by core tox for features (and I'd discourage using it going ahead).

Plus, setuptools is currently pinned against virtualenv 20 because of pypa/virtualenv#1638, so that's a non-starter for now.

That issue as I understand is supporting a feature that was implemented by exploiting the internal workings of virtualenv. The fix there will be to fix setuptools to work with the new virtualenv; not changing virtualenv. Relying on the internal implementation of virtualenv does mean that you're exposed to some breakage...