[BUG] setuptools._vendor.pyparsing.ParseException on pypy wiht tox with setuptools 60.9.0

[alex]

setuptools version

60.9.0

Python version

pypy 3.7

OS

ubuntu linux

Additional environment information

No response

Description

Runnign tox with pypy and setuptools 60.9.0 produces:

  setuptools._vendor.pyparsing.ParseException: Expected W:(abcd...) (at char 0), (line:1, col:1)

The full traceback can be seen in this CI run: https://github.com/pyca/cryptography/runs/5192122892?check_suite_focus=true

Pinning to a previous version of setuptools resolves this, as can be seen at pyca/cryptography#6868

Expected behavior

The dependencies are installed correctly, as happens on previous versions of setuptools.

How to Reproduce

1. git clone https://github.com/pyca/cryptography
2. tox -e pypy3-nocoverage

Output

Traceback (most recent call last):
    File "/tmp/pip-build-env-7y3pjwes/overlay/site-packages/setuptools/_vendor/packaging/requirements.py", line 102, in __init__
      req = REQUIREMENT.parseString(requirement_string)
    File "/tmp/pip-build-env-7y3pjwes/overlay/site-packages/setuptools/_vendor/pyparsing.py", line 1654, in parseString
      raise exc
    File "/tmp/pip-build-env-7y3pjwes/overlay/site-packages/setuptools/_vendor/pyparsing.py", line 1644, in parseString
      loc, tokens = self._parse( instring, 0 )
    File "/tmp/pip-build-env-7y3pjwes/overlay/site-packages/setuptools/_vendor/pyparsing.py", line 1402, in _parseNoCache
      loc,tokens = self.parseImpl( instring, preloc, doActions )
    File "/tmp/pip-build-env-7y3pjwes/overlay/site-packages/setuptools/_vendor/pyparsing.py", line 3417, in parseImpl
      loc, exprtokens = e._parse( instring, loc, doActions )
    File "/tmp/pip-build-env-7y3pjwes/overlay/site-packages/setuptools/_vendor/pyparsing.py", line 1402, in _parseNoCache
      loc,tokens = self.parseImpl( instring, preloc, doActions )
    File "/tmp/pip-build-env-7y3pjwes/overlay/site-packages/setuptools/_vendor/pyparsing.py", line 3739, in parseImpl
      return self.expr._parse( instring, loc, doActions, callPreParse=False )
    File "/tmp/pip-build-env-7y3pjwes/overlay/site-packages/setuptools/_vendor/pyparsing.py", line 1402, in _parseNoCache
      loc,tokens = self.parseImpl( instring, preloc, doActions )
    File "/tmp/pip-build-env-7y3pjwes/overlay/site-packages/setuptools/_vendor/pyparsing.py", line 3400, in parseImpl
      loc, resultlist = self.exprs[0]._parse( instring, loc, doActions, callPreParse=False )
    File "/tmp/pip-build-env-7y3pjwes/overlay/site-packages/setuptools/_vendor/pyparsing.py", line 1402, in _parseNoCache
      loc,tokens = self.parseImpl( instring, preloc, doActions )
    File "/tmp/pip-build-env-7y3pjwes/overlay/site-packages/setuptools/_vendor/pyparsing.py", line 2711, in parseImpl
      raise ParseException(instring, loc, self.errmsg, self)
  setuptools._vendor.pyparsing.ParseException: Expected W:(abcd...) (at char 0), (line:1, col:1)

  During handling of the above exception, another exception occurred:

  Traceback (most recent call last):
    File "/home/runner/work/cryptography/cryptography/.tox/pypy3-nocoverage/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 363, in <module>
      main()
    File "/home/runner/work/cryptography/cryptography/.tox/pypy3-nocoverage/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 345, in main
      json_out['return_val'] = hook(**hook_input['kwargs'])
    File "/home/runner/work/cryptography/cryptography/.tox/pypy3-nocoverage/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 130, in get_requires_for_build_wheel
      return hook(config_settings)
    File "/tmp/pip-build-env-7y3pjwes/overlay/site-packages/setuptools/build_meta.py", line 178, in get_requires_for_build_wheel
      config_settings, requirements=['wheel'])
    File "/tmp/pip-build-env-7y3pjwes/overlay/site-packages/setuptools/build_meta.py", line 159, in _get_build_requires
      self.run_setup()
    File "/tmp/pip-build-env-7y3pjwes/overlay/site-packages/setuptools/build_meta.py", line [174](https://github.com/pyca/cryptography/runs/5192122892?check_suite_focus=true#step:12:174), in run_setup
      exec(compile(code, __file__, 'exec'), locals())
    File "setup.py", line 54, in <module>
      rust_version=">=1.41.0",
    File "/tmp/pip-build-env-7y3pjwes/overlay/site-packages/setuptools/__init__.py", line 155, in setup
      return distutils.core.setup(**attrs)
    File "/tmp/pip-build-env-7y3pjwes/overlay/site-packages/setuptools/_distutils/core.py", line 109, in setup
      _setup_distribution = dist = klass(attrs)
    File "/tmp/pip-build-env-7y3pjwes/overlay/site-packages/setuptools/dist.py", line 462, in __init__
      for k, v in attrs.items()
    File "/tmp/pip-build-env-7y3pjwes/overlay/site-packages/setuptools/_distutils/dist.py", line 293, in __init__
      self.finalize_options()
    File "/tmp/pip-build-env-7y3pjwes/overlay/site-packages/setuptools/dist.py", line 851, in finalize_options
      ep(self)
    File "/tmp/pip-build-env-7y3pjwes/overlay/site-packages/setuptools/dist.py", line 871, in _finalize_setup_keywords
      self._install_dependencies(ep)
    File "/tmp/pip-build-env-7y3pjwes/overlay/site-packages/setuptools/dist.py", line 881, in _install_dependencies
      for req in map(requirements.Requirement, always_iterable(ep.dist.requires))
    File "/tmp/pip-build-env-7y3pjwes/overlay/site-packages/setuptools/dist.py", line 879, in <setcomp>
      reqs = {
    File "/tmp/pip-build-env-7y3pjwes/overlay/site-packages/setuptools/_vendor/packaging/requirements.py", line 105, in __init__
      f'Parse error at "{ requirement_string[e.loc : e.loc + 8]!r}": {e.msg}'
  setuptools._vendor.packaging.requirements.InvalidRequirement: Parse error at "''": Expected W:(abcd...)
  error: subprocess-exited-with-error

[jaraco]

I'm guessing that the relevant change (history) is the bump of packaging.

[jaraco]

This repro is quite difficult to follow given that it requires pypy, rust, openssl, and still the error happens in an isolated pip environment without any access to the source.

[alex]

Would it be helpful if I put together a Dockerfile that built an environment for reproducing this?

There's unfortunately not much I can do about the moving pieces here, they're what's involved in reproducing this. (As for the isolated pip environment, I can take even less responsibility for that!)

[jaraco]

I've made some progress, but thanks. Still working on it...

[jaraco]

I'm now confident 2d6cc80 is implicated.

[jaraco]

The issue stems from the fact that importlib_metadata returns different types for cffi's requires:

cryptography main $ pypy -m pip-run -q cffi importlib_metadata -- -c "import importlib_metadata as md; print(md.requires('cffi'))"

cryptography main $ py -m pip-run -q cffi importlib_metadata -- -c "import importlib_metadata as md; print(md.requires('cffi'))"
['pycparser']

An empty string is returned on PyPy. I would have expected an iterable or None.

[alex]

Hmmm, on pypy cffi is pre-installed and ships with the interpreter AFAIK (it's for this reason that we don't even list it as a dependency on pypy! https://github.com/pyca/cryptography/blob/main/pyproject.toml#L8). Where does the metadata come from in that case?

I guess in the first instance the question is whether PyPy is in the wrong for this to be an empty string, or whether it's an edge case that importlib_metadata fails to handle correctly?

[jaraco]

My guess is the use-case is under-specified. What I can say is that the data is coming from the egg-info path:

cryptography main $ pypy -m pip-run -q cffi importlib_metadata -- -i -c "import importlib_metadata as md; dist = md.distribution('cffi')"
>>>> dist._read_dist_info_reqs()
>>>> dist._read_egg_info_reqs()
''
>>>> dist._path
PosixPath('/usr/local/Cellar/pypy3/7.3.7/libexec/lib_pypy/cffi.egg-info')