Setup-python pypy-3.6 on macOS is broken

[gaborbernat]

See https://github.com/pypa/virtualenv/runs/4604337552?check_suite_focus=true

Installing and updating pip
/Users/runner/hostedtoolcache/PyPy/3.6.12/x64/bin/python -m ensurepip
Traceback (most recent call last):
  File "/Users/runner/hostedtoolcache/PyPy/3.6.12/x64/lib-python/3/runpy.py", line 196, in _run_module_as_main
    "__main__", mod_spec)
  File "/Users/runner/hostedtoolcache/PyPy/3.6.12/x64/lib-python/3/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/Users/runner/hostedtoolcache/PyPy/3.6.12/x64/lib-python/3/ensurepip/__main__.py", line 5, in <module>
    sys.exit(ensurepip._main())
  File "/Users/runner/hostedtoolcache/PyPy/3.6.12/x64/lib-python/3/ensurepip/__init__.py", line 213, in _main
    default_pip=args.default_pip,
  File "/Users/runner/hostedtoolcache/PyPy/3.6.12/x64/lib-python/3/ensurepip/__init__.py", line 126, in _bootstrap
    return _run_pip(args + [p[0] for p in _PROJECTS], additional_paths)
  File "/Users/runner/hostedtoolcache/PyPy/3.6.12/x64/lib-python/3/ensurepip/__init__.py", line 31, in _run_pip
    runpy.run_module("pip", run_name="__main__", alter_sys=True)
  File "/Users/runner/hostedtoolcache/PyPy/3.6.12/x64/lib-python/3/runpy.py", line 208, in run_module
    return _run_module_code(code, init_globals, run_name, mod_spec)
  File "/Users/runner/hostedtoolcache/PyPy/3.6.12/x64/lib-python/3/runpy.py", line 96, in _run_module_code
    mod_name, mod_spec, pkg_name, script_name)
  File "/Users/runner/hostedtoolcache/PyPy/3.6.12/x64/lib-python/3/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/tmplyzywfhl/pip-20.0.2-py2.py3-none-any.whl/pip/__main__.py", line 19, in <module>
  File "/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/tmplyzywfhl/pip-20.0.2-py2.py3-none-any.whl/pip/_internal/cli/main.py", line 73, in main
  File "/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/tmplyzywfhl/pip-20.0.2-py2.py3-none-any.whl/pip/_internal/commands/__init__.py", line 96, in create_command
  File "/Users/runner/hostedtoolcache/PyPy/3.6.12/x64/lib-python/3/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1003, in _gcd_import
  File "<frozen importlib._bootstrap>", line 980, in _find_and_load
  File "<frozen importlib._bootstrap>", line 964, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 665, in _load_unlocked
  File "<frozen importlib._bootstrap>", line 635, in _load_backward_compatible
  File "/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/tmplyzywfhl/pip-20.0.2-py2.py3-none-any.whl/pip/_internal/commands/install.py", line 24, in <module>
  File "/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/tmplyzywfhl/pip-20.0.2-py2.py3-none-any.whl/pip/_internal/cli/req_command.py", line 15, in <module>
  File "/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/tmplyzywfhl/pip-20.0.2-py2.py3-none-any.whl/pip/_internal/index/package_finder.py", line 21, in <module>
  File "/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/tmplyzywfhl/pip-20.0.2-py2.py3-none-any.whl/pip/_internal/index/collector.py", line 12, in <module>
  File "/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/tmplyzywfhl/pip-20.0.2-py2.py3-none-any.whl/pip/_vendor/html5lib/__init__.py", line 28, in <module>
  File "/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/tmplyzywfhl/pip-20.0.2-py2.py3-none-any.whl/pip/_vendor/html5lib/serializer.py", line 11, in <module>
  File "/Users/runner/hostedtoolcache/PyPy/3.6.12/x64/lib-python/3/xml/sax/saxutils.py", line 6, in <module>
    import os, urllib.parse, urllib.request
  File "/Users/runner/hostedtoolcache/PyPy/3.6.12/x64/lib-python/3/urllib/request.py", line 2616, in <module>
    from _scproxy import _get_proxy_settings, _get_proxies
  File "/Users/runner/hostedtoolcache/PyPy/3.6.12/x64/lib_pypy/_scproxy.py", line 64, in <module>
    ffi = _CFSetup()
  File "/Users/runner/hostedtoolcache/PyPy/3.6.12/x64/lib_pypy/_scproxy.py", line 63, in _CFSetup
    return CFProxy()
  File "/Users/runner/hostedtoolcache/PyPy/3.6.12/x64/lib_pypy/_scproxy.py", line 52, in __init__
    func = getattr(mod, fname)
  File "/Users/runner/hostedtoolcache/PyPy/3.6.12/x64/lib-python/3/ctypes/__init__.py", line 366, in __getattr__
    func = self.__getitem__(name)
  File "/Users/runner/hostedtoolcache/PyPy/3.6.12/x64/lib-python/3/ctypes/__init__.py", line 371, in __getitem__
    func = self._FuncPtr((name_or_ordinal, self))
  File "/Users/runner/hostedtoolcache/PyPy/3.6.12/x64/lib_pypy/_ctypes/function.py", line 255, in __init__
    ptr = self._getfuncptr([], ctypes.c_int)
  File "/Users/runner/hostedtoolcache/PyPy/3.6.12/x64/lib_pypy/_ctypes/function.py", line 442, in _getfuncptr
    self._ptr = cdll.getfunc(self.name, ffi_argtypes, ffi_restype)
AttributeError: No symbol SCDynamicStoreCopyProxies found in library <None>
Error: The process '/Users/runner/hostedtoolcache/PyPy/3.6.12/x64/bin/python' failed with exit code 1

cc @mattip do you think this is a PyPy bug or a packaging one?

[mattip]

I think this is bit-rot from the combination of a EOL Python version with an evolving macOS platform. PyPy issue 3314 was fixed in Jan 2021. The last released PyPy3.6 was in Nov 2020. The last CPython bugfix release was 3.6.8, 2018-12-24. PyPy continued for a while after that until Nov 2020 but has also given up on 3.6. Did CPython backport the change to fix this problem to CPython 3.6?

[nikita-bykov]

Hello @gaborbernat, thanks for your report! We'll look at it.

[vsafonkin]

Hi @gaborbernat, @mattip

I think this is bit-rot from the combination of a EOL Python version with an evolving macOS platform. PyPy issue 3314 was fixed in Jan 2021. The last released PyPy3.6 was in Nov 2020. The last CPython bugfix release was 3.6.8, 2018-12-24. PyPy continued for a while after that until Nov 2020 but has also given up on 3.6. Did CPython backport the change to fix this problem to CPython 3.6?

Yep, looks like this issue from PyPy side and we cannot to do something from setup-python side.

[gaborbernat]

Perhaps setup-python should mark this package unavailable on modern macOS platforms?

[mattip]

It would be helpful to move the default pypy3 to a supported version: 3.7 at least if not 3.8

[exarkun]

It would be helpful to move the default pypy3 to a supported version: 3.7 at least if not 3.8

I was surprised that "pypy3" meant 3.6 and not a newer version.

[mattip]

I think this is the problematic code in setup-python/dist/setup/index.js. I can't link to the exact line because the file is too big:

function findPythonVersion(version, architecture) {
    return __awaiter(this, void 0, void 0, function* () {
        switch (version.toUpperCase()) {
            case 'PYPY2':
                return usePyPy('2', architecture);
            case 'PYPY3':
                // keep pypy3 pointing to 3.6 for backward compatibility
                return usePyPy('3.6', architecture);
            default:
                return yield useCpythonVersion(version, architecture);
        }
    });
}
exports.findPythonVersion = findPythonVersion;

[vsafonkin]

Hi @mattip, PyPy input pypy3 was removed: #342

I'm going to close this issue, feel free to contact us if you have any concerns, thank you!

[mattip]

The documentation does not mention a pypy3 nor pypy-3.6 variant, so closing seems reasonable.