Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

pip (sometimes) fails when multiple projects are installing the same binary in parallel #8984

Closed
gaborbernat opened this issue Oct 12, 2020 · 10 comments
Closed

pip (sometimes) fails when multiple projects are installing the same binary in parallel #8984

gaborbernat opened this issue Oct 12, 2020 · 10 comments

Comments

@gaborbernat
Copy link

gaborbernat commented Oct 12, 2020 •
edited 

Traceback (most recent call last):
  File "/Users/runner/work/virtualenv/virtualenv/.tox/pypy/site-packages/virtualenv/seed/wheels/embed/pip-20.2.3-py2.py3-none-any.whl/pip/_internal/cli/base_command.py", line 228, in _main
    status = self.run(options, args)
  File "/Users/runner/work/virtualenv/virtualenv/.tox/pypy/site-packages/virtualenv/seed/wheels/embed/pip-20.2.3-py2.py3-none-any.whl/pip/_internal/cli/req_command.py", line 182, in wrapper
    return func(self, options, args)
  File "/Users/runner/work/virtualenv/virtualenv/.tox/pypy/site-packages/virtualenv/seed/wheels/embed/pip-20.2.3-py2.py3-none-any.whl/pip/_internal/commands/download.py", line 134, in run
    reqs, check_supported_wheels=True
  File "/Users/runner/work/virtualenv/virtualenv/.tox/pypy/site-packages/virtualenv/seed/wheels/embed/pip-20.2.3-py2.py3-none-any.whl/pip/_internal/resolution/legacy/resolver.py", line 183, in resolve
    discovered_reqs.extend(self._resolve_one(requirement_set, req))
  File "/Users/runner/work/virtualenv/virtualenv/.tox/pypy/site-packages/virtualenv/seed/wheels/embed/pip-20.2.3-py2.py3-none-any.whl/pip/_internal/resolution/legacy/resolver.py", line 388, in _resolve_one
    abstract_dist = self._get_abstract_dist_for(req_to_install)
  File "/Users/runner/work/virtualenv/virtualenv/.tox/pypy/site-packages/virtualenv/seed/wheels/embed/pip-20.2.3-py2.py3-none-any.whl/pip/_internal/resolution/legacy/resolver.py", line 339, in _get_abstract_dist_for
    self._populate_link(req)
  File "/Users/runner/work/virtualenv/virtualenv/.tox/pypy/site-packages/virtualenv/seed/wheels/embed/pip-20.2.3-py2.py3-none-any.whl/pip/_internal/resolution/legacy/resolver.py", line 305, in _populate_link
    req.link = self._find_requirement_link(req)
  File "/Users/runner/work/virtualenv/virtualenv/.tox/pypy/site-packages/virtualenv/seed/wheels/embed/pip-20.2.3-py2.py3-none-any.whl/pip/_internal/resolution/legacy/resolver.py", line 270, in _find_requirement_link
    best_candidate = self.finder.find_requirement(req, upgrade)
  File "/Users/runner/work/virtualenv/virtualenv/.tox/pypy/site-packages/virtualenv/seed/wheels/embed/pip-20.2.3-py2.py3-none-any.whl/pip/_internal/index/package_finder.py", line 899, in find_requirement
    req.name, specifier=req.specifier, hashes=hashes,
  File "/Users/runner/work/virtualenv/virtualenv/.tox/pypy/site-packages/virtualenv/seed/wheels/embed/pip-20.2.3-py2.py3-none-any.whl/pip/_internal/index/package_finder.py", line 881, in find_best_candidate
    candidates = self.find_all_candidates(project_name)
  File "/Users/runner/work/virtualenv/virtualenv/.tox/pypy/site-packages/virtualenv/seed/wheels/embed/pip-20.2.3-py2.py3-none-any.whl/pip/_internal/index/package_finder.py", line 826, in find_all_candidates
    project_url, link_evaluator=link_evaluator,
  File "/Users/runner/work/virtualenv/virtualenv/.tox/pypy/site-packages/virtualenv/seed/wheels/embed/pip-20.2.3-py2.py3-none-any.whl/pip/_internal/index/package_finder.py", line 790, in process_project_url
    html_page = self._link_collector.fetch_page(project_url)
  File "/Users/runner/work/virtualenv/virtualenv/.tox/pypy/site-packages/virtualenv/seed/wheels/embed/pip-20.2.3-py2.py3-none-any.whl/pip/_internal/index/collector.py", line 643, in fetch_page
    return _get_html_page(location, session=self.session)
  File "/Users/runner/work/virtualenv/virtualenv/.tox/pypy/site-packages/virtualenv/seed/wheels/embed/pip-20.2.3-py2.py3-none-any.whl/pip/_internal/index/collector.py", line 455, in _get_html_page
    resp = _get_html_response(url, session=session)
  File "/Users/runner/work/virtualenv/virtualenv/.tox/pypy/site-packages/virtualenv/seed/wheels/embed/pip-20.2.3-py2.py3-none-any.whl/pip/_internal/index/collector.py", line 169, in _get_html_response
    "Cache-Control": "max-age=0",
  File "/Users/runner/work/virtualenv/virtualenv/.tox/pypy/site-packages/virtualenv/seed/wheels/embed/pip-20.2.3-py2.py3-none-any.whl/pip/_vendor/requests/sessions.py", line 543, in get
    return self.request('GET', url, **kwargs)
  File "/Users/runner/work/virtualenv/virtualenv/.tox/pypy/site-packages/virtualenv/seed/wheels/embed/pip-20.2.3-py2.py3-none-any.whl/pip/_internal/network/session.py", line 421, in request
    return super(PipSession, self).request(method, url, *args, **kwargs)
  File "/Users/runner/work/virtualenv/virtualenv/.tox/pypy/site-packages/virtualenv/seed/wheels/embed/pip-20.2.3-py2.py3-none-any.whl/pip/_vendor/requests/sessions.py", line 530, in request
    resp = self.send(prep, **send_kwargs)
  File "/Users/runner/work/virtualenv/virtualenv/.tox/pypy/site-packages/virtualenv/seed/wheels/embed/pip-20.2.3-py2.py3-none-any.whl/pip/_vendor/requests/sessions.py", line 643, in send
    r = adapter.send(request, **kwargs)
  File "/Users/runner/work/virtualenv/virtualenv/.tox/pypy/site-packages/virtualenv/seed/wheels/embed/pip-20.2.3-py2.py3-none-any.whl/pip/_vendor/cachecontrol/adapter.py", line 53, in send
    resp = super(CacheControlAdapter, self).send(request, **kw)
  File "/Users/runner/work/virtualenv/virtualenv/.tox/pypy/site-packages/virtualenv/seed/wheels/embed/pip-20.2.3-py2.py3-none-any.whl/pip/_vendor/requests/adapters.py", line 416, in send
    self.cert_verify(conn, request.url, verify, cert)
  File "/Users/runner/work/virtualenv/virtualenv/.tox/pypy/site-packages/virtualenv/seed/wheels/embed/pip-20.2.3-py2.py3-none-any.whl/pip/_vendor/requests/adapters.py", line 224, in cert_verify
    cert_loc = extract_zipped_paths(DEFAULT_CA_BUNDLE_PATH)
  File "/Users/runner/work/virtualenv/virtualenv/.tox/pypy/site-packages/virtualenv/seed/wheels/embed/pip-20.2.3-py2.py3-none-any.whl/pip/_vendor/requests/utils.py", line 255, in extract_zipped_paths
    extracted_path = zip_file.extract(member, path=tmp)
  File "/Users/runner/hostedtoolcache/PyPy/2.7.13/x64/lib-python/2.7/zipfile.py", line 1036, in extract
    return self._extract_member(member, path, pwd)
  File "/Users/runner/hostedtoolcache/PyPy/2.7.13/x64/lib-python/2.7/zipfile.py", line 1083, in _extract_member
    os.makedirs(upperdirs)
  File "/Users/runner/work/virtualenv/virtualenv/.tox/pypy/lib-python/2.7/os.py", line 157, in makedirs
    mkdir(name, mode)
OSError: [Errno 17] File exists: '/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/pip/_vendor/certifi'
 [ERROR via_app_data:111]
271280 downloaded wheel setuptools-44.1.1-py2.py3-none-any.whl [DEBUG acquire:69]

Is it possible that multiple pip instances share the same temp folder? (/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn). The error in question focused:

    cert_loc = extract_zipped_paths(DEFAULT_CA_BUNDLE_PATH)
  File "/Users/runner/work/virtualenv/virtualenv/.tox/pypy/site-packages/virtualenv/seed/wheels/embed/pip-20.2.3-py2.py3-none-any.whl/pip/_vendor/requests/utils.py", line 255, in extract_zipped_paths
    extracted_path = zip_file.extract(member, path=tmp)
  File "/Users/runner/hostedtoolcache/PyPy/2.7.13/x64/lib-python/2.7/zipfile.py", line 1036, in extract
    return self._extract_member(member, path, pwd)
  File "/Users/runner/hostedtoolcache/PyPy/2.7.13/x64/lib-python/2.7/zipfile.py", line 1083, in _extract_member
    os.makedirs(upperdirs)
  File "/Users/runner/work/virtualenv/virtualenv/.tox/pypy/lib-python/2.7/os.py", line 157, in makedirs
    mkdir(name, mode)
OSError: [Errno 17] File exists: '/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/pip/_vendor/certifi'
 [ERROR via_app_data:111]
271280 downloaded wheel setuptools-44.1.1-py2.py3-none-any.whl [DEBUG acquire:69]
@brainwane brainwane added state: needs eyes Needs a maintainer/triager to take a closer look S: needs triage Issues/PRs that need to be triaged labels Oct 19, 2020
@pradyunsg
Copy link
Member

What operation is pip trying here? How is it being run?

@gaborbernat
Copy link
Author

@Julian can you fill in the blanks? AFAIK the install in question is a pip install of pip.

@Julian
Copy link
Contributor

Julian commented Oct 20, 2020

This was from that tox install log I pasted?

@gaborbernat
Copy link
Author

Yes.

@Julian
Copy link
Contributor

Julian commented Oct 20, 2020 •
edited

Then I think here's the same situation, which I reproduced from running tox -p 10 -r (on this toxini, and which fails on maybe two side-by-side pip install -U pip operations that are copying files from cwd which are simultaneously disappearing?):


action: py36-format-build, msg: parallel py36-format-build
cwd: /Users/julian/Development/jsonschema
cmd: /Users/julian/.dotfiles/.local/share/virtualenvs/tox/bin/python /Users/julian/.dotfiles/.local/share/virtualenvs/tox/lib/python3.9/site-packages/tox/__main__.py -p 10 -r
py36-format-build recreate: /Users/julian/Development/jsonschema/.tox/py36-format-build
py36-format-build installdeps: build
py36-format-build installed: /Users/julian/Development/jsonschema/.tox/py36-format-build/lib/python3.6/site-packages/pip/_vendor/toml/decoder.py:47: DeprecationWarning: invalid escape sequence \.,  TIME_RE = re.compile("([0-9]{2}):([0-9]{2}):([0-9]{2})(\.([0-9]{3,6}))?"),build==0.0.4,importlib-metadata==2.0.0,packaging==20.4,pep517==0.9.1,pyparsing==2.4.7,six==1.15.0,toml==0.10.1,zipp==3.3.1
py36-format-build run-test-pre: PYTHONHASHSEED='1649442921'
py36-format-build run-test: commands[0] | /Users/julian/Development/jsonschema/.tox/py36-format-build/bin/python -m pip install '/Users/julian/Development/jsonschema[format]'
Processing /Users/julian/Development/jsonschema
  Installing build dependencies: started
  Installing build dependencies: finished with status 'done'
  Getting requirements to build wheel: started
  Getting requirements to build wheel: finished with status 'done'
    Preparing wheel metadata: started
    Preparing wheel metadata: finished with status 'done'
Processing /Users/julian/Library/Caches/pip/wheels/34/13/19/294da8e11bce7e563afee51251b9fa878185e14f4b5caf00cb/pyrsistent-0.17.3-cp36-cp36m-macosx_10_15_x86_64.whl
Collecting attrs>=17.4.0
  Using cached attrs-20.2.0-py2.py3-none-any.whl (48 kB)
Requirement already satisfied: importlib-metadata; python_version < "3.8" in /Users/julian/Development/jsonschema/.tox/py36-format-build/lib/python3.6/site-packages (from jsonschema==3.2.1.dev307+ga11ab2b1) (2.0.0)
Collecting jsonpointer>1.13; extra == "format"
  Using cached jsonpointer-2.0-py2.py3-none-any.whl (7.6 kB)
Processing /Users/julian/Library/Caches/pip/wheels/bb/af/c9/b6e9fb5f9b2470e4ed2a7241c9ab3a8cdd3bc8555ae02ca2e6/strict_rfc3339-0.7-cp36-none-any.whl
Collecting idna; extra == "format"
  Using cached idna-2.10-py2.py3-none-any.whl (58 kB)
Collecting webcolors; extra == "format"
  Using cached webcolors-1.11.1-py3-none-any.whl (9.9 kB)
Collecting fqdn; extra == "format"
  Using cached fqdn-1.5.0-py3-none-any.whl (9.1 kB)
Collecting rfc3987; extra == "format"
  Using cached rfc3987-1.3.8-py2.py3-none-any.whl (13 kB)
Requirement already satisfied: zipp>=0.5 in /Users/julian/Development/jsonschema/.tox/py36-format-build/lib/python3.6/site-packages (from importlib-metadata; python_version < "3.8"->jsonschema==3.2.1.dev307+ga11ab2b1) (3.3.1)
Collecting cached-property>=1.3.0; python_version < "3.8"
  Using cached cached_property-1.5.2-py2.py3-none-any.whl (7.6 kB)
Building wheels for collected packages: jsonschema
  Building wheel for jsonschema (PEP 517): started
  Building wheel for jsonschema (PEP 517): finished with status 'done'
  Created wheel for jsonschema: filename=jsonschema-3.2.1.dev307+ga11ab2b1-py2.py3-none-any.whl size=60971 sha256=32257d8f4ea1a6e962f22cef0e2c0cbf3ff545704ac22a951fd6ea8ad396de21
  Stored in directory: /private/var/folders/cr/h4lk12bj5cn4qdx4ct9hddmm0000gn/T/pip-ephem-wheel-cache-_kc4bwhp/wheels/a7/ee/b1/e628099227e2a4c700eef9e57fd0e0948251a01f0d155cc469
Successfully built jsonschema
Installing collected packages: pyrsistent, attrs, jsonpointer, strict-rfc3339, idna, webcolors, cached-property, fqdn, rfc3987, jsonschema
Successfully installed attrs-20.2.0 cached-property-1.5.2 fqdn-1.5.0 idna-2.10 jsonpointer-2.0 jsonschema-3.2.1.dev307+ga11ab2b1 pyrsistent-0.17.3 rfc3987-1.3.8 strict-rfc3339-0.7 webcolors-1.11.1
WARNING: You are using pip version 20.1.1; however, version 20.2.3 is available.
You should consider upgrading via the '/Users/julian/Development/jsonschema/.tox/py36-format-build/bin/python -m pip install --upgrade pip' command.
py36-format-build run-test: commands[1] | /Users/julian/Development/jsonschema/.tox/py36-format-build/bin/python -m build /Users/julian/Development/jsonschema --outdir /Users/julian/Development/jsonschema/.tox/py36-format-build/tmp/dist
Looking in links: /var/folders/cr/h4lk12bj5cn4qdx4ct9hddmm0000gn/T/tmpvi78dplv
Requirement already satisfied: setuptools in /Users/julian/Development/jsonschema/.tox/py36-format-build/lib/python3.6/site-packages (49.6.0)
Requirement already satisfied: pip in /Users/julian/Development/jsonschema/.tox/py36-format-build/lib/python3.6/site-packages (20.1.1)
Requirement already satisfied: setuptools>=40.6.0 in /Users/julian/Development/jsonschema/.tox/py36-format-build/lib/python3.6/site-packages (from -r /var/folders/cr/h4lk12bj5cn4qdx4ct9hddmm0000gn/T/build-reqs-_dmysxst.txt (line 1)) (49.6.0)
Collecting setuptools_scm[toml]>=3.4
  Using cached setuptools_scm-4.1.2-py2.py3-none-any.whl (27 kB)
Requirement already satisfied: wheel in /Users/julian/Development/jsonschema/.tox/py36-format-build/lib/python3.6/site-packages (from -r /var/folders/cr/h4lk12bj5cn4qdx4ct9hddmm0000gn/T/build-reqs-_dmysxst.txt (line 3)) (0.35.1)
Requirement already satisfied: toml; extra == "toml" in /Users/julian/Development/jsonschema/.tox/py36-format-build/lib/python3.6/site-packages (from setuptools_scm[toml]>=3.4->-r /var/folders/cr/h4lk12bj5cn4qdx4ct9hddmm0000gn/T/build-reqs-_dmysxst.txt (line 2)) (0.10.1)
Installing collected packages: setuptools-scm
Successfully installed setuptools-scm
WARNING: You are using pip version 20.1.1; however, version 20.2.3 is available.
You should consider upgrading via the '/Users/julian/Development/jsonschema/.tox/py36-format-build/bin/python -m pip install --upgrade pip' command.
running sdist
running egg_info
writing jsonschema.egg-info/PKG-INFO
writing dependency_links to jsonschema.egg-info/dependency_links.txt
writing entry points to jsonschema.egg-info/entry_points.txt
writing requirements to jsonschema.egg-info/requires.txt
writing top-level names to jsonschema.egg-info/top_level.txt
reading manifest template 'MANIFEST.in'
writing manifest file 'jsonschema.egg-info/SOURCES.txt'
running check
copying files to jsonschema-3.2.1.dev307+ga11ab2b1...
'json/tests/latest' not a regular file -- skipping
copying jsonschema.egg-info/SOURCES.txt -> jsonschema-3.2.1.dev307+ga11ab2b1/jsonschema.egg-info
Writing jsonschema-3.2.1.dev307+ga11ab2b1/setup.cfg
Creating tar archive
removing 'jsonschema-3.2.1.dev307+ga11ab2b1' (and everything under it)
error removing jsonschema-3.2.1.dev307+ga11ab2b1: [Errno 2] No such file or directory: 'jsonschema-3.2.1.dev307+ga11ab2b1/json/tests/draft3/extends.json'
error removing jsonschema-3.2.1.dev307+ga11ab2b1: [Errno 2] No such file or directory: 'jsonschema-3.2.1.dev307+ga11ab2b1/json/tests/draft3/additionalProperties.json'
error removing jsonschema-3.2.1.dev307+ga11ab2b1: [Errno 2] No such file or directory: 'jsonschema-3.2.1.dev307+ga11ab2b1/json/tests/draft3/dependencies.json'
error removing jsonschema-3.2.1.dev307+ga11ab2b1: [Errno 2] No such file or directory: 'jsonschema-3.2.1.dev307+ga11ab2b1/json/tests/draft3/items.json'
error removing jsonschema-3.2.1.dev307+ga11ab2b1: [Errno 2] No such file or directory: 'jsonschema-3.2.1.dev307+ga11ab2b1/json/tests/draft3/enum.json'
error removing jsonschema-3.2.1.dev307+ga11ab2b1: [Errno 2] No such file or directory: 'jsonschema-3.2.1.dev307+ga11ab2b1/json/tests/draft3/properties.json'
error removing jsonschema-3.2.1.dev307+ga11ab2b1: [Errno 2] No such file or directory: 'jsonschema-3.2.1.dev307+ga11ab2b1/json/tests/draft3/maxLength.json'
error removing jsonschema-3.2.1.dev307+ga11ab2b1: [Errno 2] No such file or directory: 'jsonschema-3.2.1.dev307+ga11ab2b1/json/tests/draft3/minimum.json'
error removing jsonschema-3.2.1.dev307+ga11ab2b1: [Errno 2] No such file or directory: 'jsonschema-3.2.1.dev307+ga11ab2b1/json/tests/draft3/optional/zeroTerminatedFloats.json'
error removing jsonschema-3.2.1.dev307+ga11ab2b1: [Errno 2] No such file or directory: 'jsonschema-3.2.1.dev307+ga11ab2b1/json/tests/draft3/optional/bignum.json'
error removing jsonschema-3.2.1.dev307+ga11ab2b1: [Errno 2] No such file or directory: 'jsonschema-3.2.1.dev307+ga11ab2b1/json/tests/draft3/optional/non-bmp-regex.json'
error removing jsonschema-3.2.1.dev307+ga11ab2b1: [Errno 2] No such file or directory: 'jsonschema-3.2.1.dev307+ga11ab2b1/json/tests/draft3/optional/format/color.json'
error removing jsonschema-3.2.1.dev307+ga11ab2b1: [Errno 2] No such file or directory: 'jsonschema-3.2.1.dev307+ga11ab2b1/json/tests/draft3/optional/format/date.json'
error removing jsonschema-3.2.1.dev307+ga11ab2b1: [Errno 2] No such file or directory: 'jsonschema-3.2.1.dev307+ga11ab2b1/json/tests/draft3/optional/format/uri.json'
error removing jsonschema-3.2.1.dev307+ga11ab2b1: [Errno 2] No such file or directory: 'jsonschema-3.2.1.dev307+ga11ab2b1/json/tests/draft3/optional/format/host-name.json'
error removing jsonschema-3.2.1.dev307+ga11ab2b1: [Errno 2] No such file or directory: 'jsonschema-3.2.1.dev307+ga11ab2b1/json/tests/draft3/disallow.json'
error removing jsonschema-3.2.1.dev307+ga11ab2b1: [Errno 2] No such file or directory: 'jsonschema-3.2.1.dev307+ga11ab2b1/json/.github/workflows'
error removing jsonschema-3.2.1.dev307+ga11ab2b1: [Errno 2] No such file or directory: 'jsonschema-3.2.1.dev307+ga11ab2b1/json/.github'
error removing jsonschema-3.2.1.dev307+ga11ab2b1: [Errno 2] No such file or directory: 'jsonschema-3.2.1.dev307+ga11ab2b1/json/tox.ini'
error removing jsonschema-3.2.1.dev307+ga11ab2b1: [Errno 2] No such file or directory: 'jsonschema-3.2.1.dev307+ga11ab2b1/json'
error removing jsonschema-3.2.1.dev307+ga11ab2b1: [Errno 2] No such file or directory: 'jsonschema-3.2.1.dev307+ga11ab2b1/.gitignore'
error removing jsonschema-3.2.1.dev307+ga11ab2b1: [Errno 2] No such file or directory: 'jsonschema-3.2.1.dev307+ga11ab2b1/jsonschema/_types.py'
error removing jsonschema-3.2.1.dev307+ga11ab2b1: [Errno 2] No such file or directory: 'jsonschema-3.2.1.dev307+ga11ab2b1/jsonschema/_validators.py'
error removing jsonschema-3.2.1.dev307+ga11ab2b1: [Errno 2] No such file or directory: 'jsonschema-3.2.1.dev307+ga11ab2b1/jsonschema/validators.py'
error removing jsonschema-3.2.1.dev307+ga11ab2b1: [Errno 2] No such file or directory: 'jsonschema-3.2.1.dev307+ga11ab2b1/jsonschema/tests/_suite.py'
error removing jsonschema-3.2.1.dev307+ga11ab2b1: [Errno 2] No such file or directory: 'jsonschema-3.2.1.dev307+ga11ab2b1/jsonschema/tests/test_exceptions.py'
error removing jsonschema-3.2.1.dev307+ga11ab2b1: [Errno 2] No such file or directory: 'jsonschema-3.2.1.dev307+ga11ab2b1/jsonschema/tests/__init__.py'
error removing jsonschema-3.2.1.dev307+ga11ab2b1: [Errno 2] No such file or directory: 'jsonschema-3.2.1.dev307+ga11ab2b1/jsonschema/tests/test_jsonschema_test_suite.py'
error removing jsonschema-3.2.1.dev307+ga11ab2b1: [Errno 2] No such file or directory: 'jsonschema-3.2.1.dev307+ga11ab2b1/jsonschema/tests/test_validators.py'
error removing jsonschema-3.2.1.dev307+ga11ab2b1: [Errno 2] No such file or directory: 'jsonschema-3.2.1.dev307+ga11ab2b1/jsonschema/tests/test_cli.py'
error removing jsonschema-3.2.1.dev307+ga11ab2b1: [Errno 2] No such file or directory: 'jsonschema-3.2.1.dev307+ga11ab2b1/jsonschema/tests/test_format.py'
error removing jsonschema-3.2.1.dev307+ga11ab2b1: [Errno 2] No such file or directory: 'jsonschema-3.2.1.dev307+ga11ab2b1/jsonschema/tests/_helpers.py'
error removing jsonschema-3.2.1.dev307+ga11ab2b1: [Errno 2] No such file or directory: 'jsonschema-3.2.1.dev307+ga11ab2b1/jsonschema/tests/test_types.py'
error removing jsonschema-3.2.1.dev307+ga11ab2b1: [Errno 2] No such file or directory: 'jsonschema-3.2.1.dev307+ga11ab2b1/jsonschema/tests'
error removing jsonschema-3.2.1.dev307+ga11ab2b1: [Errno 2] No such file or directory: 'jsonschema-3.2.1.dev307+ga11ab2b1/jsonschema/__init__.py'
error removing jsonschema-3.2.1.dev307+ga11ab2b1: [Errno 2] No such file or directory: 'jsonschema-3.2.1.dev307+ga11ab2b1/jsonschema/schemas/draft7.json'
error removing jsonschema-3.2.1.dev307+ga11ab2b1: [Errno 2] No such file or directory: 'jsonschema-3.2.1.dev307+ga11ab2b1/jsonschema/schemas/draft6.json'
error removing jsonschema-3.2.1.dev307+ga11ab2b1: [Errno 2] No such file or directory: 'jsonschema-3.2.1.dev307+ga11ab2b1/jsonschema/schemas/draft4.json'
error removing jsonschema-3.2.1.dev307+ga11ab2b1: [Errno 2] No such file or directory: 'jsonschema-3.2.1.dev307+ga11ab2b1/jsonschema/schemas/draft3.json'
error removing jsonschema-3.2.1.dev307+ga11ab2b1: [Errno 2] No such file or directory: 'jsonschema-3.2.1.dev307+ga11ab2b1/jsonschema/schemas'
error removing jsonschema-3.2.1.dev307+ga11ab2b1: [Errno 2] No such file or directory: 'jsonschema-3.2.1.dev307+ga11ab2b1/jsonschema/_legacy_validators.py'
error removing jsonschema-3.2.1.dev307+ga11ab2b1: [Errno 2] No such file or directory: 'jsonschema-3.2.1.dev307+ga11ab2b1/jsonschema/_format.py'
error removing jsonschema-3.2.1.dev307+ga11ab2b1: [Errno 2] No such file or directory: 'jsonschema-3.2.1.dev307+ga11ab2b1/jsonschema/cli.py'
error removing jsonschema-3.2.1.dev307+ga11ab2b1: [Errno 2] No such file or directory: 'jsonschema-3.2.1.dev307+ga11ab2b1/jsonschema/benchmarks/issue232.py'
error removing jsonschema-3.2.1.dev307+ga11ab2b1: [Errno 2] No such file or directory: 'jsonschema-3.2.1.dev307+ga11ab2b1/jsonschema/benchmarks/__init__.py'
error removing jsonschema-3.2.1.dev307+ga11ab2b1: [Errno 2] No such file or directory: 'jsonschema-3.2.1.dev307+ga11ab2b1/jsonschema/benchmarks/json_schema_test_suite.py'
error removing jsonschema-3.2.1.dev307+ga11ab2b1: [Errno 2] No such file or directory: 'jsonschema-3.2.1.dev307+ga11ab2b1/jsonschema/benchmarks/issue232/issue.json'
error removing jsonschema-3.2.1.dev307+ga11ab2b1: [Errno 2] No such file or directory: 'jsonschema-3.2.1.dev307+ga11ab2b1/jsonschema/benchmarks/issue232'
error removing jsonschema-3.2.1.dev307+ga11ab2b1: [Errno 2] No such file or directory: 'jsonschema-3.2.1.dev307+ga11ab2b1/jsonschema/benchmarks'
error removing jsonschema-3.2.1.dev307+ga11ab2b1: [Errno 2] No such file or directory: 'jsonschema-3.2.1.dev307+ga11ab2b1/jsonschema/exceptions.py'
error removing jsonschema-3.2.1.dev307+ga11ab2b1: [Errno 2] No such file or directory: 'jsonschema-3.2.1.dev307+ga11ab2b1/jsonschema/_reflect.py'
error removing jsonschema-3.2.1.dev307+ga11ab2b1: [Errno 2] No such file or directory: 'jsonschema-3.2.1.dev307+ga11ab2b1/jsonschema/__main__.py'
error removing jsonschema-3.2.1.dev307+ga11ab2b1: [Errno 2] No such file or directory: 'jsonschema-3.2.1.dev307+ga11ab2b1/jsonschema/_utils.py'
error removing jsonschema-3.2.1.dev307+ga11ab2b1: [Errno 2] No such file or directory: 'jsonschema-3.2.1.dev307+ga11ab2b1/jsonschema'
error removing jsonschema-3.2.1.dev307+ga11ab2b1: [Errno 2] No such file or directory: 'jsonschema-3.2.1.dev307+ga11ab2b1/.github/FUNDING.yml'
error removing jsonschema-3.2.1.dev307+ga11ab2b1: [Errno 2] No such file or directory: 'jsonschema-3.2.1.dev307+ga11ab2b1/.github/workflows/coverage.yml'
error removing jsonschema-3.2.1.dev307+ga11ab2b1: [Errno 2] No such file or directory: 'jsonschema-3.2.1.dev307+ga11ab2b1/.github/workflows/packaging.yml'
error removing jsonschema-3.2.1.dev307+ga11ab2b1: [Errno 2] No such file or directory: 'jsonschema-3.2.1.dev307+ga11ab2b1/.github/workflows/pre-commit.yml'
error removing jsonschema-3.2.1.dev307+ga11ab2b1: [Errno 2] No such file or directory: 'jsonschema-3.2.1.dev307+ga11ab2b1/.github/workflows/ci.yml'
error removing jsonschema-3.2.1.dev307+ga11ab2b1: [Errno 2] No such file or directory: 'jsonschema-3.2.1.dev307+ga11ab2b1/.github/workflows'
error removing jsonschema-3.2.1.dev307+ga11ab2b1: [Errno 2] No such file or directory: 'jsonschema-3.2.1.dev307+ga11ab2b1/.github/SECURITY.md'
error removing jsonschema-3.2.1.dev307+ga11ab2b1: [Errno 2] No such file or directory: 'jsonschema-3.2.1.dev307+ga11ab2b1/.github'
error removing jsonschema-3.2.1.dev307+ga11ab2b1: [Errno 2] No such file or directory: 'jsonschema-3.2.1.dev307+ga11ab2b1/tox.ini'
error removing jsonschema-3.2.1.dev307+ga11ab2b1: [Errno 2] No such file or directory: 'jsonschema-3.2.1.dev307+ga11ab2b1/setup.cfg'
error removing jsonschema-3.2.1.dev307+ga11ab2b1: [Errno 2] No such file or directory: 'jsonschema-3.2.1.dev307+ga11ab2b1/README.rst'
error removing jsonschema-3.2.1.dev307+ga11ab2b1: [Errno 2] No such file or directory: 'jsonschema-3.2.1.dev307+ga11ab2b1/CHANGELOG.rst'
error removing jsonschema-3.2.1.dev307+ga11ab2b1: [Errno 2] No such file or directory: 'jsonschema-3.2.1.dev307+ga11ab2b1'
running bdist_wheel
running build
running build_py
installing to build/bdist.macosx-10.15-x86_64/wheel
running install
running install_lib
running install_egg_info
running egg_info
writing jsonschema.egg-info/PKG-INFO
writing dependency_links to jsonschema.egg-info/dependency_links.txt
writing entry points to jsonschema.egg-info/entry_points.txt
writing requirements to jsonschema.egg-info/requires.txt
writing top-level names to jsonschema.egg-info/top_level.txt
reading manifest template 'MANIFEST.in'
writing manifest file 'jsonschema.egg-info/SOURCES.txt'
removing 'build/bdist.macosx-10.15-x86_64/wheel/jsonschema-3.2.1.dev307+ga11ab2b1-py3.6.egg-info' (and everything under it)
Copying jsonschema.egg-info to build/bdist.macosx-10.15-x86_64/wheel/jsonschema-3.2.1.dev307+ga11ab2b1-py3.6.egg-info
running install_scripts
Traceback (most recent call last):
  File "/Users/julian/Development/jsonschema/.tox/py36-format-build/lib/python3.6/site-packages/pep517/_in_process.py", line 280, in <module>
    main()
  File "/Users/julian/Development/jsonschema/.tox/py36-format-build/lib/python3.6/site-packages/pep517/_in_process.py", line 263, in main
    json_out['return_val'] = hook(**hook_input['kwargs'])
  File "/Users/julian/Development/jsonschema/.tox/py36-format-build/lib/python3.6/site-packages/pep517/_in_process.py", line 205, in build_wheel
    metadata_directory)
  File "/Users/julian/Development/jsonschema/.tox/py36-format-build/lib/python3.6/site-packages/setuptools/build_meta.py", line 230, in build_wheel
    wheel_directory, config_settings)
  File "/Users/julian/Development/jsonschema/.tox/py36-format-build/lib/python3.6/site-packages/setuptools/build_meta.py", line 215, in _build_with_temp_dir
    self.run_setup()
  File "/Users/julian/Development/jsonschema/.tox/py36-format-build/lib/python3.6/site-packages/setuptools/build_meta.py", line 158, in run_setup
    exec(compile(code, __file__, 'exec'), locals())
  File "setup.py", line 1, in <module>
  File "/Users/julian/Development/jsonschema/.tox/py36-format-build/lib/python3.6/site-packages/setuptools/__init__.py", line 163, in setup
    return distutils.core.setup(**attrs)
  File "/usr/local/opt/python36/Frameworks/Python.framework/Versions/3.6/lib/python3.6/distutils/core.py", line 148, in setup
    dist.run_commands()
  File "/usr/local/opt/python36/Frameworks/Python.framework/Versions/3.6/lib/python3.6/distutils/dist.py", line 955, in run_commands
    self.run_command(cmd)
  File "/usr/local/opt/python36/Frameworks/Python.framework/Versions/3.6/lib/python3.6/distutils/dist.py", line 974, in run_command
    cmd_obj.run()
  File "/Users/julian/Development/jsonschema/.tox/py36-format-build/lib/python3.6/site-packages/wheel/bdist_wheel.py", line 342, in run
    self.egg2dist(self.egginfo_dir, distinfo_dir)
  File "/Users/julian/Development/jsonschema/.tox/py36-format-build/lib/python3.6/site-packages/wheel/bdist_wheel.py", line 445, in egg2dist
    raise ValueError(err)
ValueError: Egg metadata expected at build/bdist.macosx-10.15-x86_64/wheel/jsonschema-3.2.1.dev307+ga11ab2b1-py3.6.egg-info but not found
Traceback (most recent call last):
  File "/usr/local/opt/python36/Frameworks/Python.framework/Versions/3.6/lib/python3.6/subprocess.py", line 311, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['/Users/julian/Development/jsonschema/.tox/py36-format-build/bin/python', '/Users/julian/Development/jsonschema/.tox/py36-format-build/lib/python3.6/site-packages/pep517/_in_process.py', 'build_wheel', '/var/folders/cr/h4lk12bj5cn4qdx4ct9hddmm0000gn/T/tmpwe0_j_vx']' returned non-zero exit status 1.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/julian/Development/jsonschema/.tox/py36-format-build/lib/python3.6/site-packages/build/__init__.py", line 215, in build
    raise BuildBackendException('Backend operation failed: {}'.format(e))
build.BuildBackendException: Backend operation failed: Command '['/Users/julian/Development/jsonschema/.tox/py36-format-build/bin/python', '/Users/julian/Development/jsonschema/.tox/py36-format-build/lib/python3.6/site-packages/pep517/_in_process.py', 'build_wheel', '/var/folders/cr/h4lk12bj5cn4qdx4ct9hddmm0000gn/T/tmpwe0_j_vx']' returned non-zero exit status 1.

ERROR Backend operation failed: Command '['/Users/julian/Development/jsonschema/.tox/py36-format-build/bin/python', '/Users/julian/Development/jsonschema/.tox/py36-format-build/lib/python3.6/site-packages/pep517/_in_process.py', 'build_wheel', '/var/folders/cr/h4lk12bj5cn4qdx4ct9hddmm0000gn/T/tmpwe0_j_vx']' returned non-zero exit status 1.
ERROR: InvocationError for command /Users/julian/Development/jsonschema/.tox/py36-format-build/bin/python -m build /Users/julian/Development/jsonschema --outdir /Users/julian/Development/jsonschema/.tox/py36-format-build/tmp/dist (exited with code 1)

@pradyunsg
Copy link
Member

That looks like a a build failure in the jsonschema package, originating from within setuptools.

@Julian
Copy link
Contributor

Julian commented Oct 22, 2020

It only happens when running toxenvs in parallel (which was how the conversation started) -- everything works fine / reliably in serial.

(And yes I know this is a horrid bug report, I haven't really spent much time trying to figure out what's doing what -- but yeah it seems very much like a race condition somewhere? Obviously understandable if there's nothing to be done without diagnosing further though.)

@gaborbernat
Copy link
Author

gaborbernat commented Dec 21, 2020 •
edited

The error in question seems related to certificates. Namely when there's no certificate provided explicitly (and a network request is made here, that triggers request to provision automatically the internal certificate. This picks the temporary directory by using tempfile.gettempdir, which returns in stable fashion the same path:

>>> import tempfile
>>> tempfile.gettempdir()
'/var/folders/jh/kx1127wj32l8c3s97131rhmr0000gp/T'
>>> tempfile.gettempdir()
'/var/folders/jh/kx1127wj32l8c3s97131rhmr0000gp/T'

So there's a race condition between the start and the end of this certify extract function. For what it's worth is arguably the issue of the request library. The issue is already reported to request via psf/requests#5223.

@gaborbernat
Copy link
Author

I filled in psf/requests#5707 to solve this issue.

@pradyunsg
Copy link
Member

I guess this has been resolved at some point in the ~2 years since this was filed? We've certainly got the requests version updated.

I'll go ahead and close this, but feel welcome to holler here (or file a new issue, if this is locked) if this is still happening.

@pradyunsg pradyunsg removed S: needs triage Issues/PRs that need to be triaged state: needs eyes Needs a maintainer/triager to take a closer look labels Oct 7, 2022
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Nov 7, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@Julian @gaborbernat @brainwane @pradyunsg