Package destroyed before use in environments when recreate=True

[raffienficiaud]

Issue

When specifying several environments from the command line, there are cases where the built wheel is "cleaned" before the command runs.

rm -rf .tox
# works always
tox -r -e py311,py311-checks

# fails on the second target
rm -rf .tox
tox -e py311,py311-checks

The reason is:

• py311-checks has recreate = True
• py311 and py311-checks both rely on the same .pkg folder
• the .pkg folder is deleted after running py311 and before running py311-checks when -r is not specified on the command line

Environment

Provide at least:

• OS: macOS M2, python3.9,3.10,3.11

Output of pip list of the host Python, where tox is installed

$ pip list
Package       Version
------------- -------
cachetools    5.3.2
chardet       5.2.0
colorama      0.4.6
distlib       0.3.7
filelock      3.13.0
iniconfig     2.0.0
packaging     23.2
pip           23.3.1
platformdirs  3.11.0
pluggy        1.3.0
pyproject-api 1.6.1
pytest        7.4.3
setuptools    65.5.0
tox           4.11.3
virtualenv    20.24.6

Output of running tox

Output of tox -rvv

.pkg: _optional_hooks> python /Volumes/data/code/sandbox/venv_check_tox_issue/lib/python3.11/site-packages/pyproject_api/_backend.py True setuptools.build_meta
.pkg: get_requires_for_build_wheel> python /Volumes/data/code/sandbox/venv_check_tox_issue/lib/python3.11/site-packages/pyproject_api/_backend.py True setuptools.build_meta
.pkg: build_wheel> python /Volumes/data/code/sandbox/venv_check_tox_issue/lib/python3.11/site-packages/pyproject_api/_backend.py True setuptools.build_meta
py311: install_package> python -I -m pip install --force-reinstall --no-deps /Volumes/data/code/sandbox/eob-31-tox-issue/.tox/.tmp/package/17/tox_issue_check-0.0.0-py3-none-any.whl
py311: commands[0]> python -c 'print('"'"'should work'"'"')'
should work
py311-checks: remove tox env folder /Volumes/data/code/sandbox/eob-31-tox-issue/.tox/py311-checks
py311: OK ✔ in 0.52 seconds
.pkg: remove tox env folder /Volumes/data/code/sandbox/eob-31-tox-issue/.tox/.pkg
py311-checks: install_deps> python -I -m pip install 'pytest>=7'
.pkg: install_requires> python -I -m pip install 'setuptools>=61.0'
.pkg: install_requires_for_build_wheel> python -I -m pip install wheel
py311-checks: internal error
Traceback (most recent call last):
  File "/Volumes/data/code/sandbox/venv_check_tox_issue/lib/python3.11/site-packages/tox/session/cmd/run/single.py", line 46, in _evaluate
    tox_env.setup()
  File "/Volumes/data/code/sandbox/venv_check_tox_issue/lib/python3.11/site-packages/tox/tox_env/api.py", line 248, in setup
    self._setup_with_env()
  File "/Volumes/data/code/sandbox/venv_check_tox_issue/lib/python3.11/site-packages/tox/tox_env/runner.py", line 145, in _setup_with_env
    self._setup_pkg()
  File "/Volumes/data/code/sandbox/venv_check_tox_issue/lib/python3.11/site-packages/tox/tox_env/runner.py", line 168, in _setup_pkg
    self._packages = self._build_packages()
                     ^^^^^^^^^^^^^^^^^^^^^^
  File "/Volumes/data/code/sandbox/venv_check_tox_issue/lib/python3.11/site-packages/tox/tox_env/python/runner.py", line 119, in _build_packages
    packages = package_env.perform_packaging(self.conf)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Volumes/data/code/sandbox/venv_check_tox_issue/lib/python3.11/site-packages/tox/tox_env/package.py", line 45, in _func
    return meth(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/Volumes/data/code/sandbox/venv_check_tox_issue/lib/python3.11/site-packages/tox/tox_env/python/virtual_env/package/pyproject.py", line 260, in perform_packaging
    wheel = create_session_view(wheel, self._package_temp_path)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Volumes/data/code/sandbox/venv_check_tox_issue/lib/python3.11/site-packages/tox/util/file_view.py", line 35, in create_session_view
    shutil.copyfile(package, session_package)
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/shutil.py", line 256, in copyfile
    with open(src, 'rb') as fsrc:
         ^^^^^^^^^^^^^^^
FileNotFoundError: [Errno 2] No such file or directory: '/Volumes/data/code/sandbox/eob-31-tox-issue/.tox/.pkg/dist/tox_issue_check-0.0.0-py3-none-any.whl'
.pkg: _exit> python /Volumes/data/code/sandbox/venv_check_tox_issue/lib/python3.11/site-packages/pyproject_api/_backend.py True setuptools.build_meta
  py311: OK (0.52=setup[0.51]+cmd[0.02] seconds)
  py311-checks: FAIL code 2 (2.10 seconds)
  evaluation failed :( (2.66 seconds)

Minimal example

tox.ini

[tox]
requires =
    tox>=4
env_list =
    py{39,310,311}
    py{39,310,311}-checks
skip_missing_interpreters = true

[default]

[pkgenv]

[testenv]
description = "runs unit tests"
package = wheel
deps =
    pytest>=7
commands =
    python -c "print('should work')"


[testenv:py{39,310,311}-checks]
description = "checks"
package = wheel
recreate = true
# bug in tox: workaround: specify another package_env for this target
# package_env = .pkg-checks
deps =
    pytest>=7
commands_pre =
    python  -c "print('does not work')"

commands =
    # tests related to the package
    pytest --version
    pip list

Let me know if you need the package, but the one I tested against does nothing but print a message.

[raffienficiaud]

Possible workaround is to specify another package_env like the one commented out in the second target. The problem is that they can still do not share the same wheel.