Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New plugin version for tox 4 #42

Merged
merged 27 commits into from
Dec 14, 2022
Merged
Show file tree
Hide file tree
Changes from 24 commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
f55beb7
Rename module hooks to hooks3
frenzymadness Nov 8, 2021
a223dc5
New plugin version for tox 4 alpha
frenzymadness Nov 8, 2021
bb79c6d
Remove deprecated option --print-deps-only
frenzymadness Nov 8, 2021
5d3802d
Use sysconfig instead of hardcoded paths
frenzymadness Nov 8, 2021
46fd9f0
Use the default installer (pip) even it fails in an offline env
frenzymadness Nov 8, 2021
d6cb357
Own executor is not needed if we can override config
frenzymadness Dec 13, 2021
ba4146c
Empty list of commands means no need for SystemExit
frenzymadness Dec 13, 2021
07e8536
Add dummy installer so there is no chance to install packages
frenzymadness Dec 14, 2021
799e5dd
No longer check Python versions
frenzymadness Dec 14, 2021
c5281c6
Some more default options
frenzymadness Dec 14, 2021
70a344a
Fake Python environment
frenzymadness Dec 14, 2021
dadd0a9
Use our own subprocess executor to alter env variables
frenzymadness Jan 3, 2022
14fd3c2
Compatibility with Tox 4 beta 1
frenzymadness Feb 7, 2022
0e90086
Make __init__ methods noop and use env_dir instead of a temp dir
frenzymadness Feb 8, 2022
b9030cc
Copy of the tests for Tox 4
frenzymadness Feb 9, 2022
b0923ba
Improved tests and testing with tox 4
frenzymadness Feb 10, 2022
3e9cd58
Remove unused imports
frenzymadness Feb 10, 2022
463c745
Drop support for tox 3.15
frenzymadness Feb 25, 2022
5d5286d
Drop tox < 3.24, add Python 3.11, fix tox 4 compatibility
frenzymadness Nov 30, 2022
f5beafc
Fix tests
frenzymadness Nov 30, 2022
a72e7ba
Fix TOX_MIN_VERSION again
frenzymadness Dec 12, 2022
ad6bcf5
Fix for commands_pre and commands_post for tox 4
frenzymadness Dec 12, 2022
f71dd2a
Skip tox 3/4 tests with tox 4/3 via pytest, not tox
hroncok Dec 12, 2022
028ee0c
Avoid race conditions when running tests with xdist
hroncok Dec 13, 2022
c3497d2
Don't assume the tests allways run via tox
hroncok Dec 14, 2022
679fbc9
Always set passenv/pass_env to *
hroncok Dec 14, 2022
3438c56
Release 0.0.9
hroncok Dec 14, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
33 changes: 17 additions & 16 deletions .github/workflows/main.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -22,21 +22,22 @@ jobs:
# This information is repeated in tox.ini
# (see https://github.com/fedora-python/tox-github-action/issues/8)
# Generate it by: tox -l | sed "s/^/- /"
- py36-toxrelease
- py36-toxmaster
- py36-tox315
- py37-toxrelease
- py37-toxmaster
- py37-tox315
- py38-toxrelease
- py38-toxmaster
- py38-tox315
- py39-toxrelease
- py39-toxmaster
- py39-tox315
- py310-toxrelease
- py310-toxmaster
- py310-tox315

- py36-tox324
- py36-tox3
- py37-tox324
- py37-tox3
- py37-tox4
- py38-tox324
- py38-tox3
- py38-tox4
- py39-tox324
- py39-tox3
- py39-tox4
- py310-tox324
- py310-tox3
- py310-tox4
- py311-tox324
- py311-tox3
- py311-tox4
# Use GitHub's Linux Docker host
runs-on: ubuntu-latest
21 changes: 16 additions & 5 deletions README.rst
Original file line number Diff line number Diff line change
Expand Up @@ -157,6 +157,17 @@ To get a list of names of extras, run:
Caveats, warnings and limitations
---------------------------------

tox 4
~~~~~

The plugin is available also for tox 4. Differences in behavior between tox 3 and 4 are these:

- ``--recreate`` is no longer needed when you switch from the plugin back to standard tox. Tox
detects it and handles the recreation automatically.
- The plugin does not check the requested Python version nor the environment name. If you let
it run for multiple environments they'll all use the same Python.
- Deprecated ``--print-deps-only`` option is no longer available.

Use an isolated environment
~~~~~~~~~~~~~~~~~~~~~~~~~~~

Expand Down Expand Up @@ -221,20 +232,20 @@ installs (a newer version of) ``tox`` and the missing packages
into that environment and proxies all ``tox`` invocations trough that.
Unfortunately, this is undesired for ``tox-current-env``.

1. Starting with ``tox`` 3.23, it is possible to invoke it as
``tox --no-provision`` to prevent the provision entirely.
1. It is possible to invoke ``tox`` with ``--no-provision``
to prevent the provision entirely.
When requirements are missing, ``tox`` fails instead of provisioning.
If a path is passed as a value for ``--no-provision``,
the requirements will be serialized to the file, as JSON.
2. Starting with ``tox`` 3.22, the requires, if specified, are included in the
2. The requires, if specified, are included in the
results of ``tox --print-deps-to``.
This only works when they are installed (otherwise see the first point).
3. The minimal tox version, if specified, is included in the results of
``tox --print-deps-to`` (as ``tox >= X.Y.Z``).
``tox --print-deps-to``.
This only works when the version requirement is satisfied
(otherwise see the first point).

With ``tox >= 3.23``, the recommend way to handle this is:
The recommend way to handle this is:

1. Run ``tox --no-provision provision.json --print-deps-to=...`` or similar.
2. If the command fails, install requirements from ``provision.json`` to the
Expand Down
3 changes: 2 additions & 1 deletion setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ def long_description():
packages=find_packages("src"),
entry_points={"tox": ["current-env = tox_current_env.hooks"]},
install_requires=[
"tox>=3.15,<4",
"tox>=3.24",
"importlib_metadata; python_version < '3.8'"
],
extras_require={
Expand All @@ -42,6 +42,7 @@ def long_description():
"Programming Language :: Python :: 3.8",
"Programming Language :: Python :: 3.9",
"Programming Language :: Python :: 3.10",
"Programming Language :: Python :: 3.11",
"Programming Language :: Python :: Implementation :: CPython",
"Topic :: Software Development :: Testing",
],
Expand Down
288 changes: 5 additions & 283 deletions src/tox_current_env/hooks.py
Original file line number Diff line number Diff line change
@@ -1,284 +1,6 @@
import os
import shutil
import subprocess
import sys
import tox
import warnings
import argparse
from tox import __version__ as TOX_VERSION

try:
import importlib.metadata as importlib_metadata
except ImportError:
import importlib_metadata


@tox.hookimpl
def tox_addoption(parser):
parser.add_argument(
"--current-env",
action="store_true",
dest="current_env",
default=False,
help="Run tests in current environment, not creating any virtual environment",
)
parser.add_argument(
"--print-deps-only",
action="store_true",
dest="print_deps_only",
default=False,
help="Deprecated, equivalent to `--print-deps-to -`",
)
parser.add_argument(
"--print-deps-to",
"--print-deps-to-file",
action="store",
type=argparse.FileType('w'),
metavar="FILE",
default=None,
help="Don't run tests, only print the dependencies to the given file "
+ "(use `-` for stdout)",
)
parser.add_argument(
"--print-extras-to",
"--print-extras-to-file",
action="store",
type=argparse.FileType('w'),
metavar="FILE",
default=None,
help="Don't run tests, only print the names of the required extras to the given file "
+ "(use `-` for stdout)",
)


def _plugin_active(option):
return option.current_env or option.print_deps_to or option.print_extras_to


def _allow_all_externals(envconfig):
for option in ["allowlist_externals", "whitelist_externals"]:
# If either is set, we change it, as we cannot have both set at the same time:
if getattr(envconfig, option, None):
setattr(envconfig, option, "*")
break
else:
# If none was set, we set one of them, preferably the new one:
if hasattr(envconfig, "allowlist_externals"):
envconfig.allowlist_externals = "*"
else:
# unless we need to fallback to the old and deprecated
# TODO, drop this when we drop support for tox < 3.18
envconfig.whitelist_externals = "*"

@tox.hookimpl
def tox_configure(config):
"""Stores options in the config. Makes all commands external and skips sdist"""
if config.option.print_deps_only:
warnings.warn(
"--print-deps-only is deprecated; use `--print-deps-to -`",
DeprecationWarning,
)
if not config.option.print_deps_to:
config.option.print_deps_to = sys.stdout
else:
raise tox.exception.ConfigError(
"--print-deps-only cannot be used together "
+ "with --print-deps-to"
)
if _plugin_active(config.option):
config.skipsdist = True
for testenv in config.envconfigs:
config.envconfigs[testenv].usedevelop = False
_allow_all_externals(config.envconfigs[testenv])

# When printing dependencies/extras we don't run any commands.
# Unfortunately tox_runtest_pre/tox_runtest_post hooks don't use firstresult=True,
# so we cannot override running commands_pre/commands_post.
# We empty the lists of commands instead.
if config.option.print_deps_to or config.option.print_extras_to:
for testenv in config.envconfigs:
config.envconfigs[testenv].commands_pre = []
config.envconfigs[testenv].commands_post = []

if (getattr(config.option.print_deps_to, "name", object()) ==
getattr(config.option.print_extras_to, "name", object())):
raise tox.exception.ConfigError(
"The paths given to --print-deps-to and --print-extras-to cannot be identical."
)

return config


class InterpreterMismatch(tox.exception.InterpreterNotFound):
"""Interpreter version in current env does not match requested version"""


def _python_activate_exists(venv):
python = venv.envconfig.get_envpython()
bindir = os.path.dirname(python)
activate = os.path.join(bindir, "activate")
return os.path.exists(python), os.path.exists(activate)


def is_current_env_link(venv):
python, activate = _python_activate_exists(venv)
return python and not activate


def is_proper_venv(venv):
python, activate = _python_activate_exists(venv)
return python and activate


def is_any_env(venv):
python, activate = _python_activate_exists(venv)
return python


def rm_venv(venv):
link = venv.envconfig.get_envpython()
shutil.rmtree(os.path.dirname(os.path.dirname(link)), ignore_errors=True)


def unsupported_raise(config, venv):
if config.option.recreate:
return
if not _plugin_active(config.option) and is_current_env_link(venv):
if hasattr(tox.hookspecs, "tox_cleanup"):
raise tox.exception.ConfigError(
"Looks like previous --current-env, --print-deps-to or --print-extras-to tox run didn't finish the cleanup. "
"Run tox run with --recreate (-r) or manually remove the environment in .tox."
)
else:
raise tox.exception.ConfigError(
"Regular tox run after --current-env, --print-deps-to or --print-extras-to tox run "
"is not supported without --recreate (-r)."
)
elif config.option.current_env and is_proper_venv(venv):
raise tox.exception.ConfigError(
"--current-env after regular tox run is not supported without --recreate (-r)."
)


@tox.hookimpl
def tox_testenv_create(venv, action):
"""We create a fake virtualenv with just the symbolic link"""
config = venv.envconfig.config
create_fake_env = check_version = config.option.current_env
if config.option.print_deps_to or config.option.print_extras_to:
if is_any_env(venv):
# We don't need anything
return True
else:
# We need at least some kind of environment,
# or tox fails without a python command
# We fallback to --current-env behavior,
# because it's cheaper, faster and won't install stuff
create_fake_env = True
if check_version:
# With real --current-env, we check this, but not with --print-deps/extras-to only
version_info = venv.envconfig.python_info.version_info
if version_info is None:
raise tox.exception.InterpreterNotFound(venv.envconfig.basepython)
if version_info[:2] != sys.version_info[:2]:
raise InterpreterMismatch(
f"tox_current_env: interpreter versions do not match:\n"
+ f" in current env: {tuple(sys.version_info)}\n"
+ f" requested: {version_info}"
)
if create_fake_env:
# Make sure the `python` command on path is sys.executable.
# (We might have e.g. /usr/bin/python3, not `python`.)
# Remove the rest of the virtualenv.
link = venv.envconfig.get_envpython()
target = sys.executable
rm_venv(venv)
os.makedirs(os.path.dirname(link))
if sys.platform == "win32":
# Avoid requiring admin rights on Windows
subprocess.check_call(f'mklink /J "{link}" "{target}"', shell=True)
else:
os.symlink(target, link)
# prevent tox from creating the venv
return True
if not is_proper_venv(venv):
rm_venv(venv)
return None # let tox handle the rest


@tox.hookimpl
def tox_package(session, venv):
"""Fail early when unsupported"""
config = venv.envconfig.config
unsupported_raise(config, venv)


@tox.hookimpl
def tox_testenv_install_deps(venv, action):
"""We don't install anything"""
config = venv.envconfig.config
unsupported_raise(config, venv)
if _plugin_active(config.option):
return True


def tox_dependencies(config):
"""Get dependencies of tox itself, 'minversion' and 'requires' config options"""
if config.minversion is not None:
yield f"tox >= {config.minversion}"
# config does not have the "requires" attribute until tox 3.22:
yield from getattr(config, "requires", [])


@tox.hookimpl
def tox_runtest(venv, redirect):
"""If --print-deps-to, prints deps instead of running tests.
If --print-extras-to, prints extras instead of running tests.
Both options can be used together."""
config = venv.envconfig.config
unsupported_raise(config, venv)
ret = None

if config.option.print_deps_to:
print(
*tox_dependencies(config),
*venv.get_resolved_dependencies(),
sep="\n",
file=config.option.print_deps_to,
)
config.option.print_deps_to.flush()
ret = True

if config.option.print_extras_to:
print(
*venv.envconfig.extras,
sep="\n",
file=config.option.print_extras_to,
)
config.option.print_extras_to.flush()
ret = True

return ret


@tox.hookimpl
def tox_cleanup(session):
"""Remove the fake virtualenv not to collide with regular tox
Collisions can happen anyway (when tox is killed forcefully before this happens)
Note that we don't remove real venvs, as recreating them is expensive"""
for venv in session.venv_dict.values():
if is_current_env_link(venv):
rm_venv(venv)


@tox.hookimpl
def tox_runenvreport(venv, action):
"""Prevent using pip to display installed packages,
use importlib.metadata instead, but fallback to default without our flags."""
if not _plugin_active(venv.envconfig.config.option):
return None
return (
"{}=={}".format(d.metadata.get("name"), d.version)
for d in sorted(
importlib_metadata.distributions(), key=lambda d: d.metadata.get("name")
)
)
if TOX_VERSION[0] == "4":
from tox_current_env.hooks4 import *
else:
from tox_current_env.hooks3 import *
frenzymadness marked this conversation as resolved.
Show resolved Hide resolved