Skip to content

Commit

Permalink
python3Packages.pytest-xdist: Disable fork safety on darwin
Browse files Browse the repository at this point in the history
We run pytest with `--forked` in nixpkgs, to reduce side effects that
can occur when multiple tests mutate their environment in incompatible
ways.

Forking on macOS 10.13 and later is unsafe when an application does work
between calls to fork() and its followup exec(). This may lead to
crashes when calls into the Objective-C runtime are issued, which will
in turn coredump the Python interpreter.

One good reproducer for this scenario is when the urllib module tries
to lookup proxy configurations in `urllib.request.getproxies()` through
`get_proxies_macos_sysconf` into the native `_scproxy` module.

This is a class of issues that is of course not limited to the urllib
module. The general recommendation is to use `spawn` instead of `fork`,
but we don't have any influence on upstream developers to do one or the
other.

One often cited workaround would be to disable fork safety entirely on
calls to `initialize()`, which is probably a better solution than
running without multithreading (slow) or without the `--forked` (prone
to side effects) mode.

This currently happens on aarch64-linux only, where we use more recent
11.0 SDK version, while x86_64-darwin has been stuck on 10.12 for a
while now.

python/cpython#77906 (comment)
http://www.sealiesoftware.com/blog/archive/2017/6/5/Objective-C_and_fork_in_macOS_1013.html

Closes: NixOS#194290
  • Loading branch information
mweinelt committed Oct 4, 2022
1 parent 520f8b8 commit 2ba40b5
Showing 1 changed file with 8 additions and 0 deletions.
8 changes: 8 additions & 0 deletions pkgs/development/python-modules/pytest-xdist/setup-hook.sh
Expand Up @@ -3,6 +3,14 @@ pytestXdistHook() {
"--numprocesses=$NIX_BUILD_CORES"
"--forked"
)

# Using --forked on darwin leads to crashes when fork safety is
# enabled. This often happens when urllib tries to request proxy
# settings on MacOS through `urllib.request.getproxies()`
# - https://github.com/python/cpython/issues/77906
if [[ "$OSTYPE" == "darwin"* ]]; then
export OBJC_DISABLE_INITIALIZE_FORK_SAFETY=YES
fi
}

# the flags should be added before pytestCheckHook runs so
Expand Down

0 comments on commit 2ba40b5

Please sign in to comment.