coverage 6.3 failed for parallel execution

[MeWu-IDM]

Describe the bug
A clear and concise description of the bug.

To Reproduce
How can we reproduce the problem? Please be specific. Don't link to a failing CI job. Answer the questions below:

1. What version of Python are you using?
   3.8.12

2. What version of coverage.py shows the problem? The output of coverage debug sys is helpful.
   6.3

3. What versions of what packages do you have installed? The output of pip freeze is helpful.
   pytest =6.2.5
   coverage=6.3

4. What code shows the problem? Give us a specific commit of a specific repo that we can check out. If you've already worked around the problem, please provide a commit before that fix.

5. What commands did you run?
   pytest --cov-config=.coveragerc --cov=../our_code --workers auto --durations=0

Expected behavior
It should run tests in parallel and generate code coverage

Additional context
Getting error messages

_```
______________ ERROR at setup of TestTracing.test_trace_results _______________

self = <pytest_cov.plugin.CovPlugin object at 0x7fc911671040>
item =

def pytest_runtest_setup(self, item):
    if os.getpid() != self.pid:
        # test is run in another process than session, run
        # coverage manually

      embed.init()

/opt/hostedtoolcache/Python/3.8.12/x64/lib/python3.8/site-packages/pytest_cov/plugin.py:350:

---

/opt/hostedtoolcache/Python/3.8.12/x64/lib/python3.8/site-packages/pytest_cov/embed.py:74: in init
cov.start()
/opt/hostedtoolcache/Python/3.8.12/x64/lib/python3.8/site-packages/coverage/control.py:567: in start
self._init_for_start()
/opt/hostedtoolcache/Python/3.8.12/x64/lib/python3.8/site-packages/coverage/control.py:536: in _init_for_start
self._old_sigterm = signal.signal(signal.SIGTERM, self._on_sigterm)

---

signalnum = <Signals.SIGTERM: 15>
handler = <bound method Coverage._on_sigterm of <coverage.control.Coverage object at 0x7fc8ed755ee0>>

@_wraps(_signal.signal)
def signal(signalnum, handler):

  handler = _signal.signal(_enum_to_int(signalnum), _enum_to_int(handler))

E ValueError: signal only works in main thread

/opt/hostedtoolcache/Python/3.8.12/x64/lib/python3.8/signal.py:47: ValueError
_______________ ERROR at setup of TestDistributions.test_exp2inf _______________

self = <pytest_cov.plugin.CovPlugin object at 0x7fc911671040>
item =

def pytest_runtest_setup(self, item):
    if os.getpid() != self.pid:
        # test is run in another process than session, run
        # coverage manually

      embed.init()

/opt/hostedtoolcache/Python/3.8.12/x64/lib/python3.8/site-packages/pytest_cov/plugin.py:350:

---

/opt/hostedtoolcache/Python/3.8.12/x64/lib/python3.8/site-packages/pytest_cov/embed.py:74: in init
cov.start()
/opt/hostedtoolcache/Python/3.8.12/x64/lib/python3.8/site-packages/coverage/control.py:567: in start
self._init_for_start()
/opt/hostedtoolcache/Python/3.8.12/x64/lib/python3.8/site-packages/coverage/control.py:536: in _init_for_start
self._old_sigterm = signal.signal(signal.SIGTERM, self._on_sigterm)

---

signalnum = <Signals.SIGTERM: 15>
handler = <bound method Coverage._on_sigterm of <coverage.control.Coverage object at 0x7fc8ed7de5e0>>

@_wraps(_signal.signal)
def signal(signalnum, handler):

  handler = _signal.signal(_enum_to_int(signalnum), _enum_to_int(handler))

E ValueError: signal only works in main thread

/opt/hostedtoolcache/Python/3.8.12/x64/lib/python3.8/signal.py:47: ValueError

[cliffckerr]

Possibly also relevant:

• This is run with --workers auto as provided by pytest-parallel
• It works on Windows, but fails on Ubuntu

Sounds like it could be related to #1310 -- also an issue with multiprocessing and coverage 6.3

[cliffckerr]

Here's a minimal reproducible example.

test_simple.py:

def test_simple():
    print('OK')

requirements.txt:

pytest
pytest-parallel
pytest-cov
coverage==6.3

Then run:

pip install -r requirements.txt
pytest --cov=. --workers auto

This works with coverage==6.2, but 6.3 gives

================================================ test session starts ================================================
platform linux -- Python 3.9.7, pytest-6.2.5, py-1.11.0, pluggy-1.0.0
rootdir: /temp/tmpcov
plugins: cov-3.0.0, forked-1.4.0, parallel-0.1.1
collected 1 item                                                                                                    
pytest-parallel: 12 workers (processes), 1 test per worker (thread)
EE
====================================================== ERRORS =======================================================
___________________________________________ ERROR at setup of test_simple ___________________________________________

self = <pytest_cov.plugin.CovPlugin object at 0x7f6305f37e80>, item = <Function test_simple>

    def pytest_runtest_setup(self, item):
        if os.getpid() != self.pid:
            # test is run in another process than session, run
            # coverage manually
>           embed.init()

/software/anaconda/envs/tmpcov2/lib/python3.9/site-packages/pytest_cov/plugin.py:350: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/software/anaconda/envs/tmpcov2/lib/python3.9/site-packages/pytest_cov/embed.py:74: in init
    cov.start()
/software/anaconda/envs/tmpcov2/lib/python3.9/site-packages/coverage/control.py:567: in start
    self._init_for_start()
/software/anaconda/envs/tmpcov2/lib/python3.9/site-packages/coverage/control.py:536: in _init_for_start
    self._old_sigterm = signal.signal(signal.SIGTERM, self._on_sigterm)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

signalnum = <Signals.SIGTERM: 15>
handler = <bound method Coverage._on_sigterm of <coverage.control.Coverage object at 0x7f6305cf4df0>>

    @_wraps(_signal.signal)
    def signal(signalnum, handler):
>       handler = _signal.signal(_enum_to_int(signalnum), _enum_to_int(handler))
E       ValueError: signal only works in main thread of the main interpreter

/software/anaconda/envs/tmpcov2/lib/python3.9/signal.py:47: ValueError

----------- coverage: platform linux, python 3.9.7-final-0 -----------
Name             Stmts   Miss  Cover
------------------------------------
test_simple.py       2      1    50%
------------------------------------
TOTAL                2      1    50%

============================================== short test summary info ==============================================
ERROR test_simple.py::test_simple - ValueError: signal only works in main thread of the main interpreter
================================================= 1 error in 0.14s ==================================================

System info:

Linux 5.4.0-96-generic #109-Ubuntu SMP Wed Jan 12 16:49:16 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux

Pip freeze:

attrs==21.4.0
certifi==2021.10.8
coverage==6.3
execnet==1.9.0
iniconfig==1.1.1
packaging==21.3
pluggy==1.0.0
py==1.11.0
pyparsing==3.0.7
pytest==6.2.5
pytest-cov==3.0.0
pytest-forked==1.4.0
pytest-parallel==0.1.1
tblib==1.7.0
toml==0.10.2
tomli==2.0.0

[cliffckerr]

I could be wrong, but I think this commit might be what introduced the bug: dd575ee

[nedbat]

Thanks, sorry about that. Can you try installing a branch of coverage to see if it fixes your full system?

pip install git+https://github.com/nedbat/coveragepy@nedbat/fix-1312

[MeWu-IDM]

thanks for the prompt response, I tested coverage-6.3.1a0 it seems to work!

[nedbat]

This is fixed in commit 00da68e

[tunetheweb]

I just ran my tests against git+https://github.com/nedbat/coveragepy@master and still seeing the same issue unfortunately.

[nedbat]

@tunetheweb just to be clear, you are talking about the problem reported in #1310, right?

[tunetheweb]

Sorry yes. I presumed they were the same thing.

[MeWu-IDM]

btw, just tested again git+https://github.com/nedbat/coveragepy@master and it seems the run
pytest --cov-config=.coveragerc --cov=../our_code --workers auto --durations=0
takes much longer (we are using this in CI and it used to take less than 2 mins before 6.3 and now it hits 5 minute timeout), with the fix it does seem to be running some tests without the error described above but much much more slower.

[cliffckerr]

Interesting @MeWu-IDM, I'm not able to reproduce the performance issue. The Covasim tests take 10.8 s on coverage==6.2, and 10.9 s on current master. (This is on my Ubuntu laptop.)

[nedbat]

This is now released as part of coverage 6.3.1.