How to delay running tests until master is fully configured?

[ssbarnea]

I need to do some preparatory work before effectively running the tests and this is currently done like:

def pytest_configure(config: pytest.Config) -> None:
    """Ensure we run preparation only on master thread when running in parallel."""
    if is_master(config):
        print("doing extra work, may take 5s or more seconds!")

def is_master(config: pytest.Config) -> bool:
    """Return true if is run on master thread."""
    return not hasattr(config, "workerinput")

The problem is that on some systems, like github actions, I see tests failing in a way that makes be believe that the effective execution of the tests on nodes happens before master is finishing its pytest_configure execution.

How can I ensure that tests do not start too early?

[RonnyPfannschmidt]

its technically impossible to have a worker run beore the coordinator is configured

more details needed

[ssbarnea]

Thanks for confirming this. I think that my deadlock issue is caused by some other code that runs later, likely a test.

What makes the situation even more confusing is that I detected a new problem that happens only on Ubuntu (22.04 or 23.04): getting stuck during xdist collection with 3+ workers. With 2 it passes collection but with 3 it gets stuck: collecting: 2/3 workers. The VM has 4 cpus. Running the same test suite on Fedora or MacOS does not show these behaviors.

Still, I have no idea what how to debug this as increasing verbosity level does not help at all.

[RonnyPfannschmidt]

It may be related to thread affinity

Is the verbose flag used?, Unfortunately with verbose we have a Code path that trigger's pytest outside of the main thread