Select which test should be distributed and which test should be run without concurrence

[mscheltienne]

I would like to use pytest-xdist in mne-lsl: mne-tools/mne-lsl#229
However, in the test suite, some test need to be run while no other test are running because they spawn network resources which might be pick-up by other tests. Example:

def test1(...):
    spawn_network_resources()
    ...

def test2(...):
    resources = resolve_all_network_resources()
    assert len(resources) == 0
    spawn_network_resources()
    resources = resolve_all_network_resources()
    assert len(resources) == 1

In this example, test2 would be affected by test1 run in a different worker.
To avoid this problem, I would like new markers from pytest-xdist to mark which test should not be distributed and should be run through --dist no, either before or after all the other tests distributed to workers.

e.g.

@pytest.mark.no_xdist
def test2(...):

My current alternative idea is to create this marker and to run 2 separate pytest command.

[mscheltienne]

The difference between this request and my alternative lies that developers won't accidently run the test suit with xdist and without the -m "not no_xdist". The -m flag is not needed anymore as xdist will handle dispatching correctly.

[RonnyPfannschmidt]

When xdist is active, no tests run in the controling process

There's no plan for a meta scheduler that ensures noncoop tests are executed on a single node in isolation

[mscheltienne]

OK thank you! Could that be a feature request then? Some test need to be run in isolation; and preventing an entire test suite to run on xdist because of a minority is disappointing.

If there is no plan/desire to add this feature, feel free to close this issue!

[RonnyPfannschmidt]

there is a desire to add it, but lack of engineering time to build it