Set tests order or dependencies

[AymanElsayeed]

The goal: To be able to set a test order

Usecase:

When i have a test, its input or results depend on the data (or procedure on an object ) that other tests process, regardless of the result.

Example:

A test (test-A) during the test, a table in a database updated ( as a sequence of operations on some object)
test-B (assuming test-A finished regardless of its result) checks whether or not the database has been updated.

My case:

I have a list of objects, and different tests are executed on those objects.

As a result of the test execution on those objects, a specific data field is updated ( a state as a result of sending the object to an external system for processing data).

When all tests finish, I want to run a test that checks the data filed on each object.

What i am avoiding

• Using any mechanism like Inter-process communication (IPC), e.x, Redis, files
• Time-based mechanism with a timeout.

Notes/constraints:

• Not all objects will be used on all tests; not all tests will be executed.
• For each test, an object can be added/removed
• Test execution depends on a mark
• Some objects will be used in multiple tests
• The running time of each test depends on external system resources, etc.

[RonnyPfannschmidt]

there is no builtin mechanism for test dependencies , im not aware of a plugin that also supports xdist, xdist currently doesnt support passing over the necesary annotations and making use of them

[RonnyPfannschmidt]

This needs a design expansion as currently there's no node metadata being transferred

[AymanElsayeed]

I can start with that, transfer metadata as a prototype, see what challenges arise, and evaluate the effort needed based on that.
What do you think?

In the meantime, Can we check if this is needed in the community and see if the effort is worth it?

[RonnyPfannschmidt]

Its needed for load group scheduling as well as parameter scope affinity

[RonnyPfannschmidt]

Note that some kind of design discussion should happen before implementing this

[AymanElsayeed]

Sure, thanks. Do you have any design diagrams you can share that can help me to navigate the project? or another kind of document?

[albertino87]

I'm also interested in this feature. To maximize the running efficiency of the tests running in parallel it is important that the slower tests run first, and i do it by marking the tests with the decorator @pytest.mark.order(0) from the pytest-order library. however this is not respected by xdist, can you possibly fix it?