Incompatibility with rstest's once

[jacg]

Thank you very much for this project, it really makes the testing experience in Rust much more fun and productive.

I've come across what I suspect to be an incompatibility between nextest and another crate that helps with testing in Rust: rstest, specifically its once fixture attribute, whose purpose is to ensure that a fixture is executed only once per test run.

I use rstest's fixture in some tests, and wanted to write the data created by the fixture to a file, to be able to inspect them manually after the tests run. When using cargo nextest the fixture (which is marked with #[once] and hence should only run once in the whole test run) fails because it tries to write to the file multiple times simultaneously. Executing the same test suite with plain cargo test does not suffer from this problem. So it looks like nextest execution model is working around the rstest #[once] mechanism for ensuring single execution.

Is there any chance that these two excellent projects might be made usable together, in this particular instance?

I've opened a similar issue in rstest.

[la10736]

I guess that use one process per test makes useless #[once] attribute. I think that there are scenario where instantiate some resource shared across tests (like docker instances or populate db) only once is really useful... in this case the model one process one test don't works very well.

Is a chance introduce an option to run tests in the same executable with multiple thread that can share resources? I know why you adopt this model and all benefits that this model have, but this kinds of scenario are really common in integration tests.

[sunshowers]

Thanks for the report and the debugging! Yes, one process per test is incompatible with that kind of initialization. The options are:

• Initialize separate, isolated universes for each test, which can work with files on disk and some database situations, but is definitely harder to do with Docker by my understanding.
• Combine all the individual tests into a single #[test] so that it's all run together.

#27 tracks adding the option to add a mode where all tests are run within the same process. It is quite nontrivial, however -- one of the challenges is that individual test data isn't exposed by stable Rust. Regardless, I would be happy to welcome a pull request that does something reasonable in that case.

I think I'm going to close this as a duplicate of #27, since solving that should address this.

[jacg]

Separate, isolated universes for each test, would indeed be useful for avoiding collisions (and in the case of files colliding, this can and should be addressed with some unique filename-generation utility), but #[once]-like features are mostly useful for sharing resources between tests, either because setting up the resource is expensive, or because the tests are somehow coupled by the resource.

Combining tests into an individual test would be a possibility in the latter cases, but then you lose the benefits of fine granularity of failure reporting that comes with small, separate tests. Especially annoying would be the loss of parametrized tests (such as rstest's cases, where different data are injected into the same test).

But these are second-order details. I appreciate that it's difficult, and that solving #27 should address this, so I agree with closing.

Thanks again for the excellent crate!