Add async fixtures

[rubdos]

Would fix #86

TODO:

• Actual implementation
• Good error when async fixtures are used in non-async-tests.
• Add test with impl Trait output of a fixture (i.e., nested impl Trait); not sure whether that can work.
• Test with partials

[rubdos]

I've surely underestimated this endeavour. I'm facing a design decision now, after toying with the code for a while. I only see one option, but before I really want to implement something, I'd rather ask!

There are basically four possibilities here:

1. A synchronous test calls a synchronous fixture.
2. A synchronous test calls an async fixture.
3. An async test calls a synchronous fixture.
4. An async test calls a async fixture.

This is what happens in either case:

1. Is basically what's happening already.
2. Seems impossible without taking an assumption on the used runtime (do we agree on that?), which we shouldn't do (again, do we agree here?).
3. Is already possible
4. Is the main case that this issue/draft is about.

The way that a fixture is rendered is by generating a struct #name {}, and implementing some methods in there. An async fixture would require these methods to be async (except, again, by assuming a runtime, which we shouldn't do).

When these methods become async, as they should, the test should .await them. However, a test that uses an async fixture has no a priori knowledge about about the asyncness of the fixture -- I assumed (for too long) that the Resolver would contain that information, which it does not.

Hence my suggestion: every synchronous fixture will generate both synchronous methods (as before) and an equivalent set of asynchronous methods, prefixed or suffixed with async (that return future::ready(_);); an async fixture will only generate the async methods. A synchronous test, in turn, will call the synchronous set of methods, and an async test will call the async methods.

This way, it'll be pretty difficult to generate a clear error in the case that a synchronous test tries to call an async fixture. I'm not sure how to go about that.

Is this okay for you? If so, I can implement this tomorrow morning. If not, I'm very much open to suggestions!

[la10736]

Ok... I was going to bed but I'm trying to answer... But I've not a clear idea about that, I've just wrote the ticket to don't forget this point but I've already see that the way is not straight and clear.

I've surely underestimated this endeavour. I'm facing a design decision now, after toying with the code for a while. I only see one option, but before I really want to implement something, I'd rather ask!

There are basically four possibilities here:

1. A synchronous test calls a synchronous fixture.
2. A synchronous test calls an async fixture.
3. An async test calls a synchronous fixture.
4. An async test calls a async fixture.

I think that the 2 case is just an error. We cannot check it and we can just try to give a clear error but is not simple (there are some static check techniques to catch them but I should study them). So by now we can forget it.

My idea is that the user know exactly if he is calling an async fixture and just write the right code. If he use an async fixture value as a static value why write an async fixture? Just write a plain one.

The way that a fixture is rendered is by generating a struct #name {}, and implementing some methods in there. An async fixture would require these methods to be async (except, again, by assuming a runtime, which we shouldn't do).

Yes, my guess is that An async fixture would render all methods to be async. Again, if you try to use async fixture in a no async test code will not compile (Future type will yield lot of errors).

When these methods become async, as they should, the test should .await them. However, a test that uses an async fixture has no a priori knowledge about about the asyncness of the fixture -- I assumed (for too long) that the Resolver would contain that information, which it does not.

Again... maybe we can not care about this kind of scenarios.

Hence my suggestion: every synchronous fixture will generate both synchronous methods (as before) and an equivalent set of asynchronous methods, prefixed or suffixed with async (that return future::ready(_);); an async fixture will only generate the async methods. A synchronous test, in turn, will call the synchronous set of methods, and an async test will call the async methods.

Wait, wait... why? We don't need to transform all no async function in a fake Future. The user simply know what is async and what is not and should use it like in all other code. By this idea we force the user to handle all inputs as async. And what about inputs from parametric tests and matrix.

Today an example like

#[rstest(a, b,
    case(async { 42 }, 1),
    case(async { 21 }, 2),
    )
 ]
async fn my_async_test(a: impl Future<Output=u32>, b: u32) {
    assert_eq!(b * a.await, 42);
}

should compile and runs 2 test.

This way, it'll be pretty difficult to generate a clear error in the case that a synchronous test tries to call an async fixture. I'm not sure how to go about that.

Compiler should give already good info to understand it... we can write some example but I can bet on it.

Is this okay for you? If so, I can implement this tomorrow morning. If not, I'm very much open to suggestions!

Try first the simpler way. Maybe now the things can be simpler...

[rubdos]

Ok... I was going to bed but I'm trying to answer...

Don't feel obliged to answer! If you want to sleep, you sleep! :-)

But I've not a clear idea about that, I've just wrote the ticket to don't forget this point but I've already see that the way is not straight and clear.

I've surely underestimated this endeavour. I'm facing a design decision now, after toying with the code for a while. I only see one option, but before I really want to implement something, I'd rather ask!
There are basically four possibilities here:

1. A synchronous test calls a synchronous fixture.
2. A synchronous test calls an async fixture.
3. An async test calls a synchronous fixture.
4. An async test calls a async fixture.

I think that the 2 case is just an error. We cannot check it and we can just try to give a clear error but is not simple (there are some static check techniques to catch them but I should study them). So by now we can forget it.

Yes, only included it for "completeness", but it should be a hard error indeed.

My idea is that the user know exactly if he is calling an async fixture and just write the right code. If he use an async fixture value as a static value why write an async fixture? Just write a plain one.

The way that a fixture is rendered is by generating a struct #name {}, and implementing some methods in there. An async fixture would require these methods to be async (except, again, by assuming a runtime, which we shouldn't do).

Yes, my guess is that An async fixture would render all methods to be async. Again, if you try to use async fixture in a no async test code will not compile (Future type will yield lot of errors).

When these methods become async, as they should, the test should .await them. However, a test that uses an async fixture has no a priori knowledge about about the asyncness of the fixture -- I assumed (for too long) that the Resolver would contain that information, which it does not.

Again... maybe we can not care about this kind of scenarios.

Hence my suggestion: every synchronous fixture will generate both synchronous methods (as before) and an equivalent set of asynchronous methods, prefixed or suffixed with async (that return future::ready(_);); an async fixture will only generate the async methods. A synchronous test, in turn, will call the synchronous set of methods, and an async test will call the async methods.

Wait, wait... why? We don't need to transform all no async function in a fake Future. The user simply know what is async and what is not and should use it like in all other code. By this idea we force the user to handle all inputs as async. And what about inputs from parametric tests and matrix.

Today an example like

#[rstest(a, b,
    case(async { 42 }, 1),
    case(async { 21 }, 2),
    )
 ]
async fn my_async_test(a: impl Future<Output=u32>, b: u32) {
    assert_eq!(b * a.await, 42);
}

should compile and runs 2 test.

This way, it'll be pretty difficult to generate a clear error in the case that a synchronous test tries to call an async fixture. I'm not sure how to go about that.

Compiler should give already good info to understand it... we can write some example but I can bet on it.

Is this okay for you? If so, I can implement this tomorrow morning. If not, I'm very much open to suggestions!

Try first the simpler way. Maybe now the things can be simpler...

A good night's sleep (for me) got me to the same conclusion. This is way simpler. The only advantage of my proposal would be that the user doesn't need to manually .await. I think the simpler approach also solves the error message. If you would write

#[fixture] async fn foo() -> u32 { 42 }
#[rstest] fn bar(foo: u32) { }

the compiler will just complain that you're feeding impl Future to a function that takes u32. Should be clear enough. In my original proposal, the error message would be way more complex!

[rubdos]

The simpler way seems to work for the simplest case already. Thanks for the input :-)

[rubdos]

I'm still a bit sad that the injected fixtures are not magically awaited, but I suppose we'll have to live with that, given the complexity that it comes with.

I currently have something of the form

trait Service {}
#[fixture]
async fn create_service() -> impl Service {
    complex_async_code().await
}

#[rstest]
#[actix_rt::test]
async fn foo<T: Service>(app: impl Future<Output = T>) {
}

and it works, although it would be a lot nicer to write

#[rstest]
#[actix_rt::test]
async fn foo(app: impl Service) {
}

[la10736]

We can think (later) to introduce a attribute to remove all this boilerplate:

#[rstest]
#[actix_rt::test]
async fn foo(#[future] app: impl Service) {
}

But we cannot do it without indication: we can never know if the input is an async fixture or not. We should proceed by small steps.

[la10736]

Are you done or you would try to do more?
If you can:

1. add an entry in changelog
2. add some unit tests
3. some docs in fixture function an in readme

Otherwise I can handle them and merge your work as is.

[rubdos]

On 4/13/20 9:28 PM, la10736 wrote: Are you done or you would try to do more? If you can: 1. add an entry in changelog 2. add some unit tests 3. some docs in fixture function an in readme Otherwise I can handle them and merge your work as is.

If you can wait, I can handle some of these; I'm not sure when I'll add them though, I'm handling a few things concurrently. Otherwise, feel free to merge and add these yourself.

[la10736]

On 4/13/20 9:28 PM, la10736 wrote: Are you done or you would try to do more? If you can: 1. add an entry in changelog 2. add some unit tests 3. some docs in fixture function an in readme Otherwise I can handle them and merge your work as is.
If you can wait, I can handle some of these; I'm not sure when I'll add them though, I'm handling a few things concurrently. Otherwise, feel free to merge and add these yourself.

Yes... I can wait... Take all your time. Let me know when you're done.

THX

[svenstaro]

The boilerplate Future syntax is quite the eyesore in my async tests. Any chance for some more ergonomics for async fixtures? :)

[rubdos]

The boilerplate Future syntax is quite the eyesore in my async tests. Any chance for some more ergonomics for async fixtures? :)

having them pre-awaited would be nice indeed... I fear that it's quite a overhaul though.

[svenstaro]

Well, not even necessarily that but even just getting rid of the rather verbose impl Future<Output=MyFixtureType> would be nice. Is that possible somehow?

[rubdos]

Well, not even necessarily that but even just getting rid of the rather verbose impl Future<Output=MyFixtureType> would be nice. Is that possible somehow?

I suggest you open an issue, describing the syntax you have to use now, versus the syntax you would prefer. Feel free to tag me there, I'd love to follow the conversation. Chances are low I'd implement it myself though, it's only a "minor inconvenience" as they say :-)

[svenstaro]

Turns out there is #98 so let's go there.