XUnit ThrowsAsync<> doesn't seem to be checking the exception type. Is this a bug?

[vjmanda]

XUnit ThrowsAsync<> doesn't seem to be checking the exception type.

I am trying to check the type of the exception thrown in my unit test. However, XUnit does not seem to be not checking the exception type but only that its an Exception.

Surely the below test should fail as the exception thrown is of type NotImplementedException but the test was expecting ArgumentException. It passes as long as any kind of exception is thrown.

public class AssertThrowsTests
{
	[Fact]
	public void Tests()
	{
		// Using a method as a delegate
		Assert.ThrowsAsync<ArgumentException>(async () => await MethodThatThrows());
	}

	async Task MethodThatThrows()
	{
		await Task.Delay(100);
		throw new NotImplementedException();
	}
}

Is this a bug?

[bartelink]

I believe it's working as one might expect; I get:

Assert.Throws() Failure
Expected: typeof(System.ArgumentException)
Actual:   typeof(System.NotImplementedException): The method or operation is not implemented.

(the impl looks correct internally too, which is no surprise - lots of people use it and would have provided a repro if it was broken)
There's a specific overload wrt ArgumentException - perhaps your real problem lies there?

[vjmanda]

Apparently its due to not awaiting ThrowsAsync itself.

public class AssertThrowsTests
{
	[Fact]
	public async Task Tests()
	{
		// Using a method as a delegate
		await Assert.ThrowsAsync<NotImplementedException>(async () => await MethodThatThrows());
	}

	async Task MethodThatThrows()
	{
		await Task.Delay(100);
		throw new NotImplementedException();
	}
}

[bartelink]

Ah yes, that's a common issue

My repro was in F# and type inference did its thing, not even highlighting that issue

    let [<Fact>]t () =
        let f () : Task = task {
            do! Task.Delay 100
            NotImplementedException() |> raise
        }
        Assert.ThrowsAsync<ArgumentException>(f)