Semantic comparison of JSON with Verify

[egil]

Hey all,

Recently did a session at the Øredev conference where I show how I like to test APIs. I use Verify for snapshot testing, and did make some changes to how it handles JSON. In particular, I leverage a semantic comparison library that understands JSON instead of using string based comparison.

Find the code here, and if anybody has any suggestions for improvements, please let me know: https://github.com/egil/ApiTestingAZ

@SimonCropp I do have a question: I created a custom extension method that which works as long the test doesn't have arguments. In that case, the same file is generated. What did I do wrong?

[SimonCropp]

hey egil

thanks for presenting Verify at Øredev. if u ever need help/review of content for a conference feel free to reach out. i am happy to workshop it with you

re your extension method

public static async Task Verify(this Task<IScenarioResult> scenario, [CallerFilePath] string sourceFile = "")
{
    var result = await scenario;
    await Verifier.Verify(result, sourceFile: sourceFile);
}

That should be redundant. Verify will automatically await Task<T> and verify the result.

So your extension method usage

[Fact]
public async Task Create_todo_using_snapshots()
{
    // Arrange
    await Host.Scenario(s =>
    {
        // Act
        s.Post
            .Json(new TestCreateParams(
                    Name: "Give presentation at Oredev",
                    IsComplete: false))
            .ToUrl("/todos");

        s.StatusCodeShouldBe(StatusCodes.Status201Created);
    }).Verify();
}

should be able to be changed to

[Fact]
public Task Create_todo_using_snapshots()
{
    // Arrange
    var scenario = Host.Scenario(s =>
    {
        // Act
        s.Post
            .Json(new TestCreateParams(
                Name: "Give presentation at Oredev",
                IsComplete: false))
            .ToUrl("/todos");

        s.StatusCodeShouldBe(StatusCodes.Status201Created);
    });
    return Verify(scenario);
}

works as long the test doesn't have arguments. In that case, the same file is generated

unfortunately there is now way (at least that could discover) in xunit to derive the parameter values from within Verify. So that means you need to explicitly pass them in: https://github.com/VerifyTests/Verify/blob/main/docs/parameterised.md#xunit

It is one of the reasons that i am gradually moving over to nunit. since it exposes parameter values though an API: https://github.com/VerifyTests/Verify/blob/main/docs/parameterised.md#nunit

so while in this case i think your extension method if redundant, in the case where you do extend verify in that way, i would recommend always returning a SettingsTask so consumers can optionally customize the verify behavior. in this case by calling UseParameters on the returned SettingsTask

[egil]

I am a bit hesitant to push out new packages.

why?

Honestly I think if you put something out you also have to have time to maintain it.

[SimonCropp]

so r u suggesting i push and own a new nuget😜

[egil]

No. I think this could be part of the Verify.SystemJson package. Configurable.

[SimonCropp]

ok. that sounds reasonable. want to submit a PR?

[egil]

I'll put it on my to-do list