Granular control of which field are included in VerifiableTree expectation

[goldsam]

Description

Is there a mechanism to fine-tune expectations on VerifiableTree and friends? While the ExcpectXXX methods of Tree are fine most of the time, it would be useful to have more control over which properties of an object are included in the expectation comparison.

It looks like the tree is immutable, so I guess this would have to be accomplished using a builder. Would this have to be addressed through a new feature or am I missing something?

Progress

• Feature is implemented,
• Ensured backward-compatibility,
• Ensured good debugging experience

[goldsam]

Although Tree.ExpectEquivalent(...) supports expectation expressions as documented here, I find this cumbersome to use for large and complex objects - its not really type safe and is a bit error-prone.

Consider the following proposed interface:

public interface IExpectationBuilder<T>
{
    IExpectationBuilder<T> EquivalentTo(T? value);

    IExpectationBuilder<T> Excluding<TResult>(Expression<Func<T, TResult>> o);

    IExpectationBuilder<T> Expecting<TResult>(Expression<Func<T, TResult>> o, Func<IExpectationComposer, IExpectation<TResult>> to);
}

and a proposed companion static Tree method:

public static class Tree
{
    //...

    public static VerifiableTree Expect<T>(Action<IExpectationBuilder<T>> expect)
    
   // ...
}

This might be used in a scenario as follows:

await Runner.RunScenario(
    _ => _.Then_the_thing_should_look_like(Tree.Expect<MyType>(value => value 
        .EquivalentTo(new MyType { Z = "test" })
        .Excluding(p => p.X)
        .Expecting(p => p.Y, to => to.BeGreaterThan(7f)))));

where MyType is in this example is defined as:

public class MyType
{
    public int X { get; set; }

    public float Y { get; set; }

    public string Z { get; set; }
}

This has the benefit of being reasonably terse, type safe, and legible.

What do you think @Suremaker ?

[Suremaker]

Hi @goldsam ,

Currently the LightBDD does not offer property level mapping (include / exclude) as with the potentially complex structure of the object trees, it seems to quickly become difficult to maintain / write.

Having said that, I do not mind exploring this further if there is a good use case for it, as the underlying logic should be relatively easy to add this.

Before I go with this further, I wanted to ask if you check the Tree.ExpectContaining() (https://github.com/LightBDD/LightBDD/wiki/Advanced-Step-Parameters#treeexpectcontaining) alternatives which basically allows you to match your actual model about the expectations which are the subsets of the tree, and which will ignore the parts that you do not care about in your test, while it will still allow you to use expectation expressions in the areas where you want it.

Example test:

LightBDD/examples/Example.LightBDD.XUnit2/Features/Address_book_feature.cs

Line 43 in c07341e

public void Matching_addresses_by_email()

[Suremaker]

If you have simple types to compare, it is generally straightforward to implement such methods:

public class My_feature : FeatureFixture
{
    private MyType _actual;

    [Scenario]
    public void My_test()
    {
        Runner.RunScenario(
            _ => Given_actual(Tree.For(new MyType()
            {
                X = 5,
                Y = 3.14f,
                Z = "ZZZ"
            })),
            _ => Then_results_should_match(TreeEx.ExpectMatch<MyType>(value => value
                .EquivalentTo(new MyType { Z = "ZZZ" })
                .Excluding(p => p.X)
                .Expecting(p => p.Y, Expect.To.BeGreaterThan(3f))
            )));
    }

    private void Then_results_should_match(VerifiableTree match)
    {
        match.SetActual(_actual);
    }

    private void Given_actual(InputTree<MyType> actual)
    {
        _actual = actual.Input;
    }
}

public static class TreeEx
{
    public static VerifiableTree ExpectMatch<T>(Action<TreeExpectationBuilder<T>> builder)
    {
        var b = new TreeExpectationBuilder<T>();
        builder(b);
        return Tree.Expect(b.Build(), VerifiableTreeOptions.EquivalentMatch
            // how to treat not mapped items: Ignore or Exclude
            .WithUnexpectedNodeAction(UnexpectedValueVerificationAction.Ignore));
    }
}

public class TreeExpectationBuilder<T>
{
    private readonly Dictionary<string, object> _expectations = new();
    public TreeExpectationBuilder<T> EquivalentTo(object expected)
    {
        foreach (var property in expected.GetType().GetProperties())
            _expectations[property.Name] = property.GetValue(expected);

        return this;
    }

    public TreeExpectationBuilder<T> Expecting<TV>(Expression<Func<T, TV>> memberExpression, Expectation<TV> value)
    {
        var path = ExtractPath(memberExpression);
        if (path != null)
            _expectations[path] = value;
        return this;
    }

    public TreeExpectationBuilder<T> Excluding<TV>(Expression<Func<T, TV>> memberExpression)
    {
        var path = ExtractPath(memberExpression);
        if (path != null)
            _expectations.Remove(path);
        return this;
    }

    private static string ExtractPath<TV>(Expression<Func<T, TV>> memberExpression)
    {
        return (memberExpression.Body as MemberExpression)?.Member.Name;
    }

    public object Build() => _expectations;
}


public class MyType
{
    public int X { get; set; }

    public float Y { get; set; }

    public string Z { get; set; }

}

The following code will give the following results:

The complexity will start arising when your models will have complex objects within their hierarchy as:

• more complex inclusion / exclusion methods will need to be developed to control on what you would like to add / remove
• the builder will need to be updated to support nested dictionaries and property selection paths, to correctly reflect the object hierarchy

This is the reason why I am really wondering if excluding/expecting methods will be much better than using Tree.ExpectContaining() method with models tailored to your expectations (which could be dedicated test models, anonymous types, ExpandoObjects/dynamics or even JsonElements)

public class My_feature : FeatureFixture
{
    private MyType _actual;

    [Scenario]
    public void My_test()
    {
        Runner.RunScenario(
            _ => Given_actual(Tree.For(new MyType()
            {
                X = 5,
                Y = 3.14f,
                Z = "ZZZ"
            })),
            _ => Then_results_should_match(Tree.ExpectContaining(
                new
                {
                    Y = Expect.To.BeGreaterThan(3f),
                    Z = "ZZZ"
                })));
    }

    private void Then_results_should_match(VerifiableTree match)
    {
        match.SetActual(_actual);
    }

    private void Given_actual(InputTree<MyType> actual)
    {
        _actual = actual.Input;
    }
}

public class MyType
{
    public int X { get; set; }

    public float Y { get; set; }

    public string Z { get; set; }

}

[goldsam]

In my case, my types are generated protocol buffers. The types are generally about 3 levels deep and pretty wide with quite a few properties. I just needed to apply an exclusion or inequality comparison to only a handful of properties, but sometimes at multiple levels in the object tree. Currently, I am just using a bunch of expectation expression - I did not realize you could mix with other objects with expectations (makes sense though now that I reflect on it). This will come in handy, so thank you!

I'm inspired by the code fragments in your last response. I was envisioning a system where you could override operations (e.g. "these two objects are equivalent EXCEPT for this property") . As you pointed out, a good general purpose solution is non-trivial (and might yield unexpected results). Despite these shortcomings, this approach would definitely keep the code base much smaller and I would argue easier to read (at least for my use case).

Originally, I was thinking it would be necessary to construct a customized ObjectTree representation. Would there be any benefit to doing so instead (perhaps leveraging NodeMapper?

Also, I have to compliment you on this library - I think its simply amazing. It forces patterns which lead to much higher quality tests. I find in practice that tests are easier to maintain since intent is clearly communicated, while details are kept out-of-the-way unless you need to dive into them. The only gripe I get from my teammates is that the cost of writing tests is higher - I just retorted that the costs of writing useful tests is always higher - but the cost of low quality tests is a small fortune.

Great project! Thank you.

[Suremaker]

Hi @goldsam ,

Thank you for the kind words on the project - I am glad that you are finding it useful for you and your team!

Feel free to experiment with the extension methods that I have presented above. You can also use NodeMapper as mentioned and create a specific mapper for your types.

The mapper can be then registered on LightBDDConfiguration via cfg.ObjectTreeConfiguration().ConfigureBuilder(ObjectTreeBuilderOptions.Default.AppendMapper(/*...*/)).

You can also take a look on how json objects are mapped with JsonElementObjectMapper.

If you end up in building something of general purpose that you would be keen to share with the community, please feel free to let me know - I will be happy to incorporate it to LightBDD Framework.