Builder.WithVariations✨

[mrpmorris]

Description

We are using feature flags where I work. My tests need to test various combination of those flags.

Example

Imagine I have 3 stories in my system that all affect the same class, and therefore my unit tests.

• 123 - Disallow purchase of discontinued products
• 234 - Migrate from payment gateway server ABC over to XYZ
• 345 - Add special offer discount

In my unit tests project I might create a class for this

public class AllPossibleFlagsForThisTest
{
  public bool IsFeature123Enabled_DisallowPurchaseOfDiscontinuedProducts { get; set; }
  public bool IsFeature234Enabled_UseNewPaymentsGateway { get; set; }
  public decimal Feature345_SpecialOfferDiscountPercent { get; set; }
}

and then iterate over them

IEnumerable flagVariations = fixture.Create()
.WithVariations(x => x.IsFeature123Enabled_DisallowPurchaseOfDiscontinuedProducts, [true, false])
.WithVariations(x => x.IsFeature234Enabled_UseNewPaymentsGateway , [true, false])
.WithVariations(x => x.Feature345_SpecialOfferDiscountPercent , [0m, 0.2m, 0.6m])
.Build();

Chat GPT put something together for me, but it's overly compllicated and therefore difficult to understand. I'd like to write an implementation myself from scratch so it is more easily understood. They should then be able to switch to the (better) AutoFixture solution.

Tasks

• Confirmed feature idea with maintainers in support forums

More Information

If I can work out your code base (I've not yet looked) and possibly with your guidance then I can submit a PR.

[aivascu]

Hi @mrpmorris,
This sounds like an interesting feature request, though I'm not yet sure how it would fit into the current builder API.
This could also potentially have a severe impact on the performance, depending on how it is implemented/used.

For these kind of scenarios I'd usually recommend generating the variations elsewhere and then inject them into the current test case. For example, it should be possible to use a separate method to generate the flags object, and then inject it in other services that might use it.

[Theory]
[MemberAutoData(nameof(FeatureFlagsVariations))]
public void CanGenerateAllPossibleVariations([Frozen] AllPossibleFlagsForThisTest flags, SomeService service1)
{
    // Your test code here
}

public static IEnumerable<object[]> FeatureFlagsVariations()
{
    // Generate all possible combinations of flags
    yield return [new AllPossibleFlagsForThisTest { /* ... */ }];
}

Could you provide more details on this utility that you're using at the moment? Perhaps a snippet of code showing how you're currently implementing the tests, and how do you see AutoFixture helping with that.

[mrpmorris]

Hi Andrei, thanks for your quick response!

My company is using feature flags. I could simply write a test that says WhenFlagXIsOn_ThenDoesA and another WhenFlagYIsOn_ThenDoesB.

But then when someone else is working in the same area and has their own feature flag, I don't want to risk my code breaking because their code did something they didn't think to check for.

I wrote some code like this.

IEnumerable<MyType> allVariations = VariationsBuilder.Create<MyTable>()
  .With(x => x.Enabled, [true, false])
  .With(x => x.Obsolete, [true, false])
  .With(x => x.DeliveryLeadDays, [1, 3, 5, 7])
  .Build();

I would then loop through each IEnumerable<MyType> allVariations and execute the test code. I didn't like this approach so continued to look for a better one.

This morning I was informed of Xunit.Combinatorial which allows me to do the following

public class UnitTest1
{
    [Theory, CombinatorialData]
    public void Test1(
        [CombinatorialValues] FlagsCombination flags)
    {
    }

    [Theory, CombinatorialData]
    public void Test2(
        [CombinatorialValues] FlagsCombination flags,
        [CombinatorialValues(null, "", "  ")] string inputValue)
    {
    }

    public record struct FlagsCombination(bool Is520Enabled, bool Is123Enabled);
}

I thought my approach might be a nice addition to AutoFixture so offered to add it. Although I no longer need it just for flag combinations I thought it might be useful for building data like this

IEnumerable<Person> = fixture.CreateVariations<Person>()
  .With(x => x.GivenName, ["Bob", "Peter", "Fred"])
  .With(x => x.FamilyName, ["Monkhouse", "Pan", "Flintstone"])
  .With(x => x.Age, [5, 25, 55]);

Maybe for building large data sets or something?

[aivascu]

@mrpmorris can you provide the code of the variations generator?
I must warn you though, that you must be the legal owner of the code you're contributing. From what you mentioned so far, it seems that you wrote the implementation for work. Please before sharing any more code ensure that you are not in breach of any contract.

[mrpmorris]

The code I shared was the interface to consume the code rather than the code itself.

I got chat gpt to write something that was "good enough for now", but I had an idea for a better implementation that I intended to write in my own time.

After posting here I got impatient and wrote my better idea whilst working for them, so now I'll have to ask if they'd be willing to contribute it.

I'll let you know what they say.

I'm sorry for my impatience :)