Support Parameterized Benchmark Constructors

[smfields]

I'd like to be able to take a parameter that contains multiple values and display it in the summary table spread across multiple columns, rather than having all the values in the same column. For example, take this benchmark:

public record MyParamType(string Value1, int Value2);

public class MyBenchmark
{
    [ParamsSource(nameof(MyParamValues))]
    public MyParamType MyParameter { get; set; }

    public IEnumerable<MyParamType> MyParamValues()
    {
        yield return new MyParamType("first", 1);
        yield return new MyParamType("second", 2);
        yield return new MyParamType("third", 3);
    }

    [Benchmark]
    public void Benchmark() 
    {
    }
}

Rather than having it display like this (the default):

Method	MyParameter	Mean	Error
Benchmark	MyPar(...)= 1 } [42]	100.50 us	NA
Benchmark	MyPar(...)= 2 } [43]	99.20 us	NA
Benchmark	MyPar(...)= 3 } [42]	106.60 us	NA

I'd like to have the value of MyParameter split accross multiple columns like so:

Method	Value1	Value2	Mean	Error
Benchmark	first	1	100.50 us	NA
Benchmark	second	2	99.20 us	NA
Benchmark	third	3	106.60 us	NA

Note:

• I can't switch to using arguments because I need the values of the parameters to be available in my GlobalSetup.
• I can't split the parameter into multiple separate parameters because I need control over how the parameters are combined (i.e. not a full cartisian product)

Any ideas on whether this is something that's possible?

[timcassell]

I don't think that makes much sense. The values are tied together, so why would they have separate columns? If you just want a cleaner display, you can override ToString. See also #1634.

[Edit] From your notes, it looks like you actually want a way to specify the params combinations, rather than splitting up a param. Is that correct?

[smfields]

I don't think that makes much sense. The values are tied together, so why would they have separate columns? If you just want a cleaner display, you can override ToString. See also #1634.

Thanks for the quick reply. The values aren't exactly tied together in that way. It's more that only some combinations of parameters actually make sense, but not the full cartisian product of all parameters. So at the moment I'm forced to use ParamsSource and specify the combinations manually rather then using Params and letting the framework do all combinations.

From your notes, it looks like you actually want a way to specify the params combinations, rather than splitting up a param. Is that correct?

Yup, more fine grained control over how parameters are combined would also totally work, although I'm not quite sure what that would look like.

It might be helpful to give a more realistic example. This isn't exactly what I'm doing, but imagine I had these parameters:

• Repository: SQLRepository, NoSQLRepository, and TableStorageRepository
• DataStore: Postgres, SQL Server, MySQL, MongoDB, CosmosDB, Azure Table, Local File

In this case, I'd basically want the following test matrix:

Repository	DataStore
SQLRepository	Postgres
SQLRepository	SQL Server
SQLRepository	MySQL
NoSQLRepository	MongoDB
NoSQLRepository	CosmosDB
TableStorageRepository	Azure Table
TableStorageRepository	Local File

The GlobalSetup is then making sure the correct repository and data store are setup. Once that's done the benchmark can then work with the common IRepository interface and use the same benchmark accross all repository and data store types.

If you can think of any other way of acheiving this that would also be helpful.

[smfields]

I've managed to work out something that uses inheritence to accomplish what I need. Essentially I have an abstract base class that contains some shared parameters, the Benchmark method, and the setup and teardown methods, and then I've got multiple concrete classes that extend that base class and specify additional parameters. The parameter values from the concrete subclass are combined with the parameters from the base class, but not with the parameters from other subclasses.

What would really be ideal would be a way to specify one source method that provides multiple parameter values, similar to how the ArgumentSource works, but where those values could still be accessed from setup and teardown methods. This could be done by doing something similar to NUnit's TestFixtureSource, where you specify a source and that gets turned into constructor parameters on the class.

[timcassell]

That's not a bad idea. We currently only support parameter-less constructors. We could add parameter support.