Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Can't override DefaultConfig.Instance settings when using custom configs with IConfigSource attribute. #2556

Open
filzrev opened this issue Apr 5, 2024 · 0 comments

Comments

@filzrev
Copy link

filzrev commented Apr 5, 2024

I've created benchmark config by using ManualConfig derived class.
And using this config with [Config(typeof(CustomConfig))] attribute.

When running benchmarks.
It seems config is always merged with DefaultConfig.Instance by BenchmarkConverter::GetFullTypeConfig
And it causes unintended settings to be used when running benchmarks.

Minimum code to reproduce problems

internal class Program
{
    static void Main(string[] args)
    {
        BenchmarkRunner.Run<SampleBenchmarks>(config: null, args: args);
    }
}

public class CustomBenchmarkConfig : ManualConfig
{
    public CustomBenchmarkConfig()
    {
        AddExporter(MarkdownExporter.GitHub);
    }
}

[Config(typeof(CustomBenchmarkConfig))]
public class SampleBenchmarks
{
    [Benchmark]
    public void Benchmark01()
    {
    }
}

Warnings
Following warnings occurred when running above code.

// * Warnings *
Configuration
  Summary -> The exporter MarkdownExporter-github is already present in configuration. There may be unexpected results.

It seems default exporters are configured that are defined by DefaultConfig .

Temporary workaround
This problem can be avoided by explicitly specifying the config or pass config that is created by ManualConfig.CreateEmpty().

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant