Skip to content

Commit

Permalink
Removed support for netcoreapp3.0 and older runtimes.
Browse files Browse the repository at this point in the history
  • Loading branch information
timcassell committed Mar 10, 2024
1 parent 036f466 commit 120bf20
Show file tree
Hide file tree
Showing 40 changed files with 142 additions and 183 deletions.
24 changes: 12 additions & 12 deletions docs/articles/configs/toolchains.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,23 +18,23 @@ When you run your benchmarks without specifying the toolchain in an explicit way
If you want to test multiple frameworks, your project file **MUST target all of them** and you **MUST install the corresponding SDKs**:

```xml
<TargetFrameworks>netcoreapp3.0;netcoreapp2.1;net48</TargetFrameworks>
<TargetFrameworks>netcoreapp3.1;net8.0;net48</TargetFrameworks>
```

If you run your benchmarks without specifying any custom settings, BenchmarkDotNet is going to run the benchmarks **using the same framework as the host process**:

```cmd
dotnet run -c Release -f netcoreapp2.1 # is going to run the benchmarks using .NET Core 2.1
dotnet run -c Release -f netcoreapp3.0 # is going to run the benchmarks using .NET Core 3.0
dotnet run -c Release -f netcoreapp3.1 # is going to run the benchmarks using .NET Core 3.1
dotnet run -c Release -f net8.0 # is going to run the benchmarks using .NET 8.0
dotnet run -c Release -f net48 # is going to run the benchmarks using .NET 4.8
mono $pathToExe # is going to run the benchmarks using Mono from your PATH
```

To run the benchmarks for multiple runtimes with a single command, you need to specify the target framework moniker names via `--runtimes|-r` console argument:

```cmd
dotnet run -c Release -f netcoreapp2.1 --runtimes netcoreapp2.1 netcoreapp3.0 # is going to run the benchmarks using .NET Core 2.1 and .NET Core 3.0
dotnet run -c Release -f netcoreapp2.1 --runtimes netcoreapp2.1 net48 # is going to run the benchmarks using .NET Core 2.1 and .NET 4.8
dotnet run -c Release -f net8.0 --runtimes net8.0 netcoreapp3.1 # is going to run the benchmarks using .NET 8.0 and .NET Core 3.1
dotnet run -c Release -f net8.0 --runtimes net8.0 net48 # is going to run the benchmarks using .NET 8.0 and .NET 4.8
```

What is going to happen if you provide multiple Full .NET Framework monikers? Let's say:
Expand Down Expand Up @@ -67,8 +67,8 @@ namespace BenchmarkDotNet.Samples
{
[SimpleJob(RuntimeMoniker.Net48)]
[SimpleJob(RuntimeMoniker.Mono)]
[SimpleJob(RuntimeMoniker.NetCoreApp21)]
[SimpleJob(RuntimeMoniker.NetCoreApp30)]
[SimpleJob(RuntimeMoniker.NetCoreApp31)]
[SimpleJob(RuntimeMoniker.Net80)]
public class TheClassWithBenchmarks
```

Expand Down Expand Up @@ -115,9 +115,9 @@ public class MyConfig : ManualConfig
Add(Job.Default.With(
CsProjCoreToolchain.From(
new NetCoreAppSettings(
targetFrameworkMoniker: "netcoreapp2.1",
runtimeFrameworkVersion: "2.1.0-preview2-25628-01",
name: ".NET Core 2.1"))));
targetFrameworkMoniker: "net8.0-windows",
runtimeFrameworkVersion: "8.0.101",
name: ".NET 8.0 Windows"))));
}
}
```
Expand Down Expand Up @@ -146,11 +146,11 @@ public class CustomPathsConfig : ManualConfig
public CustomPathsConfig()
{
var dotnetCli32bit = NetCoreAppSettings
.NetCoreApp20
.NetCoreApp31
.WithCustomDotNetCliPath(@"C:\Program Files (x86)\dotnet\dotnet.exe", "32 bit cli");

var dotnetCli64bit = NetCoreAppSettings
.NetCoreApp20
.NetCoreApp31
.WithCustomDotNetCliPath(@"C:\Program Files\dotnet\dotnet.exe", "64 bit cli");

AddJob(Job.RyuJitX86.WithToolchain(CsProjCoreToolchain.From(dotnetCli32bit)).WithId("32 bit cli"));
Expand Down
8 changes: 4 additions & 4 deletions docs/articles/faq.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,17 +13,17 @@ See also: [BenchmarkDotNet#237](https://github.com/dotnet/BenchmarkDotNet/issues

* **Q** Why can't I install BenchmarkDotNet in a new .NET Core Console App in Visual Studio 2017?

**A** BenchmarkDotNet supports only netcoreapp2.0+.
**A** BenchmarkDotNet supports only netcoreapp3.1+.
Some old Visual Studio 2017 can create a new application which targets netcoreapp1.0.
You should upgrade it up to 2.0.
You should upgrade it up to 3.1.
If you want to target netcoreapp1.0 in your main assembly, it's recommended to create a separated project for benchmarks.

* **Q** I created a new .NET Core Console App in Visual Studio 2017. Now I want to run my code on CoreCLR, full .NET Framework, and Mono. How can I do it?

**A** Use the following lines in your `.csproj` file:

```xml
<TargetFrameworks>netcoreapp2.0;net46</TargetFrameworks>
<TargetFrameworks>netcoreapp3.1;net46</TargetFrameworks>
<PlatformTarget>AnyCPU</PlatformTarget>
```

Expand All @@ -33,7 +33,7 @@ If you want to target netcoreapp1.0 in your main assembly, it's recommended to c
[CoreJob, ClrJob, MonoJob]
```

* **Q** My source code targets old versions of .NET Framework or .NET Core, but BenchmarkDotNet requires `net461` and `netcoreapp2.0`. How can I run benchmarks in this case?
* **Q** My source code targets old versions of .NET Framework or .NET Core, but BenchmarkDotNet requires `net461` and `netcoreapp3.1`. How can I run benchmarks in this case?

**A** It's a good practice to introduce an additional console application (e.g. `MyAwesomeLibrary.Benchmarks`) which will depend on your code and BenchmarkDotNet.
Due to the fact that users usually run benchmarks in a develop environment and don't distribute benchmarks for users, it shouldn't be a problem.
Expand Down
22 changes: 11 additions & 11 deletions docs/articles/guides/console-args.md
Original file line number Diff line number Diff line change
Expand Up @@ -117,28 +117,28 @@ You can also filter the benchmarks by categories:
The `--runtimes` or just `-r` allows you to run the benchmarks for selected Runtimes. Available options are:

* Clr - BDN will either use Roslyn (if you run it as .NET app) or latest installed .NET SDK to build the benchmarks (if you run it as .NET Core app).
* Core - if you run it as .NET Core app, BDN will use the same target framework moniker, if you run it as .NET app it's going to use netcoreapp2.1.
* Core - if you run it as .NET Core app, BDN will use the same target framework moniker, if you run it as .NET app it's going to use net8.0.
* Mono - it's going to use the Mono from `$Path`, you can override it with `--monoPath`.
* net46, net461, net462, net47, net471, net472 - to build and run benchmarks against specific .NET framework version.
* netcoreapp2.0, netcoreapp2.1, netcoreapp2.2, netcoreapp3.0, netcoreapp3.1, net5.0, net6.0, net7.0 - to build and run benchmarks against specific .NET Core version.
* nativeaot5.0, nativeaot6.0, nativeaot7.0 - to build and run benchmarks using NativeAOT. Can be customized with additional options: `--ilcPath`, `--ilCompilerVersion`.
* netcoreapp3.1, net5.0, net6.0, net7.0, net8.0 - to build and run benchmarks against specific .NET Core version.
* nativeaot5.0, nativeaot6.0, nativeaot7.0, , nativeaot8.0 - to build and run benchmarks using NativeAOT. Can be customized with additional options: `--ilcPath`, `--ilCompilerVersion`.

Example: run the benchmarks for .NET 4.7.2 and .NET Core 2.1:
Example: run the benchmarks for .NET 4.7.2 and .NET 8.0:

```log
dotnet run -c Release -- --runtimes net472 netcoreapp2.1
dotnet run -c Release -- --runtimes net472 net8.0
```

Example: run the benchmarks for .NET Core 3.0 and latest .NET SDK installed on your PC:
Example: run the benchmarks for .NET Core 3.1 and latest .NET SDK installed on your PC:

```log
dotnet run -c Release -f netcoreapp3.0 -- --runtimes clr core
dotnet run -c Release -f netcoreapp3.1 -- --runtimes clr core
```

But same command executed with `-f netcoreapp2.0` is going to run the benchmarks for .NET Core 2.0:
But same command executed with `-f net6.0` is going to run the benchmarks for .NET 6.0:

```log
dotnet run -c Release -f netcoreapp2.0 -- --runtimes clr core
dotnet run -c Release -f net6.0 -- --runtimes clr core
```

## Number of invocations and iterations
Expand Down Expand Up @@ -207,10 +207,10 @@ To perform a Mann–Whitney U Test and display the results in a dedicated column

* `--statisticalTest`- Threshold for Mann–Whitney U Test. Examples: 5%, 10ms, 100ns, 1s

Example: run Mann–Whitney U test with relative ratio of 5% for all benchmarks for .NET Core 2.0 (base) vs .NET Core 2.1 (diff). .NET Core 2.0 will be baseline because it was first.
Example: run Mann–Whitney U test with relative ratio of 5% for all benchmarks for .NET 6.0 (base) vs .NET 8.0 (diff). .NET 6.0 will be baseline because it was first.

```log
dotnet run -c Release -- --filter * --runtimes netcoreapp2.0 netcoreapp2.1 --statisticalTest 5%
dotnet run -c Release -- --filter * --runtimes net6.0 net8.0 --statisticalTest 5%
```

## More
Expand Down
6 changes: 3 additions & 3 deletions docs/articles/guides/troubleshooting.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ How do you know that BenchmarkDotNet has failed to build the project? BDN is goi
// ***** BenchmarkRunner: Start *****
// ***** Found 1 benchmark(s) in total *****
// ***** Building 1 exe(s) in Parallel: Start *****
// start dotnet restore /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 /p:Deterministic=true /p:Optimize=true in C:\Projects\BenchmarkDotNet\samples\BenchmarkDotNet.Samples\bin\Release\netcoreapp2.1\c6045772-d3c7-4dbe-ab37-4aca6dcb6ec4
// start dotnet restore /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 /p:Deterministic=true /p:Optimize=true in C:\Projects\BenchmarkDotNet\samples\BenchmarkDotNet.Samples\bin\Release\net8.0\c6045772-d3c7-4dbe-ab37-4aca6dcb6ec4
// command took 0.51s and exited with 1
// ***** Done, took 00:00:00 (0.66 sec) *****
// Found 1 benchmarks:
Expand All @@ -20,10 +20,10 @@ How do you know that BenchmarkDotNet has failed to build the project? BDN is goi
// Build Error: Standard output:
Standard error:
C:\Projects\BenchmarkDotNet\samples\BenchmarkDotNet.Samples\bin\Release\netcoreapp2.1\c6045772-d3c7-4dbe-ab37-4aca6dcb6ec4\BenchmarkDotNet.Autogenerated.csproj(36,1): error MSB4025: The project file could not be loaded. Unexpected end of file while parsing Comment has occurred. Line 36, position 1.
C:\Projects\BenchmarkDotNet\samples\BenchmarkDotNet.Samples\bin\Release\net8.0\c6045772-d3c7-4dbe-ab37-4aca6dcb6ec4\BenchmarkDotNet.Autogenerated.csproj(36,1): error MSB4025: The project file could not be loaded. Unexpected end of file while parsing Comment has occurred. Line 36, position 1.
// BenchmarkDotNet has failed to build the auto-generated boilerplate code.
// It can be found in C:\Projects\BenchmarkDotNet\samples\BenchmarkDotNet.Samples\bin\Release\netcoreapp2.1\c6045772-d3c7-4dbe-ab37-4aca6dcb6ec4
// It can be found in C:\Projects\BenchmarkDotNet\samples\BenchmarkDotNet.Samples\bin\Release\net8.0\c6045772-d3c7-4dbe-ab37-4aca6dcb6ec4
// Please follow the troubleshooting guide: https://benchmarkdotnet.org/articles/guides/troubleshooting.html
```

Expand Down
4 changes: 2 additions & 2 deletions samples/BenchmarkDotNet.Samples/IntroEnvVars.cs
Original file line number Diff line number Diff line change
Expand Up @@ -14,8 +14,8 @@ private class ConfigWithCustomEnvVars : ManualConfig

public ConfigWithCustomEnvVars()
{
AddJob(Job.Default.WithRuntime(CoreRuntime.Core21).WithId("Inlining enabled"));
AddJob(Job.Default.WithRuntime(CoreRuntime.Core21)
AddJob(Job.Default.WithRuntime(CoreRuntime.Core80).WithId("Inlining enabled"));
AddJob(Job.Default.WithRuntime(CoreRuntime.Core80)
.WithEnvironmentVariables(new EnvironmentVariable(JitNoInline, "1"))
.WithId("Inlining disabled"));
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ public static void Run()
.Run<Algo_Md5VsSha256>(
DefaultConfig.Instance
.AddJob(Job.Default.WithRuntime(ClrRuntime.Net462))
.AddJob(Job.Default.WithRuntime(CoreRuntime.Core21))
.AddJob(Job.Default.WithRuntime(CoreRuntime.Core80))
.AddValidator(ExecutionValidator.FailOnError));
}
}
Expand Down
13 changes: 9 additions & 4 deletions src/BenchmarkDotNet.Annotations/Jobs/RuntimeMoniker.cs
Original file line number Diff line number Diff line change
Expand Up @@ -57,22 +57,27 @@ public enum RuntimeMoniker
/// <summary>
/// .NET Core 2.0
/// </summary>
NetCoreApp20,
[Obsolete("This runtime is no longer supported. Use a newer runtime or use BenchmarkDotNet v0.13.X or older.", true)]
// Assigning explicit values so we can check for them without the compiler erroring.
NetCoreApp20 = 10,

/// <summary>
/// .NET Core 2.1
/// </summary>
NetCoreApp21,
[Obsolete("This runtime is no longer supported. Use a newer runtime or use BenchmarkDotNet v0.13.X or older.", true)]
NetCoreApp21 = 11,

/// <summary>
/// .NET Core 2.2
/// </summary>
NetCoreApp22,
[Obsolete("This runtime is no longer supported. Use a newer runtime or use BenchmarkDotNet v0.13.X or older.", true)]
NetCoreApp22 = 12,

/// <summary>
/// .NET Core 3.0
/// </summary>
NetCoreApp30,
[Obsolete("This runtime is no longer supported. Use a newer runtime or use BenchmarkDotNet v0.13.X or older.", true)]
NetCoreApp30 = 13,

/// <summary>
/// .NET Core 3.1
Expand Down
10 changes: 5 additions & 5 deletions src/BenchmarkDotNet.Diagnostics.dotTrace/DotTraceDiagnoser.cs
Original file line number Diff line number Diff line change
Expand Up @@ -99,6 +99,11 @@ internal static bool IsSupported(RuntimeMoniker runtimeMoniker)
case RuntimeMoniker.Net80:
case RuntimeMoniker.Net90:
return true;
// netcoreapp20 - netcoreapp30 are no longer supported.
case (RuntimeMoniker) 10:
case (RuntimeMoniker) 11:
case (RuntimeMoniker) 12:
case (RuntimeMoniker) 13:
case RuntimeMoniker.NotRecognized:
case RuntimeMoniker.Mono:
case RuntimeMoniker.NativeAot60:
Expand All @@ -124,11 +129,6 @@ internal static bool IsSupported(RuntimeMoniker runtimeMoniker)
case RuntimeMoniker.NetCoreApp50:
#pragma warning restore CS0618 // Type or member is obsolete
return false;
case RuntimeMoniker.NetCoreApp20:
case RuntimeMoniker.NetCoreApp21:
case RuntimeMoniker.NetCoreApp22:
return RuntimeInformation.IsWindows();
case RuntimeMoniker.NetCoreApp30:
case RuntimeMoniker.NetCoreApp31:
return RuntimeInformation.IsWindows() || RuntimeInformation.IsLinux();
default:
Expand Down
6 changes: 3 additions & 3 deletions src/BenchmarkDotNet/BenchmarkDotNet.csproj
Original file line number Diff line number Diff line change
Expand Up @@ -23,9 +23,9 @@
<PackageReference Include="Perfolizer" Version="[0.3.16]" />
<PackageReference Include="Microsoft.Diagnostics.Tracing.TraceEvent" Version="3.1.8" PrivateAssets="contentfiles;analyzers" />
<PackageReference Include="Microsoft.DotNet.PlatformAbstractions" Version="3.1.6" />
<!-- Do not update these packages, or else netcoreapp3.0 and older will no longer work -->
<PackageReference Include="System.Management" Version="5.0.0" />
<PackageReference Include="Microsoft.CodeAnalysis.CSharp" Version="4.1.0" />
<!-- Do not update these packages, or else netcoreapp3.1 may no longer work -->
<PackageReference Include="System.Management" Version="6.0.0" />
<PackageReference Include="Microsoft.CodeAnalysis.CSharp" Version="4.5.0" />
</ItemGroup>
<ItemGroup Condition=" '$(TargetFramework)' == 'netstandard2.0' ">
<PackageReference Include="Microsoft.Win32.Registry" Version="5.0.0" />
Expand Down
8 changes: 4 additions & 4 deletions src/BenchmarkDotNet/ConsoleArguments/CommandLineOptions.cs
Original file line number Diff line number Diff line change
Expand Up @@ -237,8 +237,8 @@ public static IEnumerable<Example> Examples

yield return new Example("Use Job.ShortRun for running the benchmarks", shortName, new CommandLineOptions { BaseJob = "short" });
yield return new Example("Run benchmarks in process", shortName, new CommandLineOptions { RunInProcess = true });
yield return new Example("Run benchmarks for .NET 4.7.2, .NET Core 2.1 and Mono. .NET 4.7.2 will be baseline because it was first.", longName, new CommandLineOptions { Runtimes = new[] { "net472", "netcoreapp2.1", "Mono" } });
yield return new Example("Run benchmarks for .NET Core 2.0, .NET Core 2.1 and .NET Core 2.2. .NET Core 2.0 will be baseline because it was first.", longName, new CommandLineOptions { Runtimes = new[] { "netcoreapp2.0", "netcoreapp2.1", "netcoreapp2.2" } });
yield return new Example("Run benchmarks for .NET 4.7.2, .NET 8.0 and Mono. .NET 4.7.2 will be baseline because it was first.", longName, new CommandLineOptions { Runtimes = new[] { "net472", "net8.0", "Mono" } });
yield return new Example("Run benchmarks for .NET Core 3.1, .NET 6.0 and .NET 8.0. .NET Core 3.1 will be baseline because it was first.", longName, new CommandLineOptions { Runtimes = new[] { "netcoreapp3.1", "net6.0", "net8.0" } });
yield return new Example("Use MemoryDiagnoser to get GC stats", shortName, new CommandLineOptions { UseMemoryDiagnoser = true });
yield return new Example("Use DisassemblyDiagnoser to get disassembly", shortName, new CommandLineOptions { UseDisassemblyDiagnoser = true });
yield return new Example("Use HardwareCountersDiagnoser to get hardware counter info", longName, new CommandLineOptions { HardwareCounters = new[] { nameof(HardwareCounter.CacheMisses), nameof(HardwareCounter.InstructionRetired) } });
Expand All @@ -250,8 +250,8 @@ public static IEnumerable<Example> Examples
yield return new Example("Run selected benchmarks once per iteration", longName, new CommandLineOptions { RunOncePerIteration = true });
yield return new Example("Run selected benchmarks 100 times per iteration. Perform single warmup iteration and 5 actual workload iterations", longName, new CommandLineOptions { InvocationCount = 100, WarmupIterationCount = 1, IterationCount = 5});
yield return new Example("Run selected benchmarks 250ms per iteration. Perform from 9 to 15 iterations", longName, new CommandLineOptions { IterationTimeInMilliseconds = 250, MinIterationCount = 9, MaxIterationCount = 15});
yield return new Example("Run MannWhitney test with relative ratio of 5% for all benchmarks for .NET Core 2.0 (base) vs .NET Core 2.1 (diff). .NET Core 2.0 will be baseline because it was provided as first.", longName,
new CommandLineOptions { Filters = new[] { "*"}, Runtimes = new[] { "netcoreapp2.0", "netcoreapp2.1" }, StatisticalTestThreshold = "5%" });
yield return new Example("Run MannWhitney test with relative ratio of 5% for all benchmarks for .NET 6.0 (base) vs .NET 8.0 (diff). .NET Core 6.0 will be baseline because it was provided as first.", longName,
new CommandLineOptions { Filters = new[] { "*"}, Runtimes = new[] { "net6.0", "net8.0" }, StatisticalTestThreshold = "5%" });
yield return new Example("Run benchmarks using environment variables 'ENV_VAR_KEY_1' with value 'value_1' and 'ENV_VAR_KEY_2' with value 'value_2'", longName,
new CommandLineOptions { EnvironmentVariables = new[] { "ENV_VAR_KEY_1:value_1", "ENV_VAR_KEY_2:value_2" } });
yield return new Example("Hide Mean and Ratio columns (use double quotes for multi-word columns: \"Alloc Ratio\")", shortName, new CommandLineOptions { HiddenColumns = new[] { "Mean", "Ratio" }, });
Expand Down
4 changes: 0 additions & 4 deletions src/BenchmarkDotNet/ConsoleArguments/ConfigParser.cs
Original file line number Diff line number Diff line change
Expand Up @@ -519,10 +519,6 @@ private static Job CreateJobForGivenRuntime(Job baseJob, string runtimeId, Comma
.WithRuntime(runtimeMoniker.GetRuntime())
.WithToolchain(CsProjClassicNetToolchain.From(runtimeId, options.RestorePath?.FullName, options.CliPath?.FullName));

case RuntimeMoniker.NetCoreApp20:
case RuntimeMoniker.NetCoreApp21:
case RuntimeMoniker.NetCoreApp22:
case RuntimeMoniker.NetCoreApp30:
case RuntimeMoniker.NetCoreApp31:
#pragma warning disable CS0618 // Type or member is obsolete
case RuntimeMoniker.NetCoreApp50:
Expand Down
4 changes: 2 additions & 2 deletions src/BenchmarkDotNet/Diagnosers/EventPipeProfiler.cs
Original file line number Diff line number Diff line change
Expand Up @@ -62,9 +62,9 @@ public IEnumerable<ValidationError> Validate(ValidationParameters validationPara
{
var runtime = benchmark.Job.ResolveValue(EnvironmentMode.RuntimeCharacteristic, EnvironmentResolver.Instance);

if (runtime.RuntimeMoniker < RuntimeMoniker.NetCoreApp30)
if (runtime.RuntimeMoniker < RuntimeMoniker.NetCoreApp31)
{
yield return new ValidationError(true, $"{nameof(EventPipeProfiler)} supports only .NET Core 3.0+", benchmark);
yield return new ValidationError(true, $"{nameof(EventPipeProfiler)} supports only .NET Core 3.1+", benchmark);
}
}
}
Expand Down

0 comments on commit 120bf20

Please sign in to comment.