Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Initial support of Microsoft.Testing.Platform #1153

Merged
merged 13 commits into from
May 3, 2024
Merged

Initial support of Microsoft.Testing.Platform #1153

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
13 commits
Select commit Hold shift + click to select a range
d7dc497
Initial support of Microsoft.Testing.Platform
Evangelink Feb 20, 2024
9da8cd9
Bump bridge version
Evangelink Mar 14, 2024
24133f3
Drop telemetry
Evangelink Mar 14, 2024
49326bd
Enable trace writer fix for NUnit
Evangelink Mar 14, 2024
3789fff
Merge branch 'master' into microsoft-testing-platform
Evangelink Apr 22, 2024
8754cd1
Bump version of the bridge
Evangelink Apr 22, 2024
c3fa697
Comment out object model v11
Evangelink Apr 22, 2024
8970bc7
Remove samples and "local" entry in nuget.config
Evangelink Apr 22, 2024
b588819
Tests: drop Microsoft.TestPlatform.ObjectModel v11
Evangelink Apr 22, 2024
e742849
Fix ObjectModel v11 test
Evangelink Apr 22, 2024
9a340ce
Fix broken tests
Evangelink Apr 22, 2024
23c6da7
Bump platform version in nuspec
Evangelink May 2, 2024
80da537
Nit: tab vs space
Evangelink May 2, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
2 changes: 2 additions & 0 deletions NUnit3TestAdapter.sln
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,7 @@ EndProject
Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "net462", "net462", "{D41249E3-C080-4B66-8CBD-99CE4D309A10}"
ProjectSection(SolutionItems) = preProject
nuget\net462\NUnit3TestAdapter.props = nuget\net462\NUnit3TestAdapter.props
nuget\net462\NUnit3TestAdapter.targets = nuget\net462\NUnit3TestAdapter.targets
EndProjectSection
EndProject
Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "NUnit.TestAdapter.Tests.Acceptance", "src\NUnit.TestAdapter.Tests.Acceptance\NUnit.TestAdapter.Tests.Acceptance.csproj", "{3FAC7EE0-664F-4B11-918B-8E0FF865EE4C}"
Expand All @@ -60,6 +61,7 @@ EndProject
Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "netcoreapp3.1", "netcoreapp3.1", "{2F940513-5B8F-45A5-A188-7C5D03D1B50D}"
ProjectSection(SolutionItems) = preProject
nuget\netcoreapp3.1\NUnit3TestAdapter.props = nuget\netcoreapp3.1\NUnit3TestAdapter.props
nuget\netcoreapp3.1\NUnit3TestAdapter.targets = nuget\netcoreapp3.1\NUnit3TestAdapter.targets
EndProjectSection
EndProject
Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "images", "images", "{3D1159FD-FC4D-4750-87EA-F477B3912B3C}"
Expand Down
2 changes: 2 additions & 0 deletions build.cake
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -262,6 +262,7 @@ Task("CreateWorkingImage")
CreateDirectory(net462Dir);
CopyFiles(net462Files, net462Dir);
CopyFileToDirectory("nuget/net462/NUnit3TestAdapter.props", net462Dir);
CopyFileToDirectory("nuget/net462/NUnit3TestAdapter.targets", net462Dir);

var netcoreDir = PACKAGE_IMAGE_DIR + "build/" + NETCOREAPP_TFM;
DotNetCorePublish(ADAPTER_PROJECT, new DotNetCorePublishSettings
Expand All @@ -271,6 +272,7 @@ Task("CreateWorkingImage")
Framework = NETCOREAPP_TFM
});
CopyFileToDirectory($"nuget/{NETCOREAPP_TFM}/NUnit3TestAdapter.props", netcoreDir);
CopyFileToDirectory($"nuget/{NETCOREAPP_TFM}/NUnit3TestAdapter.targets", netcoreDir);
});

Task("PackageZip")
Expand Down
2 changes: 1 addition & 1 deletion nuget.config
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,6 @@
<packageSources>
<clear />
<add key="nuget" value="https://api.nuget.org/v3/index.json" />
<!--<add key="Local" value="c:\nuget" />-->
<add key="Local" value="C:\src\nunit3-vs-adapter\package" />
Evangelink marked this conversation as resolved.
Show resolved Hide resolved
</packageSources>
</configuration>
15 changes: 15 additions & 0 deletions nuget/NUnit3TestAdapter.nuspec
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,19 @@
<tags>test visualstudio testadapter nunit nunit3 dotnet</tags>

<developmentDependency>false</developmentDependency>

<dependencies>
<group targetFramework="net462">
<dependency id="Microsoft.Testing.Extensions.VSTestBridge" version="1.0.1" />
OsirisTerje marked this conversation as resolved.
Show resolved Hide resolved
<dependency id="Microsoft.Testing.Extensions.Telemetry" version="1.0.1" />
Evangelink marked this conversation as resolved.
Show resolved Hide resolved
<dependency id="Microsoft.Testing.Platform.MSBuild" version="1.0.1" />
OsirisTerje marked this conversation as resolved.
Show resolved Hide resolved
</group>
<group targetFramework="netcoreapp3.1">
<dependency id="Microsoft.Testing.Extensions.VSTestBridge" version="1.0.1" />
<dependency id="Microsoft.Testing.Extensions.Telemetry" version="1.0.1" />
<dependency id="Microsoft.Testing.Platform.MSBuild" version="1.0.1" />
</group>
</dependencies>
</metadata>
<files>
<file src="..\..\images\nunit_256.png" target="" />
Expand All @@ -36,6 +49,7 @@
<file src="build\net462\nunit.engine.core.dll" target="build\net462\nunit.engine.core.dll" />
<file src="build\net462\testcentric.engine.metadata.dll" target="build\net462\testcentric.engine.metadata.dll"/>
<file src="build\net462\NUnit3TestAdapter.props" target="build\net462\NUnit3TestAdapter.props" />
<file src="build\net462\NUnit3TestAdapter.targets" target="build\net462\NUnit3TestAdapter.targets" />

<file src="build\netcoreapp3.1\NUnit3.TestAdapter.dll" target="build\netcoreapp3.1\NUnit3.TestAdapter.dll" />
<file src="build\netcoreapp3.1\NUnit3.TestAdapter.pdb" target="build\netcoreapp3.1\NUnit3.TestAdapter.pdb" />
Expand All @@ -44,6 +58,7 @@
<file src="build\netcoreapp3.1\nunit.engine.core.dll" target="build\netcoreapp3.1\nunit.engine.core.dll" />
<file src="build\netcoreapp3.1\testcentric.engine.metadata.dll" target="build\netcoreapp3.1\testcentric.engine.metadata.dll"/>
<file src="build\netcoreapp3.1\NUnit3TestAdapter.props" target="build\netcoreapp3.1\NUnit3TestAdapter.props" />
<file src="build\netcoreapp3.1\NUnit3TestAdapter.targets" target="build\netcoreapp3.1\NUnit3TestAdapter.targets" />


</files>
Expand Down
23 changes: 18 additions & 5 deletions nuget/net462/NUnit3TestAdapter.props
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,11 +1,24 @@
<?xml version="1.0" encoding="utf-8"?>
<Project ToolsVersion="12.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">

<PropertyGroup>
<EnableNUnitRunner Condition=" '$(EnableNUnitRunner)' == '' ">false</EnableNUnitRunner>
Evangelink marked this conversation as resolved.
Show resolved Hide resolved
<IsTestingPlatformApplication>$(EnableNUnitRunner)</IsTestingPlatformApplication>
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some equivalent to the "TestContainer" capability for the new platform.

</PropertyGroup>

<ItemGroup>
<!--
!!! IMPORTANT !!!
DO NOT CHANGE THE GUID, IT'S A WELL KNOWN EXTENSION POINT AND THIS EXTENSION NEEDS TO BE REGISTERED AT THE END
WE HAVE CODE INSIDE THE TASK 'TestingPlatformEntryPoint' TO ENSURE THE ORDER OF THE REGISTRATION BASED ON THIS GUID
-->
<TestingPlatformBuilderHook Include="2E8E7F63-61DB-4EDB-A21E-5BF48279A7B8" Condition=" $(GenerateTestingPlatformEntryPoint) == 'true' " >
<DisplayName>NUnit</DisplayName>
<TypeFullName>NUnit.VisualStudio.TestAdapter.TestingPlatformAdapter.TestingPlatformBuilderHook</TypeFullName>
</TestingPlatformBuilderHook>
Comment on lines +15 to +18
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Register the hook called by Microsoft.Testing.Platform.MSBuild when auto-generating the entry point.

</ItemGroup>

<ItemGroup>
<None Include="$(MSBuildThisFileDirectory)NUnit3.TestAdapter.dll">
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moved to the target, see there for explanation.

<Link>NUnit3.TestAdapter.dll</Link>
<CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
<Visible>False</Visible>
</None>
<None Include="$(MSBuildThisFileDirectory)NUnit3.TestAdapter.pdb" Condition="Exists('$(MSBuildThisFileDirectory)NUnit3.TestAdapter.pdb')">
<Link>NUnit3.TestAdapter.pdb</Link>
<CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
Expand Down
31 changes: 31 additions & 0 deletions nuget/net462/NUnit3TestAdapter.targets
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
<?xml version="1.0" encoding="utf-8"?>
<Project ToolsVersion="12.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">

<!-- Handle the coexistance between Microsoft.Testing.Platform and Microsoft.NET.Test.Sdk -->
<PropertyGroup>
<GenerateTestingPlatformEntryPoint Condition=" '$(GenerateTestingPlatformEntryPoint)' == '' ">$(EnableNUnitRunner)</GenerateTestingPlatformEntryPoint>
<GenerateProgramFile Condition=" '$(EnableNUnitRunner)' == 'true' ">false</GenerateProgramFile>
Comment on lines +6 to +7
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The first line enables/disables the generation of the entry point for the new platform and the second line disables the VSTest entry point if the new platform is enabled.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it necessary to disable the VSTest entry point?
It would be better if the user decides without having to use any parameters. If he runs it using dotnet test, it would use that entry point, if he runs the executable, it would use that entry point.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

VSTest always creates its own entry point. Our new platform also wants to auto-generate it (to simplify user experience) so we need to have them mutually exclusive.

If he runs it using dotnet test

Not really, as you could see in the example because dotnet test is pretty common gesture, we have a mode to hook into it skipping VSTest redirection and calling the runner instead.

</PropertyGroup>

<Choose>
<!-- Avoid false warning about missing reference (msbuild bug) -->
<!-- https://github.com/dotnet/msbuild/issues/9698#issuecomment-1945763467 -->
<When Condition=" '$(EnableNUnitRunner)' == 'true' ">
<ItemGroup>
<Reference Include="NUnit3.TestAdapter">
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With the current layout, the adapter now needs to be added as a reference as it contains the AddNUnit extension point used by the hook. It's definitely possible to produce a different dll if you prefer.

<HintPath>$(MSBuildThisFileDirectory)NUnit3.TestAdapter.dll</HintPath>
</Reference>
</ItemGroup>
</When>
<Otherwise>
<ItemGroup>
<None Include="$(MSBuildThisFileDirectory)NUnit3.TestAdapter.dll">
<Link>NUnit3.TestAdapter.dll</Link>
<CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
<Visible>False</Visible>
</None>
</ItemGroup>
</Otherwise>
</Choose>

</Project>
23 changes: 18 additions & 5 deletions nuget/netcoreapp3.1/NUnit3TestAdapter.props
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,11 +1,24 @@
<?xml version="1.0" encoding="utf-8"?>
<Project ToolsVersion="12.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">

<PropertyGroup>
<EnableNUnitRunner Condition=" '$(EnableNUnitRunner)' == '' ">false</EnableNUnitRunner>
<IsTestingPlatformApplication>$(EnableNUnitRunner)</IsTestingPlatformApplication>
</PropertyGroup>

<ItemGroup>
<!--
!!! IMPORTANT !!!
DO NOT CHANGE THE GUID, IT'S A WELL KNOWN EXTENSION POINT AND THIS EXTENSION NEEDS TO BE REGISTERED AT THE END
WE HAVE CODE INSIDE THE TASK 'TestingPlatformEntryPoint' TO ENSURE THE ORDER OF THE REGISTRATION BASED ON THIS GUID
-->
<TestingPlatformBuilderHook Include="2E8E7F63-61DB-4EDB-A21E-5BF48279A7B8" Condition=" $(GenerateTestingPlatformEntryPoint) == 'true' " >
<DisplayName>NUnit</DisplayName>
<TypeFullName>NUnit.VisualStudio.TestAdapter.TestingPlatformAdapter.TestingPlatformBuilderHook</TypeFullName>
</TestingPlatformBuilderHook>
</ItemGroup>

<ItemGroup>
<None Include="$(MSBuildThisFileDirectory)NUnit3.TestAdapter.dll">
<Link>NUnit3.TestAdapter.dll</Link>
<CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
<Visible>False</Visible>
</None>
<None Include="$(MSBuildThisFileDirectory)NUnit3.TestAdapter.pdb" Condition="Exists('$(MSBuildThisFileDirectory)NUnit3.TestAdapter.pdb')">
<Link>NUnit3.TestAdapter.pdb</Link>
<CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
Expand Down
31 changes: 31 additions & 0 deletions nuget/netcoreapp3.1/NUnit3TestAdapter.targets
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
<?xml version="1.0" encoding="utf-8"?>
<Project ToolsVersion="12.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">

<!-- Handle the coexistance between Microsoft.Testing.Platform and Microsoft.NET.Test.Sdk -->
<PropertyGroup>
<GenerateTestingPlatformEntryPoint Condition=" '$(GenerateTestingPlatformEntryPoint)' == '' ">$(EnableNUnitRunner)</GenerateTestingPlatformEntryPoint>
<GenerateProgramFile Condition=" '$(EnableNUnitRunner)' == 'true' ">false</GenerateProgramFile>
</PropertyGroup>

<Choose>
<!-- Avoid false warning about missing reference (msbuild bug) -->
<!-- https://github.com/dotnet/msbuild/issues/9698#issuecomment-1945763467 -->
<When Condition=" '$(EnableNUnitRunner)' == 'true' ">
<ItemGroup>
<Reference Include="NUnit3.TestAdapter">
<HintPath>$(MSBuildThisFileDirectory)NUnit3.TestAdapter.dll</HintPath>
</Reference>
</ItemGroup>
</When>
<Otherwise>
<ItemGroup>
<None Include="$(MSBuildThisFileDirectory)NUnit3.TestAdapter.dll">
<Link>NUnit3.TestAdapter.dll</Link>
<CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
<Visible>False</Visible>
</None>
</ItemGroup>
</Otherwise>
</Choose>

</Project>
1 change: 1 addition & 0 deletions samples/1-nunit-runner/GlobalUsings.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
global using NUnit.Framework;
15 changes: 15 additions & 0 deletions samples/1-nunit-runner/UnitTest1.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
namespace nunit_runner;

public class Tests
{
[SetUp]
public void Setup()
{
}

[Test]
public void Test1()
{
Assert.Pass();
}
}
28 changes: 28 additions & 0 deletions samples/1-nunit-runner/nunit-runner.csproj
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
<Project Sdk="Microsoft.NET.Sdk">

<PropertyGroup>
<TargetFramework>net6.0</TargetFramework>
<RootNamespace>nunit_runner</RootNamespace>
<ImplicitUsings>enable</ImplicitUsings>
<Nullable>enable</Nullable>

<IsPackable>false</IsPackable>
<IsTestProject>true</IsTestProject>

<!--
These 2 lines are required as some opt-in to the new platform
to minimize disruptancy for end users.
-->
<EnableNUnitRunner>true</EnableNUnitRunner>
<OutputType>Exe</OutputType>
</PropertyGroup>

<ItemGroup>
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="17.6.0" />
<PackageReference Include="NUnit" Version="3.13.3" />
<PackageReference Include="NUnit.Analyzers" Version="3.6.1" />
<!-- Use locally produced NuGet -->
<PackageReference Include="NUnit3TestAdapter" Version="4.6.0-beta.1" />
</ItemGroup>

</Project>
9 changes: 9 additions & 0 deletions samples/1-nunit-runner/readme.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
# NUnit runner

This sample project shows enabling the NUnit runner (name to be defined) powered by Microsoft.Testing.Platform.

When the runner is enabled an executable is produced and when run will use the new platform mode. Thanks to the compatibility layer (Microsoft.Testing.Extensions.VSTestBridge) all features and mode supported by VSTest remain available (dotnet test, VS/VS Code Test Explorer, AzDO VSTest task...) in addition to this new mode (calling the exe).

For example running `dotnet test` will still produce the regular output.

To go further, look at nunit-runner-dotnet-test
1 change: 1 addition & 0 deletions samples/2-nunit-runner-dotnet-test/GlobalUsings.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
global using NUnit.Framework;
15 changes: 15 additions & 0 deletions samples/2-nunit-runner-dotnet-test/UnitTest1.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
namespace nunit_runner_dotnet_test;

public class Tests
{
[SetUp]
public void Setup()
{
}

[Test]
public void Test1()
{
Assert.Pass();
}
}
30 changes: 30 additions & 0 deletions samples/2-nunit-runner-dotnet-test/nunit-runner-dotnet-test.csproj
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
<Project Sdk="Microsoft.NET.Sdk">

<PropertyGroup>
<TargetFramework>net6.0</TargetFramework>
<RootNamespace>nunit_runner_dotnet_test</RootNamespace>
<ImplicitUsings>enable</ImplicitUsings>
<Nullable>enable</Nullable>

<!--
These 2 lines are required as some opt-in to the new platform
to minimize disruptancy for end users.
-->
<EnableNUnitRunner>true</EnableNUnitRunner>
<OutputType>Exe</OutputType>
<!--
Set the property below to update dotnet test behavior to
use the new platform instead of VSTest
-->
<TestingPlatformDotnetTestSupport>true</TestingPlatformDotnetTestSupport>
</PropertyGroup>

<ItemGroup>
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="17.6.0" />
<PackageReference Include="NUnit" Version="3.13.3" />
<PackageReference Include="NUnit.Analyzers" Version="3.6.1" />
<!-- Use locally produced NuGet -->
<PackageReference Include="NUnit3TestAdapter" Version="4.6.0-beta.1" />
</ItemGroup>

</Project>
3 changes: 3 additions & 0 deletions samples/2-nunit-runner-dotnet-test/readme.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
# NUnit runner with support for dotnet test

This sample project builds on top of the previous sample project to show that it's possible to update `dotnet test` to use the new platform.
9 changes: 1 addition & 8 deletions src/NUnitTestAdapter/NUnit.TestAdapter.csproj
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,19 +32,12 @@
</PropertyGroup>

<ItemGroup>
<PackageReference Include="Microsoft.Testing.Extensions.VSTestBridge" Version="1.0.1" />
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The bridge as a deps on object model (Microsoft.TestPlatform.ObjectModel >= 17.5.0). If that's a too big constraint for you, we need to do some manual tests here to see if we would break something when doing a forced downgrade here.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We have been running on the old 11.0 for net framework, and with 15.0.0 for .net.

I would assume that 17.5 would limit this to VS 2022, and the question then is if we should add another target. That is something we could do, to keep backwards compatibility a bit longer.

<PackageReference Include="SourceLink.Create.CommandLine" Version="2.8.3" PrivateAssets="All" />
<PackageReference Include="nunit.engine" Version="3.15.5" />
<PackageReference Include="TestCentric.Metadata" Version="2.0.0" Aliases="TestCentric" />
</ItemGroup>

<ItemGroup Condition="'$(TargetFramework)' == 'net462'">
<PackageReference Include="Microsoft.TestPlatform.ObjectModel" Version="11.0.0" PrivateAssets="All" />
</ItemGroup>

<ItemGroup Condition="'$(TargetFrameworkIdentifier)' == '.NETCoreApp'">
<PackageReference Include="Microsoft.TestPlatform.ObjectModel" Version="15.0.0" PrivateAssets="All" />
</ItemGroup>

<ItemGroup>
<Folder Include="TestFilterConverter\" />
</ItemGroup>
Expand Down