How do you run x64 .NET Framework tests with Microsoft.Testing.Platform.MSBuild?

[bradwilson]

It appears that dotnet test with the new testing platform runs .NET Framework tests compiled as AnyCPU as x86 rather than x64 by default.

How can I choose the bitness of the test execution engine for .NET Framework tests via dotnet test with the new testing platform when the assembly is AnyCPU?

[nohwnd]

How do you repro that?

I am running

dotnet test
dotnet test  bin\Debug\net9.0\mstest113.dll
.\bin\Debug\net9.0\mstest113.exe

In all 3 cases the process is 64-bit.

The dll is being compiled as anyCPU (64-bit preferred) with the project below:

<!-- file mstest113.csproj -->
<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <TargetFramework>net9.0</TargetFramework>
    <ImplicitUsings>enable</ImplicitUsings>
    <Nullable>enable</Nullable>

    <IsPackable>false</IsPackable>
    <IsTestProject>true</IsTestProject>
    <EnableMSTestRunner>true</EnableMSTestRunner>
    <OutputType>Exe</OutputType>
  </PropertyGroup>

  <ItemGroup>
    <PackageReference Include="MSTest" Version="3.2.0" />
    <PackageReference Include="Microsoft.Testing.Extensions.CodeCoverage" Version="17.10.1" />
  </ItemGroup>

  <ItemGroup>
    <Using Include="Microsoft.VisualStudio.TestTools.UnitTesting" />
  </ItemGroup>

</Project>

// file MSTestSettings.cs
[assembly: Parallelize(Scope = ExecutionScope.MethodLevel)]

// file UnitTest1.cs
namespace mstest113;

[TestClass]
public class UnitTest1
{
    [TestMethod]
    public void TestMethod1()
    {
        System.Threading.Thread.Sleep(10_000);
    }
}

[MarcoRossignoli]

I think that could also be related to this xunit/visualstudio.xunit#403 (comment)?

[bradwilson]

My binaries are marked properly:

$ corflags .\src\xunit.v3.assert.tests\bin\Release\net472\xunit.v3.assert.tests.exe

Microsoft (R) .NET Framework CorFlags Conversion Tool.  Version  4.8.3928.0
Copyright (c) Microsoft Corporation.  All rights reserved.

Version   : v4.0.30319
CLR Header: 2.5
PE        : PE32
CorFlags  : 0x9
ILONLY    : 1
32BITREQ  : 0
32BITPREF : 0
Signed    : 1

$ corflags .\src\xunit.v3.assert.tests\bin\Release_x86\net472\xunit.v3.assert.tests.x86.exe

Microsoft (R) .NET Framework CorFlags Conversion Tool.  Version  4.8.3928.0
Copyright (c) Microsoft Corporation.  All rights reserved.

Version   : v4.0.30319
CLR Header: 2.5
PE        : PE32
CorFlags  : 0xb
ILONLY    : 1
32BITREQ  : 1
32BITPREF : 0
Signed    : 1

Yet the resulting binary is run as x86:

$ dotnet test --no-build --framework net472 --configuration Release .\src\xunit.v3.assert.tests

  Run tests: 'C:\Dev\xunit\xunit\src\xunit.v3.assert.tests\bin\Release\net472\xunit.v3.assert.tests.exe' [net472|x86]
  Tests succeeded: 'C:\Dev\xunit\xunit\src\xunit.v3.assert.tests\bin\Release\net472\xunit.v3.assert.tests.exe' [net472|x86]

[bradwilson]

If I simply run the binary directly, you can see it runs in x64 mode:

$ .\src\xunit.v3.assert.tests\bin\Release\net472\xunit.v3.assert.tests.exe

xUnit.net v3 In-Process Runner v0.1.1-pre.382-dev+1ddae2c055 (64-bit .NET Framework 4.8.9181.0)
  Discovering: xunit.v3.assert.tests (method display = ClassAndMethod, method display options = None)
  Discovered:  xunit.v3.assert.tests (885 test cases to be run)
  Starting:    xunit.v3.assert.tests (parallel test collections = on [24 threads], stop on fail = off, explicit = off, seed = 1351680298, culture = invariant)
  Finished:    xunit.v3.assert.tests
=== TEST EXECUTION SUMMARY ===
   xunit.v3.assert.tests  Total: 1024, Errors: 0, Failed: 0, Skipped: 0, Not Run: 0, Time: 0.357s

[bradwilson]

Actually, looking at the log, it may be running in x64 and the output from Microsoft.Testing.Platform.MSBuild is just lying:

Microsoft(R) Testing Platform Execution Command Line Tool
Version: 1.0.2+1686a9c93 (UTC 2024/02/21)
RuntimeInformation: .NET Framework 4.8.9181.0 (x64)
Copyright(c) Microsoft Corporation.  All rights reserved.
  Discovering: xunit.v3.assert.tests (method display = ClassAndMethod, method display options = None)
  Discovered:  xunit.v3.assert.tests (885 test cases to be run)
  Starting:    xunit.v3.assert.tests (parallel test collections = on [24 threads], stop on fail = off, explicit = off, seed = 1351680298, culture = invariant)
  Finished:    xunit.v3.assert.tests
=== TEST EXECUTION SUMMARY ===
   xunit.v3.assert.tests  Total: 1024, Errors: 0, Failed: 0, Skipped: 0, Not Run: 0, Time: 0.353s
Passed! - Failed: 0, Passed: 1024, Skipped: 0, Total: 1024, Duration: 575ms - xunit.v3.assert.tests.exe (.NET Framework 4.8.9181.0)

=== COMMAND LINE ===
C:\Dev\xunit\xunit\src\xunit.v3.assert.tests\bin\Release\net472\xunit.v3.assert.tests.exe --internal-msbuild-node testingplatform.pipe.7d48f74043ac4e1293835f03d5d2cc61

(As an aside, what's up with the garbage characters in the log?)

[bradwilson]

Can verify that the actual x86 binary is run in 32-bit mode:

Microsoft(R) Testing Platform Execution Command Line Tool
Version: 1.0.2+1686a9c93 (UTC 2024/02/21)
RuntimeInformation: .NET Framework 4.8.9181.0 (x86)
Copyright(c) Microsoft Corporation.  All rights reserved.
  Discovering: xunit.v3.assert.tests.x86 (method display = ClassAndMethod, method display options = None)
  Discovered:  xunit.v3.assert.tests.x86 (885 test cases to be run)
  Starting:    xunit.v3.assert.tests.x86 (parallel test collections = on [24 threads], stop on fail = off, explicit = off, seed = 48512649, culture = invariant)
  Finished:    xunit.v3.assert.tests.x86
=== TEST EXECUTION SUMMARY ===
   xunit.v3.assert.tests.x86  Total: 1024, Errors: 0, Failed: 0, Skipped: 0, Not Run: 0, Time: 0.262s
Passed! - Failed: 0, Passed: 1024, Skipped: 0, Total: 1024, Duration: 451ms - xunit.v3.assert.tests.x86.exe (.NET Framework 4.8.9181.0)

=== COMMAND LINE ===
C:\Dev\xunit\xunit\src\xunit.v3.assert.tests\bin\Release_x86\net472\xunit.v3.assert.tests.x86.exe --internal-msbuild-node testingplatform.pipe.f7f78ce502b44f758916bb327d221219

[MarcoRossignoli]

Actually, looking at the log, it may be running in x64 and the output from Microsoft.Testing.Platform.MSBuild is just lying:

This is what we do inside the msbuild target

<PropertyGroup>
    <_TestArchitecture>$(PlatformTarget)</_TestArchitecture>
    <_TestArchitecture Condition="'$(PlatformTarget)' == '' or '$(PlatformTarget)' == 'AnyCpu'">x64</_TestArchitecture>
    <TestingPlatformShowTestsFailure Condition=" '$(TestingPlatformShowTestsFailure)' == '' ">False</TestingPlatformShowTestsFailure>
    <TestingPlatformCaptureOutput Condition=" '$(TestingPlatformCaptureOutput)' == '' " >True</TestingPlatformCaptureOutput>
  </PropertyGroup>
  <Target Name="InvokeTestingPlatform" >
    <InvokeTestingPlatformTask TargetPath="$(TargetPath)"
                               TargetFrameworkIdentifier="$(TargetFrameworkIdentifier)"
                               TargetFramework="$(TargetFramework)"
                               TestArchitecture="$(_TestArchitecture)"
                               TestingPlatformShowTestsFailure="$(TestingPlatformShowTestsFailure)"
                               TestingPlatformCommandLineArguments="$(TestingPlatformCommandLineArguments)"
                               TestingPlatformCaptureOutput="$(TestingPlatformCaptureOutput)"
                               DotnetHostPath="$(DOTNET_HOST_PATH)" />
  </Target>

I saw the same issue and looks like arcade is solving with this default too
https://github.com/dotnet/arcade/blob/4e73daf131ef02e8264b0d3e850c4bdafe6e02b6/src/Microsoft.DotNet.Arcade.Sdk/tools/Tests.targets#L31

Can run under binlog? dotnet test --no-build --framework net472 --configuration Release .\src\xunit.v3.assert.tests /bl and check the param TestArchitecture?

[bradwilson]

@MarcoRossignoli _TestArchitecture = x86

Also, this feels odd:

Platform = AnyCPU
PlatformName = AnyCPU
Platforms = AnyCPU
PlatformTarget = x86

I will set <PlatformTarget>x86</PlatformTarget> when building the x86 configurations:

  <!-- ============= -->
  <!-- 32-bit builds -->

  <PropertyGroup Condition="$(Configuration.EndsWith('_x86'))">
    <AssemblyName>$(AssemblyName).x86</AssemblyName>
    <DefineConstants>$(DefineConstants);BUILD_X86</DefineConstants>
    <PlatformTarget>x86</PlatformTarget>
    <RunCommand Condition=" '$(OS)' == 'Windows_NT' ">$(MSBuildProgramFiles32)\dotnet\dotnet</RunCommand>
  </PropertyGroup>

Otherwise, there's no mention of PlatformTarget (or AnyCPU for that matter) anywhere in my source tree.

[bradwilson]

@MarcoRossignoli I just went down a rabbit hole just trying to set <PlatformTarget>AnyCPU</PlatformTarget> by default and suddenly the x86 builds don't work, because apparently dotnet restore does not understand how to handle projects that have multiple conditional PlatformTarget values. To make builds work I either need to create new projects for the x86 versions of everything, or I need to run msbuild (and not dotnet msbuild) twice, including both Restore and Build targets for the given configuration, which changes the package restore behavior (which means I'm in a constant state of the restore being out of date, depending on which configuration I built most recently).

That said...

After I hacked it together (😁) to at least prove it does the right thing, I get the right output:

I guess now I'm off to make extra projects, since I appear to have violated some dotnet restore state rule but gotten away with it because of an implicit <PlatformTarget>x86</PlatformTarget> even when it was building AnyCPU binaries. 🤷🏼

[bradwilson]

Annoying but done. 😭😂 xunit/xunit@4e1033d