Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Running dotnet test in libraries failing to execute net462 targets #68384

Closed
eiriktsarpalis opened this issue Apr 22, 2022 · 13 comments · Fixed by #68863
Closed

Running dotnet test in libraries failing to execute net462 targets #68384

eiriktsarpalis opened this issue Apr 22, 2022 · 13 comments · Fixed by #68863

Comments

@eiriktsarpalis
Copy link
Member

Running the following command from the base of the runtime repo

dotnet test src/libraries/System.Text.Json/tests/System.Text.Json.Tests/System.Text.Json.Tests.csproj -f net462

Will produce the warning

No test is available in C:\Users\eitsarpa\devel\dotnet\runtime\artifacts\bin\System.Text.Json.Tests\Debug\net462\System.Text.Json.Tests.dll. 
Make sure that test discoverer & executors are registered and platform & framework version settings are appropriate and try again.

Tests run as expected when using the Test target:

dotnet build -t:Test src/libraries/System.Text.Json/tests/System.Text.Json.Tests/System.Text.Json.Tests.csproj -f net462

This is an issue for my workflow, since running dotnet test solution-wide will skip all our netfx targets.

cc @ViktorHofer

@dotnet-issue-labeler dotnet-issue-labeler bot added area-System.Text.Json untriaged New issue has not been triaged by the area owner labels Apr 22, 2022
@ghost
Copy link

ghost commented Apr 22, 2022

Tagging subscribers to this area: @dotnet/area-system-text-json, @gregsdennis
See info in area-owners.md if you want to be subscribed.

Issue Details

Running the following command from the base of the runtime repo

dotnet test src/libraries/System.Text.Json/tests/System.Text.Json.Tests/System.Text.Json.Tests.csproj -f net462

Will produce the warning

No test is available in C:\Users\eitsarpa\devel\dotnet\runtime\artifacts\bin\System.Text.Json.Tests\Debug\net462\System.Text.Json.Tests.dll. 
Make sure that test discoverer & executors are registered and platform & framework version settings are appropriate and try again.

Tests run as expected when using the Test target:

dotnet build -t:Test src/libraries/System.Text.Json/tests/System.Text.Json.Tests/System.Text.Json.Tests.csproj -f net462

This is an issue for my workflow, since running dotnet test solution-wide will skip all our netfx targets.

cc @ViktorHofer

Author: eiriktsarpalis
Assignees: -
Labels:

area-System.Text.Json, untriaged

Milestone: -

@ghost
Copy link

ghost commented Apr 22, 2022

Tagging subscribers to this area: @dotnet/runtime-infrastructure
See info in area-owners.md if you want to be subscribed.

Issue Details

Running the following command from the base of the runtime repo

dotnet test src/libraries/System.Text.Json/tests/System.Text.Json.Tests/System.Text.Json.Tests.csproj -f net462

Will produce the warning

No test is available in C:\Users\eitsarpa\devel\dotnet\runtime\artifacts\bin\System.Text.Json.Tests\Debug\net462\System.Text.Json.Tests.dll. 
Make sure that test discoverer & executors are registered and platform & framework version settings are appropriate and try again.

Tests run as expected when using the Test target:

dotnet build -t:Test src/libraries/System.Text.Json/tests/System.Text.Json.Tests/System.Text.Json.Tests.csproj -f net462

This is an issue for my workflow, since running dotnet test solution-wide will skip all our netfx targets.

cc @ViktorHofer

Author: eiriktsarpalis
Assignees: -
Labels:

area-Infrastructure, untriaged

Milestone: -

@ViktorHofer
Copy link
Member

ViktorHofer commented Apr 22, 2022

I can repro that failure and found the culprit by emitting diagnostic files: .\dotnet.cmd test src/libraries/System.Text.Json/tests/System.Text.Json.Tests/System.Text.Json.Tests.csproj -f net462 --diag "diag.txt"

Failed to get types from assembly 'Microsoft.Diagnostics.NETCore.Client, Version=0.2.4.21401, Culture=neutral, PublicKeyToken=31bf3856ad364e35'.  Skipping test extension scan for this assembly.  Error: System.Reflection.ReflectionTypeLoadException: Unable to load one or more of the requested types.
Could not load file or assembly 'Microsoft.Bcl.AsyncInterfaces, Version=1.0.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51'. The system cannot find the file specified.
   at System.Reflection.RuntimeModule.GetTypes(RuntimeModule module)
   at System.Reflection.Assembly.GetTypes()
   at Microsoft.VisualStudio.TestPlatform.Common.ExtensionFramework.TestPluginDiscoverer.GetTestExtensionsFromAssembly[TPluginInfo,TExtension](Assembly assembly, Dictionary`2 pluginInfos)
System.IO.FileNotFoundException: Could not load file or assembly 'Microsoft.Bcl.AsyncInterfaces, Version=1.0.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51'. The system cannot find the file specified.
File name: 'Microsoft.Bcl.AsyncInterfaces, Version=1.0.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51'
 ---> System.IO.FileNotFoundException: Could not load file or assembly 'Microsoft.Bcl.AsyncInterfaces, Culture=neutral, PublicKeyToken=null'. The system cannot find the file specified.
File name: 'Microsoft.Bcl.AsyncInterfaces, Culture=neutral, PublicKeyToken=null'
   at System.Reflection.RuntimeAssembly.InternalLoad(ObjectHandleOnStack assemblyName, ObjectHandleOnStack requestingAssembly, StackCrawlMarkHandle stackMark, Boolean throwOnFileNotFound, ObjectHandleOnStack assemblyLoadContext, ObjectHandleOnStack retAssembly)
   at System.Reflection.RuntimeAssembly.InternalLoad(AssemblyName assemblyName, RuntimeAssembly requestingAssembly, StackCrawlMark& stackMark, Boolean throwOnFileNotFound, AssemblyLoadContext assemblyLoadContext)
   at System.Reflection.Assembly.Load(AssemblyName assemblyRef)
   at Microsoft.VisualStudio.TestPlatform.Common.ExtensionFramework.TestPluginCache.CurrentDomainAssemblyResolve(Object sender, AssemblyResolveEventArgs args)
   at Microsoft.VisualStudio.TestPlatform.PlatformAbstractions.PlatformAssemblyResolver.AssemblyResolverEvent(Object sender, Object eventArgs)
   at System.Runtime.Loader.AssemblyLoadContext.GetFirstResolvedAssemblyFromResolvingEvent(AssemblyName assemblyName)
   at System.Runtime.Loader.AssemblyLoadContext.ResolveUsingEvent(AssemblyName assemblyName)
   at System.Runtime.Loader.AssemblyLoadContext.ResolveUsingResolvingEvent(IntPtr gchManagedAssemblyLoadContext, AssemblyName assemblyName)
TpTrace Warning: 0 : 1844, 1, 2022/04/22, 15:45:57.545, 5018653626776, vstest.console.dll, LoaderExceptions: System.IO.FileNotFoundException: Could not load file or assembly 'Microsoft.Bcl.AsyncInterfaces, Version=1.0.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51'. The system cannot find the file specified.
File name: 'Microsoft.Bcl.AsyncInterfaces, Version=1.0.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51'
 ---> System.IO.FileNotFoundException: Could not load file or assembly 'Microsoft.Bcl.AsyncInterfaces, Culture=neutral, PublicKeyToken=null'. The system cannot find the file specified.
File name: 'Microsoft.Bcl.AsyncInterfaces, Culture=neutral, PublicKeyToken=null'
   at System.Reflection.RuntimeAssembly.InternalLoad(ObjectHandleOnStack assemblyName, ObjectHandleOnStack requestingAssembly, StackCrawlMarkHandle stackMark, Boolean throwOnFileNotFound, ObjectHandleOnStack assemblyLoadContext, ObjectHandleOnStack retAssembly)
   at System.Reflection.RuntimeAssembly.InternalLoad(AssemblyName assemblyName, RuntimeAssembly requestingAssembly, StackCrawlMark& stackMark, Boolean throwOnFileNotFound, AssemblyLoadContext assemblyLoadContext)
   at System.Reflection.Assembly.Load(AssemblyName assemblyRef)
   at Microsoft.VisualStudio.TestPlatform.Common.ExtensionFramework.TestPluginCache.CurrentDomainAssemblyResolve(Object sender, AssemblyResolveEventArgs args)
   at Microsoft.VisualStudio.TestPlatform.PlatformAbstractions.PlatformAssemblyResolver.AssemblyResolverEvent(Object sender, Object eventArgs)
   at System.Runtime.Loader.AssemblyLoadContext.GetFirstResolvedAssemblyFromResolvingEvent(AssemblyName assemblyName)
   at System.Runtime.Loader.AssemblyLoadContext.ResolveUsingEvent(AssemblyName assemblyName)
   at System.Runtime.Loader.AssemblyLoadContext.ResolveUsingResolvingEvent(IntPtr gchManagedAssemblyLoadContext, AssemblyName assemblyName)

@nohwnd @MarcoRossignoli is that a known issue? We can't invoke net462 tests with vstest anymore. Happening with a 6.0.100 SDK.

diag.txt
diag.datacollector.22-04-22_15-49-34_12043_1.txt
diag.host.22-04-22_15-49-35_86061_1.txt

@nohwnd
Copy link
Member

nohwnd commented Apr 26, 2022

Not a known issue, the logs you post are probably not the culprit:

I think this is, you can see that there are no runners or discoverers on the bottom. What is also curious is that somewhere in the middle the resolve changes from xunitnet462 to xunitnet452:

TpTrace Information: 0 : 3024, 3, 2022/04/22, 15:49:38.071, 5020858885600, testhost.net462.exe, AssemblyResolver.OnResolve: xunit.runner.visualstudio.testadapter: Resolving assembly.
TpTrace Information: 0 : 3024, 3, 2022/04/22, 15:49:38.071, 5020858887303, testhost.net462.exe, AssemblyResolver.OnResolve: xunit.runner.visualstudio.testadapter: Searching in: 'C:\git\runtime5\.dotnet\sdk\6.0.100\Extensions'.
TpTrace Information: 0 : 3024, 3, 2022/04/22, 15:49:38.071, 5020858889149, testhost.net462.exe, AssemblyResolver.OnResolve: xunit.runner.visualstudio.testadapter: Assembly path does not exist: 'C:\git\runtime5\.dotnet\sdk\6.0.100\Extensions\xunit.runner.visualstudio.testadapter.dll', returning.
TpTrace Information: 0 : 3024, 3, 2022/04/22, 15:49:38.071, 5020858890740, testhost.net462.exe, AssemblyResolver.OnResolve: xunit.runner.visualstudio.testadapter: Assembly path does not exist: 'C:\git\runtime5\.dotnet\sdk\6.0.100\Extensions\xunit.runner.visualstudio.testadapter.exe', returning.
TpTrace Information: 0 : 3024, 3, 2022/04/22, 15:49:38.071, 5020858891754, testhost.net462.exe, AssemblyResolver.OnResolve: xunit.runner.visualstudio.testadapter: Searching in: 'C:\git\runtime5\artifacts\bin\System.Text.Json.Tests\Debug\net462'.
TpTrace Information: 0 : 3024, 3, 2022/04/22, 15:49:38.073, 5020858910868, testhost.net462.exe, AssemblyResolver.OnResolve: xunit.runner.visualstudio.testadapter: Loading assembly 'C:\git\runtime5\artifacts\bin\System.Text.Json.Tests\Debug\net462\xunit.runner.visualstudio.testadapter.dll'.
TpTrace Information: 0 : 3024, 3, 2022/04/22, 15:49:38.079, 5020858965823, testhost.net462.exe, AssemblyResolver.OnResolve: Resolved assembly: xunit.runner.visualstudio.testadapter, from path: C:\git\runtime5\artifacts\bin\System.Text.Json.Tests\Debug\net462\xunit.runner.visualstudio.testadapter.dll
TpTrace Information: 0 : 3024, 3, 2022/04/22, 15:49:38.085, 5020859030522, testhost.net462.exe, AssemblyResolver.OnResolve: xunit.runner.utility.net452, Version=2.4.1.0, Culture=neutral, PublicKeyToken=8d05b1bb7a6fdb6c: Resolving assembly.
TpTrace Information: 0 : 3024, 3, 2022/04/22, 15:49:38.085, 5020859033554, testhost.net462.exe, AssemblyResolver.OnResolve: xunit.runner.utility.net452, Version=2.4.1.0, Culture=neutral, PublicKeyToken=8d05b1bb7a6fdb6c: Searching in: 'C:\git\runtime5\.dotnet\sdk\6.0.100\Extensions'.
TpTrace Information: 0 : 3024, 3, 2022/04/22, 15:49:38.086, 5020859035827, testhost.net462.exe, AssemblyResolver.OnResolve: xunit.runner.utility.net452, Version=2.4.1.0, Culture=neutral, PublicKeyToken=8d05b1bb7a6fdb6c: Assembly path does not exist: 'C:\git\runtime5\.dotnet\sdk\6.0.100\Extensions\xunit.runner.utility.net452.dll', returning.
TpTrace Information: 0 : 3024, 3, 2022/04/22, 15:49:38.086, 5020859037792, testhost.net462.exe, AssemblyResolver.OnResolve: xunit.runner.utility.net452, Version=2.4.1.0, Culture=neutral, PublicKeyToken=8d05b1bb7a6fdb6c: Assembly path does not exist: 'C:\git\runtime5\.dotnet\sdk\6.0.100\Extensions\xunit.runner.utility.net452.exe', returning.
TpTrace Information: 0 : 3024, 3, 2022/04/22, 15:49:38.086, 5020859039047, testhost.net462.exe, AssemblyResolver.OnResolve: xunit.runner.utility.net452, Version=2.4.1.0, Culture=neutral, PublicKeyToken=8d05b1bb7a6fdb6c: Searching in: 'C:\git\runtime5\artifacts\bin\System.Text.Json.Tests\Debug\net462'.
TpTrace Information: 0 : 3024, 3, 2022/04/22, 15:49:38.088, 5020859055062, testhost.net462.exe, AssemblyResolver.OnResolve: xunit.runner.utility.net452, Version=2.4.1.0, Culture=neutral, PublicKeyToken=8d05b1bb7a6fdb6c: File exists but version/public key is wrong. Try next extension.
TpTrace Information: 0 : 3024, 3, 2022/04/22, 15:49:38.088, 5020859057794, testhost.net462.exe, AssemblyResolver.OnResolve: xunit.runner.utility.net452, Version=2.4.1.0, Culture=neutral, PublicKeyToken=8d05b1bb7a6fdb6c: Assembly path does not exist: 'C:\git\runtime5\artifacts\bin\System.Text.Json.Tests\Debug\net462\xunit.runner.utility.net452.exe', returning.
TpTrace Information: 0 : 3024, 3, 2022/04/22, 15:49:38.088, 5020859058997, testhost.net462.exe, AssemblyResolver.OnResolve: xunit.runner.utility.net452, Version=2.4.1.0, Culture=neutral, PublicKeyToken=8d05b1bb7a6fdb6c: Searching in: 'C:\git\runtime5\.dotnet\sdk\6.0.100\TestHost'.
TpTrace Information: 0 : 3024, 3, 2022/04/22, 15:49:38.088, 5020859060635, testhost.net462.exe, AssemblyResolver.OnResolve: xunit.runner.utility.net452, Version=2.4.1.0, Culture=neutral, PublicKeyToken=8d05b1bb7a6fdb6c: Assembly path does not exist: 'C:\git\runtime5\.dotnet\sdk\6.0.100\TestHost\xunit.runner.utility.net452.dll', returning.
TpTrace Information: 0 : 3024, 3, 2022/04/22, 15:49:38.088, 5020859062327, testhost.net462.exe, AssemblyResolver.OnResolve: xunit.runner.utility.net452, Version=2.4.1.0, Culture=neutral, PublicKeyToken=8d05b1bb7a6fdb6c: Assembly path does not exist: 'C:\git\runtime5\.dotnet\sdk\6.0.100\TestHost\xunit.runner.utility.net452.exe', returning.
TpTrace Information: 0 : 3024, 3, 2022/04/22, 15:49:38.088, 5020859063439, testhost.net462.exe, AssemblyResolver.OnResolve: xunit.runner.utility.net452, Version=2.4.1.0, Culture=neutral, PublicKeyToken=8d05b1bb7a6fdb6c: Failed to load assembly.
TpTrace Verbose: 0 : 3024, 3, 2022/04/22, 15:49:38.088, 5020859064597, testhost.net462.exe, CurrentDomainAssemblyResolve: Resolving assembly 'xunit.runner.utility.net452, Version=2.4.1.0, Culture=neutral, PublicKeyToken=8d05b1bb7a6fdb6c'.
TpTrace Information: 0 : 3024, 3, 2022/04/22, 15:49:38.090, 5020859083975, testhost.net462.exe, AssemblyResolver.OnResolve: xunit.runner.utility.net452, Version=2.4.1.0, Culture=neutral, PublicKeyToken=8d05b1bb7a6fdb6c: Resolving assembly.
TpTrace Information: 0 : 3024, 3, 2022/04/22, 15:49:38.091, 5020859086234, testhost.net462.exe, AssemblyResolver.OnResolve: xunit.runner.utility.net452, Version=2.4.1.0, Culture=neutral, PublicKeyToken=8d05b1bb7a6fdb6c: Resolved from cache.
TpTrace Verbose: 0 : 3024, 3, 2022/04/22, 15:49:38.091, 5020859087777, testhost.net462.exe, CurrentDomainAssemblyResolve: Resolving assembly 'xunit.runner.utility.net452, Version=2.4.1.0, Culture=neutral, PublicKeyToken=8d05b1bb7a6fdb6c'.
TpTrace Verbose: 0 : 3024, 3, 2022/04/22, 15:49:38.091, 5020859088993, testhost.net462.exe, CurrentDomainAssemblyResolve: Failed to resolve assembly 'xunit.runner.utility.net452, Version=2.4.1.0, Culture=neutral, PublicKeyToken=8d05b1bb7a6fdb6c'.
TpTrace Verbose: 0 : 3024, 3, 2022/04/22, 15:49:38.091, 5020859092348, testhost.net462.exe, CurrentDomainAssemblyResolve: Failed to resolve assembly 'xunit.runner.utility.net452, Version=2.4.1.0, Culture=neutral, PublicKeyToken=8d05b1bb7a6fdb6c'.
TpTrace Warning: 0 : 3024, 3, 2022/04/22, 15:49:38.103, 5020859210208, testhost.net462.exe, TestPluginDiscoverer: Failed to get types from assembly 'xunit.runner.visualstudio.testadapter, Version=2.4.0.0, Culture=neutral, PublicKeyToken=8d05b1bb7a6fdb6c'.  Skipping test extension scan for this assembly.  Error: System.Reflection.ReflectionTypeLoadException: Unable to load one or more of the requested types. Retrieve the LoaderExceptions property for more information.
   at System.Reflection.RuntimeModule.GetTypes(RuntimeModule module)
   at System.Reflection.Assembly.GetTypes()
   at Microsoft.VisualStudio.TestPlatform.Common.ExtensionFramework.TestPluginDiscoverer.GetTestExtensionsFromAssembly[TPluginInfo,TExtension](Assembly assembly, Dictionary`2 pluginInfos)
TpTrace Warning: 0 : 3024, 3, 2022/04/22, 15:49:38.103, 5020859214317, testhost.net462.exe, LoaderExceptions: System.IO.FileNotFoundException: Could not load file or assembly 'xunit.runner.utility.net452, Version=2.4.1.0, Culture=neutral, PublicKeyToken=8d05b1bb7a6fdb6c' or one of its dependencies. The system cannot find the file specified.
File name: 'xunit.runner.utility.net452, Version=2.4.1.0, Culture=neutral, PublicKeyToken=8d05b1bb7a6fdb6c' ---> System.IO.FileNotFoundException: Could not load file or assembly 'xunit.runner.utility.net452, Version=2.4.1.0, Culture=neutral, PublicKeyToken=8d05b1bb7a6fdb6c' or one of its dependencies. The system cannot find the file specified.
File name: 'xunit.runner.utility.net452, Version=2.4.1.0, Culture=neutral, PublicKeyToken=8d05b1bb7a6fdb6c'
   at System.Reflection.RuntimeAssembly._nLoad(AssemblyName fileName, String codeBase, Evidence assemblySecurity, RuntimeAssembly locationHint, StackCrawlMark& stackMark, IntPtr pPrivHostBinder, Boolean throwOnFileNotFound, Boolean forIntrospection, Boolean suppressSecurityChecks)
   at System.Reflection.RuntimeAssembly.InternalLoadAssemblyName(AssemblyName assemblyRef, Evidence assemblySecurity, RuntimeAssembly reqAssembly, StackCrawlMark& stackMark, IntPtr pPrivHostBinder, Boolean throwOnFileNotFound, Boolean forIntrospection, Boolean suppressSecurityChecks)
   at System.Reflection.Assembly.Load(AssemblyName assemblyRef)
   at Microsoft.VisualStudio.TestPlatform.Common.ExtensionFramework.TestPluginCache.CurrentDomainAssemblyResolve(Object sender, AssemblyResolveEventArgs args)
   at System.AppDomain.OnAssemblyResolveEvent(RuntimeAssembly assembly, String assemblyFullName)

WRN: Assembly binding logging is turned OFF.
To enable assembly bind failure logging, set the registry value [HKLM\Software\Microsoft\Fusion!EnableLog] (DWORD) to 1.
Note: There is some performance penalty associated with assembly bind failure logging.
To turn this feature off, remove the registry value [HKLM\Software\Microsoft\Fusion!EnableLog].



TpTrace Warning: 0 : 3024, 3, 2022/04/22, 15:49:38.104, 5020859218201, testhost.net462.exe, LoaderExceptions: System.IO.FileNotFoundException: Could not load file or assembly 'xunit.runner.utility.net452, Version=2.4.1.0, Culture=neutral, PublicKeyToken=8d05b1bb7a6fdb6c' or one of its dependencies. The system cannot find the file specified.
File name: 'xunit.runner.utility.net452, Version=2.4.1.0, Culture=neutral, PublicKeyToken=8d05b1bb7a6fdb6c'
   at System.Reflection.RuntimeAssembly._nLoad(AssemblyName fileName, String codeBase, Evidence assemblySecurity, RuntimeAssembly locationHint, StackCrawlMark& stackMark, IntPtr pPrivHostBinder, Boolean throwOnFileNotFound, Boolean forIntrospection, Boolean suppressSecurityChecks)
   at System.Reflection.RuntimeAssembly.InternalLoadAssemblyName(AssemblyName assemblyRef, Evidence assemblySecurity, RuntimeAssembly reqAssembly, StackCrawlMark& stackMark, IntPtr pPrivHostBinder, Boolean throwOnFileNotFound, Boolean forIntrospection, Boolean suppressSecurityChecks)
   at System.Reflection.Assembly.Load(AssemblyName assemblyRef)
   at Microsoft.VisualStudio.TestPlatform.Common.ExtensionFramework.TestPluginCache.CurrentDomainAssemblyResolve(Object sender, AssemblyResolveEventArgs args)
   at System.AppDomain.OnAssemblyResolveEvent(RuntimeAssembly assembly, String assemblyFullName)

WRN: Assembly binding logging is turned OFF.
To enable assembly bind failure logging, set the registry value [HKLM\Software\Microsoft\Fusion!EnableLog] (DWORD) to 1.
Note: There is some performance penalty associated with assembly bind failure logging.
To turn this feature off, remove the registry value [HKLM\Software\Microsoft\Fusion!EnableLog].

TpTrace Warning: 0 : 3024, 3, 2022/04/22, 15:49:38.104, 5020859221967, testhost.net462.exe, LoaderExceptions: System.IO.FileNotFoundException: Could not load file or assembly 'xunit.runner.utility.net452, Version=2.4.1.0, Culture=neutral, PublicKeyToken=8d05b1bb7a6fdb6c' or one of its dependencies. The system cannot find the file specified.
File name: 'xunit.runner.utility.net452, Version=2.4.1.0, Culture=neutral, PublicKeyToken=8d05b1bb7a6fdb6c'
   at System.Reflection.RuntimeAssembly._nLoad(AssemblyName fileName, String codeBase, Evidence assemblySecurity, RuntimeAssembly locationHint, StackCrawlMark& stackMark, IntPtr pPrivHostBinder, Boolean throwOnFileNotFound, Boolean forIntrospection, Boolean suppressSecurityChecks)
   at System.Reflection.RuntimeAssembly.InternalLoadAssemblyName(AssemblyName assemblyRef, Evidence assemblySecurity, RuntimeAssembly reqAssembly, StackCrawlMark& stackMark, IntPtr pPrivHostBinder, Boolean throwOnFileNotFound, Boolean forIntrospection, Boolean suppressSecurityChecks)
   at System.Reflection.Assembly.Load(AssemblyName assemblyRef)
   at Microsoft.VisualStudio.TestPlatform.Common.ExtensionFramework.TestPluginCache.CurrentDomainAssemblyResolve(Object sender, AssemblyResolveEventArgs args)
   at System.AppDomain.OnAssemblyResolveEvent(RuntimeAssembly assembly, String assemblyFullName)

WRN: Assembly binding logging is turned OFF.
To enable assembly bind failure logging, set the registry value [HKLM\Software\Microsoft\Fusion!EnableLog] (DWORD) to 1.
Note: There is some performance penalty associated with assembly bind failure logging.
To turn this feature off, remove the registry value [HKLM\Software\Microsoft\Fusion!EnableLog].

TpTrace Warning: 0 : 3024, 3, 2022/04/22, 15:49:38.105, 5020859225922, testhost.net462.exe, LoaderExceptions: System.IO.FileNotFoundException: Could not load file or assembly 'xunit.runner.utility.net452, Version=2.4.1.0, Culture=neutral, PublicKeyToken=8d05b1bb7a6fdb6c' or one of its dependencies. The system cannot find the file specified.
File name: 'xunit.runner.utility.net452, Version=2.4.1.0, Culture=neutral, PublicKeyToken=8d05b1bb7a6fdb6c'
   at System.Reflection.RuntimeAssembly._nLoad(AssemblyName fileName, String codeBase, Evidence assemblySecurity, RuntimeAssembly locationHint, StackCrawlMark& stackMark, IntPtr pPrivHostBinder, Boolean throwOnFileNotFound, Boolean forIntrospection, Boolean suppressSecurityChecks)
   at System.Reflection.RuntimeAssembly.InternalLoadAssemblyName(AssemblyName assemblyRef, Evidence assemblySecurity, RuntimeAssembly reqAssembly, StackCrawlMark& stackMark, IntPtr pPrivHostBinder, Boolean throwOnFileNotFound, Boolean forIntrospection, Boolean suppressSecurityChecks)
   at System.Reflection.Assembly.Load(AssemblyName assemblyRef)
   at Microsoft.VisualStudio.TestPlatform.Common.ExtensionFramework.TestPluginCache.CurrentDomainAssemblyResolve(Object sender, AssemblyResolveEventArgs args)
   at System.AppDomain.OnAssemblyResolveEvent(RuntimeAssembly assembly, String assemblyFullName)

WRN: Assembly binding logging is turned OFF.
To enable assembly bind failure logging, set the registry value [HKLM\Software\Microsoft\Fusion!EnableLog] (DWORD) to 1.
Note: There is some performance penalty associated with assembly bind failure logging.
To turn this feature off, remove the registry value [HKLM\Software\Microsoft\Fusion!EnableLog].

TpTrace Warning: 0 : 3024, 3, 2022/04/22, 15:49:38.105, 5020859229454, testhost.net462.exe, LoaderExceptions: System.IO.FileNotFoundException: Could not load file or assembly 'xunit.runner.utility.net452, Version=2.4.1.0, Culture=neutral, PublicKeyToken=8d05b1bb7a6fdb6c' or one of its dependencies. The system cannot find the file specified.
File name: 'xunit.runner.utility.net452, Version=2.4.1.0, Culture=neutral, PublicKeyToken=8d05b1bb7a6fdb6c'
   at System.Reflection.RuntimeAssembly._nLoad(AssemblyName fileName, String codeBase, Evidence assemblySecurity, RuntimeAssembly locationHint, StackCrawlMark& stackMark, IntPtr pPrivHostBinder, Boolean throwOnFileNotFound, Boolean forIntrospection, Boolean suppressSecurityChecks)
   at System.Reflection.RuntimeAssembly.InternalLoadAssemblyName(AssemblyName assemblyRef, Evidence assemblySecurity, RuntimeAssembly reqAssembly, StackCrawlMark& stackMark, IntPtr pPrivHostBinder, Boolean throwOnFileNotFound, Boolean forIntrospection, Boolean suppressSecurityChecks)
   at System.Reflection.Assembly.Load(AssemblyName assemblyRef)
   at Microsoft.VisualStudio.TestPlatform.Common.ExtensionFramework.TestPluginCache.CurrentDomainAssemblyResolve(Object sender, AssemblyResolveEventArgs args)
   at System.AppDomain.OnAssemblyResolveEvent(RuntimeAssembly assembly, String assemblyFullName)

WRN: Assembly binding logging is turned OFF.
To enable assembly bind failure logging, set the registry value [HKLM\Software\Microsoft\Fusion!EnableLog] (DWORD) to 1.
Note: There is some performance penalty associated with assembly bind failure logging.
To turn this feature off, remove the registry value [HKLM\Software\Microsoft\Fusion!EnableLog].

TpTrace Warning: 0 : 3024, 3, 2022/04/22, 15:49:38.105, 5020859233005, testhost.net462.exe, LoaderExceptions: System.IO.FileNotFoundException: Could not load file or assembly 'xunit.runner.utility.net452, Version=2.4.1.0, Culture=neutral, PublicKeyToken=8d05b1bb7a6fdb6c' or one of its dependencies. The system cannot find the file specified.
File name: 'xunit.runner.utility.net452, Version=2.4.1.0, Culture=neutral, PublicKeyToken=8d05b1bb7a6fdb6c'
   at System.Reflection.RuntimeAssembly._nLoad(AssemblyName fileName, String codeBase, Evidence assemblySecurity, RuntimeAssembly locationHint, StackCrawlMark& stackMark, IntPtr pPrivHostBinder, Boolean throwOnFileNotFound, Boolean forIntrospection, Boolean suppressSecurityChecks)
   at System.Reflection.RuntimeAssembly.InternalLoadAssemblyName(AssemblyName assemblyRef, Evidence assemblySecurity, RuntimeAssembly reqAssembly, StackCrawlMark& stackMark, IntPtr pPrivHostBinder, Boolean throwOnFileNotFound, Boolean forIntrospection, Boolean suppressSecurityChecks)
   at System.Reflection.Assembly.Load(AssemblyName assemblyRef)
   at Microsoft.VisualStudio.TestPlatform.Common.ExtensionFramework.TestPluginCache.CurrentDomainAssemblyResolve(Object sender, AssemblyResolveEventArgs args)
   at System.AppDomain.OnAssemblyResolveEvent(RuntimeAssembly assembly, String assemblyFullName)

WRN: Assembly binding logging is turned OFF.
To enable assembly bind failure logging, set the registry value [HKLM\Software\Microsoft\Fusion!EnableLog] (DWORD) to 1.
Note: There is some performance penalty associated with assembly bind failure logging.
To turn this feature off, remove the registry value [HKLM\Software\Microsoft\Fusion!EnableLog].

TpTrace Warning: 0 : 3024, 3, 2022/04/22, 15:49:38.106, 5020859236533, testhost.net462.exe, LoaderExceptions: System.IO.FileNotFoundException: Could not load file or assembly 'xunit.runner.utility.net452, Version=2.4.1.0, Culture=neutral, PublicKeyToken=8d05b1bb7a6fdb6c' or one of its dependencies. The system cannot find the file specified.
File name: 'xunit.runner.utility.net452, Version=2.4.1.0, Culture=neutral, PublicKeyToken=8d05b1bb7a6fdb6c'
   at System.Reflection.RuntimeAssembly._nLoad(AssemblyName fileName, String codeBase, Evidence assemblySecurity, RuntimeAssembly locationHint, StackCrawlMark& stackMark, IntPtr pPrivHostBinder, Boolean throwOnFileNotFound, Boolean forIntrospection, Boolean suppressSecurityChecks)
   at System.Reflection.RuntimeAssembly.InternalLoadAssemblyName(AssemblyName assemblyRef, Evidence assemblySecurity, RuntimeAssembly reqAssembly, StackCrawlMark& stackMark, IntPtr pPrivHostBinder, Boolean throwOnFileNotFound, Boolean forIntrospection, Boolean suppressSecurityChecks)
   at System.Reflection.Assembly.Load(AssemblyName assemblyRef)
   at Microsoft.VisualStudio.TestPlatform.Common.ExtensionFramework.TestPluginCache.CurrentDomainAssemblyResolve(Object sender, AssemblyResolveEventArgs args)
   at System.AppDomain.OnAssemblyResolveEvent(RuntimeAssembly assembly, String assemblyFullName)

WRN: Assembly binding logging is turned OFF.
To enable assembly bind failure logging, set the registry value [HKLM\Software\Microsoft\Fusion!EnableLog] (DWORD) to 1.
Note: There is some performance penalty associated with assembly bind failure logging.
To turn this feature off, remove the registry value [HKLM\Software\Microsoft\Fusion!EnableLog].

TpTrace Warning: 0 : 3024, 3, 2022/04/22, 15:49:38.106, 5020859239881, testhost.net462.exe, LoaderExceptions: System.IO.FileNotFoundException: Could not load file or assembly 'xunit.runner.utility.net452, Version=2.4.1.0, Culture=neutral, PublicKeyToken=8d05b1bb7a6fdb6c' or one of its dependencies. The system cannot find the file specified.
File name: 'xunit.runner.utility.net452, Version=2.4.1.0, Culture=neutral, PublicKeyToken=8d05b1bb7a6fdb6c'
   at System.Reflection.RuntimeAssembly._nLoad(AssemblyName fileName, String codeBase, Evidence assemblySecurity, RuntimeAssembly locationHint, StackCrawlMark& stackMark, IntPtr pPrivHostBinder, Boolean throwOnFileNotFound, Boolean forIntrospection, Boolean suppressSecurityChecks)
   at System.Reflection.RuntimeAssembly.InternalLoadAssemblyName(AssemblyName assemblyRef, Evidence assemblySecurity, RuntimeAssembly reqAssembly, StackCrawlMark& stackMark, IntPtr pPrivHostBinder, Boolean throwOnFileNotFound, Boolean forIntrospection, Boolean suppressSecurityChecks)
   at System.Reflection.Assembly.Load(AssemblyName assemblyRef)
   at Microsoft.VisualStudio.TestPlatform.Common.ExtensionFramework.TestPluginCache.CurrentDomainAssemblyResolve(Object sender, AssemblyResolveEventArgs args)
   at System.AppDomain.OnAssemblyResolveEvent(RuntimeAssembly assembly, String assemblyFullName)

WRN: Assembly binding logging is turned OFF.
To enable assembly bind failure logging, set the registry value [HKLM\Software\Microsoft\Fusion!EnableLog] (DWORD) to 1.
Note: There is some performance penalty associated with assembly bind failure logging.
To turn this feature off, remove the registry value [HKLM\Software\Microsoft\Fusion!EnableLog].

TpTrace Verbose: 0 : 3024, 3, 2022/04/22, 15:49:38.119, 5020859371399, testhost.net462.exe, TestPluginCache: Discovered the extensions using extension path ''.
TpTrace Verbose: 0 : 3024, 3, 2022/04/22, 15:49:38.123, 5020859412635, testhost.net462.exe, TestPluginCache: Discoverers are ''.
TpTrace Verbose: 0 : 3024, 3, 2022/04/22, 15:49:38.123, 5020859414534, testhost.net462.exe, TestPluginCache: Executors are ''.
TpTrace Verbose: 0 : 3024, 3, 2022/04/22, 15:49:38.124, 5020859415795, testhost.net462.exe, TestPluginCache: Executors2 are ''.
TpTrace Verbose: 0 : 3024, 3, 2022/04/22, 15:49:38.124, 5020859416662, testhost.net462.exe, TestPluginCache: Setting providers are ''.
TpTrace Verbose: 0 : 3024, 3, 2022/04/22, 15:49:38.124, 5020859417697, testhost.net462.exe, TestPluginCache: Loggers are ''.

@ViktorHofer
Copy link
Member

@nohwnd thanks for the initial investigation. Curious, who would be the best person to look into this more closely? I currently have no idea how to make progress and unblock our devs.

@nohwnd
Copy link
Member

nohwnd commented May 2, 2022

@ViktorHofer Me or someone from our team. Putting this on my work item list.

@nohwnd nohwnd self-assigned this May 2, 2022
@nohwnd
Copy link
Member

nohwnd commented May 2, 2022

@ViktorHofer The issue is that both xunit.runner.visualstudio (what TestPlatform uses to run tests), and xunit.runner.console copy xunit.runner.utility.net452.dll into the output folder. In project xunit.runner.console is newer, and ends up copying a version of the dll that causes version mismatch when loading the dll (according to fusion log).

image

image

When I compare a project that uses the same versions, I get a working test project, and I can see from binlog that the dll is copied from xunit.runner.visualstudio, and NOT from xunit.runner.console:

This works:

<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <TargetFramework>net462</TargetFramework>
    
    <Nullable>enable</Nullable>
	  <LangVersion>latest</LangVersion>
    <IsPackable>false</IsPackable>
  </PropertyGroup>

  <ItemGroup>
    <PackageReference Include="Microsoft.NET.Test.Sdk" Version="16.9.0-preview-20201201-01" />


	  <PackageReference Include="xunit.runner.console" Version="2.4.2-pre.22">
		  <PrivateAssets>all</PrivateAssets>
		  <IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
	  </PackageReference>
	  
	  <PackageReference Include="xunit.runner.visualstudio" Version="2.4.2">
		  <IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
		  <PrivateAssets>all</PrivateAssets>
	  </PackageReference>

	  <PackageReference Include="xunit" Version="2.4.2-pre.22" />

  </ItemGroup>

</Project>

image

I tried to figure out why is the broken project prefers the xunit.console runner, but I could not figure that out. Hopefully it will give you a starting point. :)

@nohwnd
Copy link
Member

nohwnd commented May 2, 2022

I can also repro the same issue in my project when I add explicit reference to:

<PackageReference Include="xunit.runner.utility" Version="2.4.2-pre.22" />

Which I see happen in few targets in runtime repo and which I originally thought is the issue, but it is not. It only has the same end result. The dll is overwritten by a version that is unexpected.

@ViktorHofer ViktorHofer added this to the 7.0.0 milestone May 2, 2022
@ViktorHofer ViktorHofer removed the untriaged New issue has not been triaged by the area owner label May 2, 2022
@ViktorHofer
Copy link
Member

@ericstj the upgrade from xunit.runner.console/2.4.1 to 2.4.2 broke xunit.runner.visualstudio which relies on xunit.runner.utility.*/2.4.1. The impact is that dotnet test and VS Test Explorer runs are broken for .NETFramework where only exact assembly version matches are resolved (without binding redirects).

@clairernovotny could we release a new version of xunit.runner.visualstudio that depends on xunit.runner.utility.*/2.4.2 or would it make more sense to revert the assembly upgrade in xunit.runner.console/2.4.2?

@clairernovotny
Copy link
Member

We can release a new version. Any chance you could submit a PR with the dependency update?

@ViktorHofer
Copy link
Member

Absolutely. Will get to it later today.

@ViktorHofer
Copy link
Member

@nohwnd nohwnd removed their assignment May 3, 2022
ViktorHofer added a commit that referenced this issue May 4, 2022
Fixes #68384

For a detailed explanation why updating the xunit.runner.visualstudio fixes .NETFramework dotnet test and VSTest invocation please see the above linked issue.
@ghost ghost added the in-pr There is an active PR which will close this issue when it is merged label May 4, 2022
@ViktorHofer
Copy link
Member

Wohooo, the final fix for this issue is out: #68863. Thanks @nohwnd and @clairernovotny for your help.

ViktorHofer added a commit that referenced this issue May 5, 2022
* Fix VSTest and dotnet test broken on .NETFramework

Fixes #68384

For a detailed explanation why updating the xunit.runner.visualstudio fixes .NETFramework dotnet test and VSTest invocation please see the above linked issue.
@ghost ghost removed the in-pr There is an active PR which will close this issue when it is merged label May 5, 2022
@dotnet dotnet locked as resolved and limited conversation to collaborators Jun 5, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants