Listing tests broken with terminal logger

[akoeplinger]

Description

Using .NET 9 Preview 1 SDK which enabled the terminal logger in VSTest it's no longer possible to list tests from an assembly

Steps to reproduce

$ dotnet new mstest
$ dotnet test --list-tests

Expected behavior

List the tests in the assembly.

Actual behavior

$ dotnet test --list-tests
Restore complete (0,2s)
You are using a preview version of .NET. See: https://aka.ms/dotnet-support-policy
  vstest succeeded (4,7s) → bin/Debug/net9.0/vstest.dll

Build succeeded in 5,6s

A workaround is to disable the terminal logger in VSTest by passing -p:VsTestUseMSBuildOutput=false

Diagnostic logs

Environment

macOS 14, dotnet SDK 9.0 preview1

/cc @nohwnd

[nohwnd]

This is an issue that is common to all terminal logger enabled workloads I think. @baronfel I think there is issue on msbuild about not outputting the console, we would see the same here for logs, or for additional xunit output that it writes to console. I cannot find the issue, do you know which one I mean, please?

[Evangelink]

Ping @baronfel

[baronfel]

Thanks for the ping - discussing with the team now. I think the issue being referenced is dotnet/msbuild#9608.

[baronfel]

So there's not a great answer here for a few reasons. This stopped working because in .NET 8 the test infrastructure was writing directly to MSBuild's stdout stream, but now Test logs messages. TL doesn't show messages by default, but we recently added a way to get them to show up, with the -tlp:verbosity=normal flag. I'd suggest that the test command in the SDK start adding this parameter - it's safe to add even in cases when terminal logger isn't enabled.

However, when that is done the test infrastructure would need to start logging the test discovery messages with High Importance instead of Low Importance - TL will never render any messages that are not High.

Longer term we want a better message for this purpose, but we don't have a design for that.