Tests not executed if Console.WriteLine() is used

[avendel]

If the code path executed during a test contains a Console.WriteLine() without text content, the test will not execute. The following error is reported in the test output:

System.ArgumentException: The parameter cannot be null or empty.
Parameter name: message
   at Microsoft.VisualStudio.TestPlatform.Common.Logging.TestSessionMessageLogger.SendMessage(TestMessageLevel testMessageLevel, String message)
   at NUnit.VisualStudio.TestAdapter.NUnitEventListener.TestFinished(INUnitTestEventTestCase resultNode) in D:\repos\NUnit\nunit3-vs-adapter\src\NUnitTestAdapter\NUnitEventListener.cs:line 164
   at NUnit.VisualStudio.TestAdapter.NUnitEventListener.OnTestEvent(String report) in D:\repos\NUnit\nunit3-vs-adapter\src\NUnitTestAdapter\NUnitEventListener.cs:line 92

Here is code that reproduces this problem:

[TestFixture]
public class Tests {
    [Test]
    public void DoesNotWorkIn4alpha1() {
        Console.WriteLine();
        Assert.Pass();
    }

    [Test]
    public void Works() {
        Console.WriteLine("content");
        Assert.Pass();
    }
}

NUnit3TestAdapter 4.0.0-alpha.1
NUnit 3.12.0
Visual Studio Professional 2019 - 16.6.2
.Net Framework 4.8

This worked in NUnit3TestAdapter 3.15.1.

[OsirisTerje]

Thanks for reporting! I would assume this will also fail in 3.17, and is related to the console outputting introduced there.

[avendel]

Yes, this also fails in 3.17, but not in 3.16.1. I should also clarify; tests do get executed, but their results aren't reported, which could lead to a developer missing a failing test.

[avendel]

This seems to have the same root cause as in #220 and #516:

        public void SendMessage(TestMessageLevel testMessageLevel, string message)
        {
            if (string.IsNullOrWhiteSpace(message))
            {
                throw new ArgumentException(ObjectModelCommonResources.CannotBeNullOrEmpty, "message");
            }

https://github.com/Microsoft/vstest/blob/master/src/Microsoft.TestPlatform.Common/Logging/TestSessionMessageLogger.cs#L65

[nico008]

I just faced the same problem, good to hear you are aware of this.

[OsirisTerje]

Until fixed, the workaround is to set the ConsoleOut property in the runsettings to 0.

[MisinformedDNA]

Looks like there was never a 3.18 release. Is that still planned?

[OsirisTerje]

It is still planned, but the way forward is the 4.0.0. We expect 4.0.0 to not support legacy csproj with old package.configs, and it will not have a vsix either. 3.18 will try to cover as much as it can of the 4.0.0 features, but will still support the old legacy and will have a vsix. The release will follow the 4.0.0 release though.

[OsirisTerje]

@MisinformedDNA Just to clarify the current state. 3.18 will not be released, 3.17 will be the latest in the 3.X series. The 4.X series do support legacy csproj with old packages.config, and will continue to do so as long as we are able to maintain it. The 4.X series will not have any VSIX.