Console coverage report uses unfiltered coverage results?

[CNBoland]

I want to exclude code decorated with GeneratedCodeAttribute. It appears to be excluded in final /TestResults output, but included in the console coverage report, even though the .runsettings file has ExcludeByAttribute configured with this attribute.

To summarize what I see, a class decorated with GeneratedCodeAttribute is included in coverage.json (unexpected), but excluded in /TestResults//coverage.cobertura.xml (expected).

Is this correct behavior?

Steps to Reproduce:

1. Using example project, VSTest/HelloWorld, duplicate Class1 renamed to Class2 and add GeneratedCodeAttribute to Class2. Contents of Class1.cs should be:

namespace ClassLibrary1
{
    public class Class1
    {
        public int Method()
        {
            return 42;
        }
    }

    [System.CodeDom.Compiler.GeneratedCode("MyTool", "1.0.0")]
    public class Class2
    {
        public int Method()
        {
            return 42;
        }
    }
}

Add ExcludeByAttribute element to runsettings.xml. File contents of runsettings.xml should be:

<?xml version="1.0" encoding="utf-8" ?>
<RunSettings>
  <DataCollectionRunSettings>
    <DataCollectors>
      <DataCollector friendlyName="XPlat code coverage">
        <Configuration>
          <Format>lcov,cobertura</Format>
          <ExcludeByAttribute>GeneratedCode</ExcludeByAttribute>
        </Configuration>
      </DataCollector>
    </DataCollectors>
  </DataCollectionRunSettings>
</RunSettings>

Execute test and code coverage from command line at solution root:
dotnet test --collect "XPlat Code Coverage" --settings .\XUnitTestProject1\runsettings.xml /p:CollectCoverage=true

Expected outcome:

+---------------+------+--------+--------+
| Module        | Line | Branch | Method |
+---------------+------+--------+--------+
| ClassLibrary1 | 100% | 100%   | 100%   |
+---------------+------+--------+--------+

Note: 100% is expected because Class2 isn't covered by any test and is expected to be excluded from coverage. This leaves only Class1, which is 100% covered by tests.

Actual outcome:

+---------------+------+--------+--------+
| Module        | Line | Branch | Method |
+---------------+------+--------+--------+
| ClassLibrary1 | 50%  | 100%   | 50%    |
+---------------+------+--------+--------+

Analysis
XUnitTestProject1/coverage.json contains both Class1 and Class2.
XUnitTestProject1/TestResults//coverage.cobertura.xml contains only Class2.

Question: The coverage report appears to be based on the unfiltered XUnitTestProject1/coverage.json rather than the filtered XUnitTestProject1/TestResults//coverage.cobertura.xml file. Shouldn't this report factor in the ExcludeByAttribute filtering as well?

[CNBoland]

I see the difference now. Exclusions in runsettings.xml are applied to /TestResults output. For the console coverage report, exclusions are applied on the command line with /p:ExcludeByAttribute="GeneratedCodeAttribute".

To be clear: Exclusions in runsettings.xml affect /TestResults output. Exclusions on the command line (e.g. /p:ExcludeByAttribute=) affect the MSBuild console coverage report. The two configurations are independent.

[CNBoland]

The complete command line command would be:

dotnet test --collect "XPlat Code Coverage" --settings .\XUnitTestProject1\runsettings.xml /p:CollectCoverage=true /p:ExcludeByAttribute="GeneratedCodeAttribute"

which produces the expected outcome;

+---------------+------+--------+--------+
| Module        | Line | Branch | Method |
+---------------+------+--------+--------+
| ClassLibrary1 | 100% | 100%   | 100%   |
+---------------+------+--------+--------+