Support for Source Generator

[latop2604]

Hello,

I use altcover (global tool) to generate code coverage for an .Net5.0 app. And use the result as open cover format to generate report with Report-Generator.

The line coverage is ok. But report generator is unable to find related file. (Understandable as source generated file are in memory only by default.

I don't know if it's in altcover scope or in report generator. Or both.

And btw, thx for your great work on this tool.

[latop2604]

Same issue open in Report generator repo here danielpalme/ReportGenerator#457

[SteveGilham]

Any .net coverage tool relies on the debug information that is present in the .pdb file to tie the compiled IL to the original source. In the case of source generators -- having just now built and played with the official sample solution -- it seems that the compilation process invents file names under a notional folder $(ProjectDir)\[Generator Assembly Name]\, and presents them in the debug information as if they were real.

There's no easy labelling on the code that results -- no [GeneratedCode] or similar attributes, so there's no quick fix work-around I can offer. It looks like I need to extend the --localSource flag to skip parts of an assembly, on a method by method or even line by line basis, rather than just testing the first line of code it sees, now that it turns out that the possibility exists of such phantom files.

There are also some interesting bits in the debug metadata that will bear further investigation -- embedded source blocks, which may actually represent the generated code, for example. Such things would make sense combined with the rest of the debug information as enabling step-into debugging, after all, and might be of use for reporting purposes.

[latop2604]

When adding the property true in the csproj file. The generated file is available but in Obj directory. Maybe some path lookup fallback can be an option ?

[SteveGilham]

You forgot to obfuscate your file names here ☺️

That would help ReportGenerator turn the current coverage information into a proper listing with red/green coverage indication -- that program would need to know per assembly the relative path from $(ProjectDir) to obj\...\generated; the choice of target platform, and the possibility of customization in the project of $(IntermediateOutputPath) means it can't be automated. If the .pdb file does indeed contain the generated code embedded source, then it could be given the location of the (possibly embedded) .pdb files instead.

The same issues would also apply to the AltCover Visualizer, of course. That would be the opposite tack to simply suppressing the non-existent auto-generated code from coverage.

I think this might keep me busy through the .net 6 release and resultant tooling upgrades I'd been planning to absorb.

[latop2604]

I reconsidered my security needs. And was not that important to take me the time needed to hide it 😛

For nown I will patch myself the opencover file to use the generated file in obj as suggested in ReportGenerator issue.

If you find a way handle this case directly in altcover, I'll be happy to test it.

[latop2604]

To stay sync with ReportGenerator issue, we got a workaround :

Only a chunk of the path was missing in the opencover file and luckily it does not depend on the project the generator is installed. So this bash command for all project at once. It's still a hack but not so much. We can live with it.

- sed -i "s/Marketplace.Common.Generators\//obj\/Debug\/net5.0\/generated\/Marketplace.Common.Generators\//g" *.opencover.xml

I hope it could help other people.

[SteveGilham]

Post-processing the output like this in a build process that knows the context is much simpler than trying to infer whether and what substitution to make in a general purpose coverage tool. It's a pity that the compiled-in path read from the .pdb is entirely fictitious, when there's a real, however optional and possibly transient, location that could have been used instead.

There's actually a Roslyn issue here, if you want to go about addressing the root cause.

[SteveGilham]

New release v8.2.825 addresses one immediate issue related to source generators -- the --localSource flag intended to exclude third-party assemblies en masse works by testing whether the first source document referred to by an assembly's debug information exists locally; for source generator code, depending on the how the source is organized, this could potentially lead to locally built assemblies also being excluded. Now the check is made to see whether any referenced source file exists.

[SteveGilham]

What I am implementing now is this

• When detecting that a document is associated with embedded source in the debug information (true by default for the Source Generator files), adding that information to the appropriate coverage record.
• In the AltCover Visualizer, noting that such an embedded code record exists, reading the document from that, instead of looking to the file system either at the defined path or some educated guess at some transformation of that path. Other downstream consumers of reports from AltCover would also be able to avail themselves of this information.

Currently this is in proof-of-concept state for (extended-)OpenCover format only.

[SteveGilham]

This is now feature complete and will be in the next release of AltCover.

[SteveGilham]

Released as v8.2.828.