Coverage statistics

[mjvankampen]

Would it be possible to highlight coverage changes per file or something along those lines? I currently spend quite a bit of time hunting for the missing coverage. I currently just open the coverage pages of both builds and start scrolling to compare, quite tedious. Would be great to have this in BQC, similar to warnings statistics!

[ReneSchumacher]

Hi @mjvankampen,

thank you for your suggestion! Currently, we don't support anything like coverage statistics or more detailed coverage information because the BQC task does not parse the coverage data directly. It simply reads the coverage summary information from Azure DevOps, which only contains aggregated values. For simple policy evaluation the overall coverage values are sufficient and the current solution frees us from having to understand several different coverage formats.

I'll put the idea on our backlog and see what we can do. Since we're thinking about porting BQC to GitHub Actions, we might need to do the parsing in the future anyway.

René

[mjvankampen]

Fair enough, I'll try tools like https://diff-cover.readthedocs.io/en/latest/README.html and cross my fingers for https://docs.microsoft.com/en-us/azure/devops/pipelines/test/codecoverage-for-pullrequests?view=azure-devops to implement cobertura

[ReneSchumacher]

It looks like you're using Cobertura for reporting coverage, right? This might be a good first start since the format of the Cobertura report is quite easy to understand, parse, and compare. I'm not sure how quickly we can add something but we'll make sure to start with Cobertura coverage ;-) Our own (i.e., Microsoft) coverage format is binary, thus, it'll be somewhat harder to work with that.

[stan-sz]

@mjvankampen CC for PRs now supports the binary format, if that helps: microsoft/vstest#981 (comment)