New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Show summary tables for group nodes in TestAdapter #2502
Comments
I have spoken with one of the devs on the VSTest team about this and they have confirmed to me that this feature is not possible to complete with the current VSTest protocol, I've started a discussion with them to see what it might take to get this added to the VSTest protocol but no promises at this time. |
@caaavik-msft thanks for the update, much appreciate it. I guess, we can support it not on the vstest side, but on the IDE side. At least, I can try to do it in Rider/R#. =) It shouldn't be so hard to parse the results of individual benchmarks and recombine them to a summary table for a grouping node. However, I'm not sure that it's the right way of supporting it. While it's nice to have the out-of-the-box unit test adapter, it would be better to develop a separate Rider/R# plugin that directly calls BenchmarkDotNet without vstest and properly process the raw results. This would allow to natively support more interesting features. This task has existed in my backlog for a long time, and I hope to solve it one day, but no ETA for now. |
Actually I expose a custom VSTest property exactly for this purpose: https://github.com/dotnet/BenchmarkDotNet/blob/master/src/BenchmarkDotNet.TestAdapter/VSTestProperties.cs#L15 This test property is attached as a custom property onto every testcase. It is definitely possible to build custom behaviour on top of the test adapter that uses this property. The value of this property is a |
At the moment, the TestAdapter shows statistics for independent benchmark runs (leave nodes of the tree), but doesn't show anything for grouping nodes.
It would be nice to automatically show summary tables for the given grouping nodes that define a subset of running benchmarks.
The text was updated successfully, but these errors were encountered: