0% coverage is being shown after running tests #110
Comments
|
Hi @robvanpamel can you try with the previous version of Coverlet (v1.2.0)? This seems to be a 2.0.0 issue. Let me know |
|
Hi @tonerdo, Using macOS 10.13.4, both 2.0.0 and 1.2.0 work as expected. Linux dotnet cofiguration macOS dotnet configuration |
|
@tonerdo, Linux dotnet 2.0.7 configuration Linux dotnet configuration 2.1-rc1 FYI @robvanpamel |
|
Hi, |
|
@robvanpamel Can you try to exclude NUnit3 Test Adapter assemblies: |
|
I'm facing a similar problem.
|
|
I have the issue of 0% coverage even on coverlet version 1.2.0 Note: It works on other netstandard/netcore projects i have, running same framework versions etc. Update 2:
Update 3:
Update 4: Bumped coverlet to 2.0.0 and i still get coverage results :-D There is a discrepancy in the results from coverlet 1.2 to 2.0.
Coverlet 2.0
|
|
I'm seeing this as well in one of my test suites. I moved to using the nuget package, just to eliminate potential variables: I know that coverage should actually be around 60-70% - certainly not 0. Interestingly, it generates an enormous coverage.json file full of lines, but all the Is there a diagnostic logger for coverlet that I can enable? |
|
Hi @queen-of-code @robvanpamel @tlogik take a look at #72. Try adding the following line to your csproj: <CopyLocalLockFileAssemblies>true</CopyLocalLockFileAssemblies>
Not at this time |
|
Unfortunately no luck. I added that, and also tried switching my test csproj from Aside from making my test run exponentially slower, it didn't provide any coverage information. Just a dump full of 0% coverage-claims. |
|
#181 @queen-of-code completely changes the way Coverlet instruments assemblies and makes the test run orders of magnitude faster. You can see perf improvements here: #181 (comment). I expect this change should fix the 0% coverage issue you're all getting, should make a release in 48 hours so you can test it |
|
Looking forward to it @tonerdo - when you cut the new release, I will test it out ASAP. |
|
@queen-of-code The new releases are out Coverlet Global Tool: https://www.nuget.org/packages/coverlet.console/1.2.0 Let me know if your issue gets fixed when you upgrade. |
|
@queen-of-code I didn't notice this before but you should try changing |
|
I have good news and I have bad news. The good news is that our total execution time is down from ~50 minutes to ~22 minutes. That's an amazing improvement! Great work! The bad news is that I'm still getting no coverage results. I've included the problematic test csproj. The command I'm running is |
|
@queen-of-code I'm bummed that you're still having issues. Your csproj looks accurate enough and you shouldn't still be having problems. Unfortunately, I'm unable to debug without having access to the problematic project. As a result of this limitation, I'm adding a new flag to log diagnostics information, making it easy for me to debug similar issues without needing a repro project. This will be ready/released in ~2 days
Glad to hear that. Out of curiousity, how long do your tests take to run when Coverlet isn't included? |
|
Our current test suite (using dotnet test test.sln) runs in just over 1 minute. I just verified test-only numbers with the latest coverlet version, and it's at about 8.5 minutes. So it's still a significant overhead, most of which I suspect is due to running tests serially via **/*.csproj. SonarQube is involved in the build process as well, so it's certainly possible that it's leaving cruft in the dotnet assemblies. |
|
I having what seems to be a similar issue of 0% results for code coverage using 2.3.0, however I do get coverage for 2 of the 4 projects with the .sln I'm trying to get coverage for. I do get results using 1.2.0. Happening on both centos7 and osx. No coverage: Coverage: |
|
Hello everyone on the thread. This error message appears. Now all I did was a search on the bugs and found this post.. I notice you are referencing Is there anybody that can confirm that Coverlet works with Moq? |
|
I'm also getting 0% code coverage on one of my projects. You can view the source here: https://github.com/StephenMP/SLOBSharp/tree/cc9bfd373730f9fa46454bf9cca65039edae4651 Also @ppumkin not sure if this is the correct place for your question, but yes, it works with Moq :) Update: I use NCrunch to run my tests locally and it has no issue calculating metrics in this scenario, so not sure why Coverlet struggles? Update 2: |
|
That is interesting. Coverlet needs to rewrite parts of the DLL and I have noticed if it fails to do so it wont crash and burn it will just return 0% coverage. (I need to double check this but that is what I remember happening when I was messing around with my extension using coverlet - Probably need a message to say that the DLL's was not attached instead of returning 0%) I wonder if there is a race condition or just a security constraint on rewriting the DLL nCrunch uses a different technique to monitor for coverage. |
|
I can confirm that I have at least one test project using Moq that works fine, but it's true that the one that doesn't work is using Moq. |
|
I also get 0% test coverage on my project at work, also have Moq installed. |
|
Facing the same issue. Got a feeling it is something on the lines of caching but cant put my figure to the pattern followed. I tried running the test with different options and sometimes it works. But this is not consistent |
|
I can confirm same issue. The coverage have been 0% after we added a dependency to Reactive Extensions (Rx) for .NET. |
|
For me coverage doesn't work unless I add references to:
And remove |
|
Same here in corefx. Spinning up a Ubuntu machine now to debug what's going on. |
|
Ok sorry, it is working for me, I was missing passing the include-directory with the right formatted path to coverlet. |
Based on this comment I tried running the console in Admin mode and for atleast one time it gave a accurate coverage. Not sure this would be reproducible though. |
|
Continuing to get more info on the 0% coverage, I'm running coverlet.console on the debugger, with references to the coverlet.core project. I added traces to ModuleTrackerTemplate class and I can see calls to RecordHit and Registering the Unload Events. However, the UnloadModule method doesn't seem to be called for the tests for which I get 0% coverage. The only other thing I see is a lonely message in the Application Output window that reads:
without any additional context. Does this message rings a bell to anybody as to whether or not may be related to an exception that would prevent the UnloadModule from running? |
@vagisha-nidhi can you confirm that last release of sdk 2.2.401(runtime 2.2.6) inject inproc data collector? |
|
I can confirm that at least some of my issues were FIXED in this combo of
releases. Thank you all for your effort here!
…On Fri, Sep 6, 2019 at 6:48 AM Marco Rossignoli ***@***.***> wrote:
I have released vstest with this, but this will flow into dotnet sdk 2.2.4
only.
@vagisha-nidhi <https://github.com/vagisha-nidhi> can you confirm that
last release of sdk 2.2.401(runtime 2.2.6) inject inproc data collector?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#110>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AEIOSMJYPTER34BZUKBBH5DQIJNSNANCNFSM4FCAAE2A>
.
|
Glad to hear thank's @queen-of-code for info. I'll update docs to reflect this. |
|
I am having the same problem. |
@AmirSasson you're using msbuild integration...you should try collectors to understand if this is the correct issue you're experiencing https://github.com/tonerdo/coverlet/blob/master/Documentation/VSTestIntegration.md |
|
hi @MarcoRossignoli , i tried the vstest integration using the collectors. does it matter if use xunit and not "mstest"?(btw i was using msbuild integration cause it has "out of the box" minimum threshold that i have enforced in our CI). |
|
@AmirSasson you mean |
|
hi @MarcoRossignoli GOOD NEWS I have managed to solve this.apparently i had a running thread that probably was still running when the code coverage was calculated. |
|
@MarcoRossignoli I just finished configuring a GH Actions workflow. All is well except my 100% coverage solution is reported as 0% running under GH Actions. The same project running on Travis (xenial) and local (Windows) is reported as 100%. I've tried 2.2.108, 204, 401, 402 - along with 2.6.3. I tried all the workarounds listed on this thread, no luck. If you're aware of any other fixes I may try please let me know. |
|
@Rjae you should try with collectors and version v2.2.401 or newer sdk, the issue could be related to vs test platform that kill process before hit files are written and could be dependant on ci machine load. |
|
Thanks @MarcoRossignoli I'm giving collectors a shot right now. Getting $ dotnet test --settings <absolute path to runsettings>
Data collection : Could not find data collector 'XPlat code coverage'My runsettings file: I've followed the doc - but must be missing something. I'll keep digging and update thread here. |
|
Force rebuild was necessary, so locally all good. Will use in GH workflow now and update thread with results. |
|
Sadly, switching to collector did not resolve issue. Again, works locally (both collector and msbuild), on Travis, and on Jenkins. |
|
@Rjae can you try to enable coverlet logging? msbuild collectors |
|
I'm going to close this issue as resolved by in-proc collectors. |
|
@MarcoRossignoli I finally had time to enable logging (see attached). Not knowing coverlet code yet, I mostly skimmed through. I did notice many socket exceptions in host log. Please let me know if there is something else I may try. |
|
@Rjae can you open new issue(I'm oof at the moment) with copy paste your messages from this thread so we can go on there on clean one. |
|
Just curious, is this even necessary? |
|
Needed to avoid that msbuild harness package coverlet.msbuild(also props,targets) flow to project that uses it https://docs.microsoft.com/en-us/nuget/consume-packages/package-references-in-project-files#controlling-dependency-assets |
and others
Hi,
After running my tests with Nunit I get an unexpected result from coverlet.
My folder structure is
I executed the tests from within the testprojectfolder
The result is printed below.
I'm running on a Linux Mint machine with this dotnet configuration
The text was updated successfully, but these errors were encountered: