Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Coverage results are incorrect. #18

Closed
StillLearnin opened this issue Mar 26, 2018 · 11 comments
Closed

Coverage results are incorrect. #18

StillLearnin opened this issue Mar 26, 2018 · 11 comments

Comments

@StillLearnin
Copy link

I have 3 projects and 21 tests all written against Candle.Services

Candle.DbContext
   |
   +--(References) Candle.Services
   |    |
   +----+--(References) Candle.Services.Tests

Calculating coverage result...
  Generating report 'C:\..\Candle.Services.Tests\coverage.json'

+------------------+----------+
| Module           | Coverage |
+------------------+----------+
| Candle.DbContext | 21%      |
+------------------+----------+
| Candle.Services  | 0%       |
+------------------+----------+
@tonerdo
Copy link
Collaborator

tonerdo commented Mar 26, 2018

@StillLearnin take a look at the actual coverage results file. This could be a bug in just the summary generation

@StillLearnin
Copy link
Author

Here's a sample:

For what it's worth, the Get() method is in a base class.

Actual code being tested:

image

coverage.json relevant portion:

   "Candle.Services.SubjectsService/<ListAsync>d__3": {
    "System.Void Candle.Services.SubjectsService/<ListAsync>d__3::MoveNext()": {
     "33": 0,
     "34": 0,
     "35": 0,
     "36": 0,
     "37": 0
    }
   }

Test with ListAsync() highlighted:

image

@StillLearnin
Copy link
Author

See #7 also. I started this new issue because I thought #7 might have been fixed with the previous pull requests or might be a different issue.

@tonerdo
Copy link
Collaborator

tonerdo commented Mar 26, 2018

This doesn't look like much of a #7 issue. I'm thinking it might have something to do with all async methods. Although, one of Coverlet's users did mention that it works well with async tests. Is this project open source?

@StillLearnin
Copy link
Author

Not open source but I trimmed the project down to about 1/10th its size and can still reproduce the issue. You can get the trimmed down version here

@StillLearnin
Copy link
Author

I think it may be an issue with timing or memory usage because if you comment out line 16 in Main so as to not seed postal codes, coverlet seems to work.

The other option that fixes the issue is to trim down PostalCodesUSA.csv until the total number of lines doesn't exceed ~2000. It doesn't matter which lines you take out, just that the total number of lines is sufficiently small.

@tonerdo
Copy link
Collaborator

tonerdo commented Mar 26, 2018

Thanks for providing the repro project. I'll clone it and check things out

@StillLearnin
Copy link
Author

If you could give me some instructions on how to debug this type of project I'd be glad to tackle it.

@tonerdo
Copy link
Collaborator

tonerdo commented Apr 2, 2018

@StillLearnin I figured out the problem. The CoverageTracker class writes to the hits file when the ProcessExit event is thrown, however, because there's so many hits to write it's unable to finish within the ~200ms window. There are a lot less hits when you comment out that line that's why it's able to finish successfully.

@tonerdo
Copy link
Collaborator

tonerdo commented Apr 2, 2018

I'm gonna look for a more efficient way to write to the hits result file

@StillLearnin
Copy link
Author

I'd be glad to take a look at this and open a pull request

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants