Zero 0% coverage reported

[lukka]

Coverall works fine with all repos, but not with this one:

https://coveralls.io/github/lukka/run-vcpkg

Any suggestion is welcome, thanks!

[afinetooth]

Hi @lukka.

The state of your build pages looked strange to me so I re-ran all the jobs in order again, to allow background jobs to re-run, and, among other things, re-calculate coverage % from your constituent coverage reports.

Things look normal now:
https://coveralls.io/github/lukka/run-vcpkg

But I'm afraid I can't account for why this didn't occur initially. I assume one or more of your jobs got held up in queue, or failed to run and were awaiting a retry, but I can't determine a why. For now, I'll have to attribute it to a one-off.

Closing for now, but let me know if it happens again and we can re-open this issue.

In that case, it will be helpful to run a build in verbose mode, which can be enabled for the Coveralls Github Action by adding the following CI env var to your GHA config yaml so it's available to the coveralls step(s):
NODE_COVERALLS_DEBUG=1

The NODE_ part is there because the Coveralls Github Action uses the node-coveralls integration under-the-hood.

[WonderPanda]

@afinetooth I'm also seeing a ton of builds randomly showing 0% in the final coverage report even though I can clearly see individual jobs from the parallel run reporting coverage.

The individual jobs in the parallel run also don't seem to reliably report checks back either. Sometimes I see them for one job, sometimes both and sometimes none at all.

[afinetooth]

Hi @WonderPanda I'm re-opening this.

Are you on the same repo as @lukka?

If not, please share your Coveralls repo URL and I'll have a look.

[WonderPanda]

Hey @afinetooth thanks for reopening. Its a private github repo but our URL is https://github.com/circadian-risk/project-ember-web

[lukka]

@afinetooth it is now indeed working, without any change on run-vcpkg repository.

For helping tracking down the issue, I enabled in this PR two different changes:

• using parallels with unique flag name;
• enabling verbose mode by setting environment variable NODE_COVERALLS_DEBUG to 1.

If the problem is reoccurring, at least we have more debugging info to look at.

[afinetooth]

@WonderPanda since most of the builds described that way, now, are from today, I assume it relates to the performance issues we're having today, described here:
https://status.coveralls.io/

It's unusual in your case that some of your builds appear complete but with 0 files processed (and thus 0% coverage), so I'm thinking those (background) jobs may have failed after a server timeout or similar. I will try to re-run those as soon as we've addressed the current issues, also described here.

[afinetooth]

@lukka great.

If you're experiencing problems similar to @WonderPanda they are probably related to current issues described on our status page.

But if it occurs again, will be very happy to examine your verbose build logs for clues.

[runejac]

@afinetooth Hi! I am a student and new to this, also experiencing the same thing as OP. I wondered if you could have a look here at my repo? Also, where do I put NODE_COVERALLS_DEBUG=1? I have no idea what verbose mode is. Do I put this on some line in my workflow.yml file? Thanks in advance.

Edit:

I managed to use the env variable. Does this point out on anything that I can fix the zero-coverage displayed at Coveralls.io? Using shrib to post the debug text here: https://shrib.com/#Lillie9nwvkV0

Edit2:

Ok my problem was that it got fixed when I moved /coverage to root dir, even though I pointed the path-to-lcov to the subdir. Now when coverage dir is on root Coveralls reads it!

[afinetooth]

Hi @runejac. I'm glad you discovered more about verbose mode and using the NODE_COVERALLS_DEBUG=1 environment variable for the Coveralls Github Action, which uses the node-coveralls integration under-the-hood. (Thus, the NODE_ part.)

Your verbose build log ultimately represents a successful POST to the Coveralls API...

[debug] "2022-02-11T14:57:19.519Z"  200
[debug] "2022-02-11T14:57:19.519Z"  '{"message":"Job #1829939948.1","url":"https://coveralls.io/jobs/94376409"}'

The problem seems to have occurred somewhere between reading in your LCOV file:

Using lcov file: client/coverage/[lcov.info](http://lcov.info/)
[debug] "2022-02-11T14:57:18.747Z"  'TN:\n' +
  'sf:Quiz.tsx\n' +
  'FN:21,FrontPage\n' +
  'FN:37,MapQuestions\n' +
  'FN:42,checkAnswer\n' +
  'FN:43,(anonymous_3)\n' +
  'FN:45,(anonymous_4)\n' +
  'FN:57,(anonymous_5)\n' +
  'FN:63,(anonymous_6)\n' +
  'FN:85,(anonymous_7)\n' +
  'FN:114,(anonymous_8)\n' +
  'FNF:9\n' +
  'FNH:7\n' +
  'FNDA:2,FrontPage\n' +
  'FNDA:3,MapQuestions\n' +
  'FNDA:2,checkAnswer\n' +
  'FNDA:0,(anonymous_3)\n' +
  'FNDA:0,(anonymous_4)\n' +
  'FNDA:10,(anonymous_5)\n' +
  'FNDA:2,(anonymous_6)\n' +
  'FNDA:2,(anonymous_7)\n' +
  'FNDA:1,(anonymous_8)\n' +
  'DA:12,1\n' +
  'DA:22,2\n' +
  'DA:38,3\n' +
  'DA:39,3\n' +
  'DA:40,3\n' +
  'DA:43,2\n' +
  'DA:44,2\n' +
  'DA:45,1\n' +
  'DA:46,1\n' +
  'DA:48,1\n' +
  'DA:52,3\n' +
  'DA:58,10\n' +
  'DA:63,2\n' +
  'DA:85,1\n' +
  'DA:86,2\n' +
  'DA:114,1\n' +
  'DA:115,1\n' +
  'DA:116,1\n' +
  'DA:118,1\n' +
  'LF:19\n' +
  'LH:19\n' +
  'BRDA:44,0,0,1\n' +
  'BRDA:44,0,1,1\n' +
  'BRDA:58,1,0,2\n' +
  'BRDA:58,1,1,8\n' +
  'BRF:4\n' +
  'BRH:4\n' +
  'end_of_record\n' +
  'TN:\n' +
  'sf:questions.ts\n' +
  'FN:19,randomQuestion\n' +
  'FN:23,isCorrectAnswer\n' +
  'FNF:2\n' +
  'FNH:2\n' +
  'FNDA:1,randomQuestion\n' +
  'FNDA:2,isCorrectAnswer\n' +
  'DA:20,1\n' +
  'DA:24,2\n' +
  'DA:29,1\n' +
  'LF:3\n' +
  'LH:3\n' +
  'BRF:0\n' +
  'BRH:0\n' +
  'end_of_record\n'

And parsing and converting the LCOV data into JSON, which results in this JSON:

[info] "2022-02-11T14:57:18.784Z"  'sending this to [coveralls.io](http://coveralls.io/): ' '{"source_files":[],"git":{"head":{"id":"ee1582ec5203cf9abaf92ccc514480e279ed127d","committer_name":"runejac","committer_email":"[69790920+runejac@users.noreply.github.com](mailto:69790920+runejac@users.noreply.github.com)","message":"pushpushpush","author_name":"runejac","author_email":"[69790920+runejac@users.noreply.github.com](mailto:69790920+runejac@users.noreply.github.com)"},"branch":"refs/heads/main","remotes":[{"name":"origin","url":"https://github.com/runejac/web-api-own-account-innlevering-runejac"}]},"run_at":"2022-02-11T14:57:18.749Z","service_name":"github","service_job_id":"1829939948","repo_token":"***"}'

Note that the source_files attribute is empty there:

"source_files":[]

Right now, I'm at a loss for why that happened, but am tracking the possibility of a breaking change in a dependency that's causing that parsing to fail silently.

Will feedback asap.

[runejac]

Hi and thanks for your answer. I fixed this by moving the test dir to root. When I had client/tests and therefore coverage dir inside client dir, this would happen. When I moved tests outside to root it would make a coverage dir on root aswell and then it was OK, Coveralls got the build. This was even when i redirected the "path-to-lcov" to the subdir folder. I would guess Coveralls work with subdir? Any way to fix this if I want to have tests inside a sub dir and also then coverage dir inside a sub dir?

[afinetooth]

Hi @runejac, I see, great. Thanks for the update.

Yes, if I understand correctly, you want to correct for the relative paths in your coverage report if you move your tests files into a subdirectory? If so, then you can use the base-path parameter as described in these usage instructions.

[runejac]

Hi @runejac, I see, great. Thanks for the update.

Yes, if I understand correctly, you want to correct for the relative paths in your coverage report if you move your tests files into a subdirectory? If so, then you can use the base-path parameter as described in these usage instructions.

All right, so if I have client/__tests__ and therefore coverage/ sub dir in there, I can use base-path: client/coverage? Do I also need path-to-lcov: client/coverage/lcov.info?

[StianOek]

^ I also wondering about the same thing @runejac asking :)

[afinetooth]

@runejac, @StianOek, yes, those assumptions sound correct. Except for the fact that I'm not sure what you mean by client/__tests__.

Let's use this scenario:

Your coverage file lives in the default location, ./coverage/lcov.info, and your source files live in a subdirectory called client/.

(Note that in, this case, you don't need to pass a value for path-to-lcov since you're using the correct default location; but we will pass it anyway for clarity.)

So the relevant portion of your GHA config yaml would look like this:

path-to-lcov: ./coverage/lcov.info
base-path: ./client/

Such that, when your lcov.info lists files like this:

'SF:Quiz.tsx\n' +

Those will be corrected to:

'SF:./client/Quiz.tsx\n' +

The point of base-path is to help the integration find your actual source files if the file paths listed in your LCOV file won't point it to the right place.

Hope that helps.

[StianOek]

Really nice! :) Thank you very much. It all works fine now @afinetooth

[afinetooth]

@StianOek great. Glad to hear it.

[runejac]

@afinetooth Hi again. What if I have two dirs that is containing tests? client/__tests__ and also server/__tests__, where it makes two different coverage folders inside those two subdirs. How can I combine two different lcov.info files?

[afinetooth]

@runejac so as regards your files in __tests__ (aka. test files and coverage reports)—as opposed to source code files—you're fine as long as you include the full path for each coverage report (each lcov.info) when you pass it to the path-to-lcov parameter.

But since you can only pass one value for path-to-lcov at-a-time ("one per job"), if you have two separate coverage files in your build, you have two options:

1. Split the build into separate, parallel jobs - Probably the easiest approach. You'll just need to edit your GHA config.yaml to run your build as a matrix build, running one set of tests and passing one lcov.info per job. You'll also need to modify your config for the Coveralls Github Action to handle a parallel job, so see the parallel build config example from its README. (Also, here's a sample project demonstrating how to configure parallel builds.)
2. Merge coverage reports on the CI side - I'm not sure how to accomplish this exactly given your use case, but many coverage libraries provide features and instructions on how to merge coverage files. For instance, the popular Ruby coverage library, simplecov, does it automatically in the same execution environment, or by configuration across execution environments. Here are a couple of resolved issues I found when searching for similar capabilities with jest, nyc and istanbul: Jest: How to merge coverage reports from different jest test runs | Combining coverage from 2 test runs?

Hope that helps.

[afinetooth]

Hi guys, I'm going to close this for now since there have been no updates since Mar. (It's Sep now.)