Python coverage support broken (v0.6.10 of coveralls report problem)

[set-soft]

Yesterday the "coveralls report" 0.6.10 was released. This started to generate the following error:

🔍 Detected coverage file: .coverage
⚠️ Coverage reporter does not yet know how to process this file: .coverage
🚨 Nothing to report

Forcing the format to python I got:

✏️ Forced coverage format: python
🔍 Detected coverage file: .coverage
#<CoverageReporter::CoveragepyParser::ParserError:There was an error processing .coverage: --: 1: coverage: not found


To use the python format, do one of the following:
1. Make sure that the coverage executable is available in the
   runner environment, or
2. Convert the .coverage file to a coverage.xml file by running
   `coverage xml`. Then pass the input option `format: cobertura`
   (for Coveralls GitHub Action or orb), or pass `--format=cobertura`
   if using the coverage reporter alone.

The first option seems to be impossible for an action. So I tried the second. Now I get:

✏️ Forced coverage format: cobertura
🔍 Detected coverage file: coverage.xml
🔍 Detected coverage file: .coverage
⭐️ Running in parallel mode. You must call the webhook after all jobs finish: `coveralls done --build-number 8752691638`
#<IO::Error:Error reading file: Is a directory>

I still trying things, but I think the action should fix it.

[MarcelGeo]

Hello there. I have same issue :) This is caused by PR coverallsapp/coverage-reporter#122 , but not sure - how to fix that :)

[set-soft]

I dropped the action and I'm just using coveralls reporter 0.6.9, which works for Python. I couldn't get 0.6.10 working, I tried using the recommended XML export and also installing the Python coverage tool, always failed.

I use something like this:

    - name: Upload Coverage optional
      env:
        COVERALLS_DEBUG: true
        COVERALLS_FLAG_NAME: ${{ matrix.ki_release }}_${{ matrix.w_tests }}
        COVERALLS_PARALLEL: true
        COVERALLS_ENDPOINT: https://coveralls.io
        COVERALLS_REPO_TOKEN: ${{ secrets.GITHUB_TOKEN }}
        COVERALLS_SOURCE_HEADER: github-action
      run: |
        mkdir -p ~/bin/
        echo ~/bin >> $GITHUB_PATH
        export CURDIR=`pwd`
        cd ~/bin/
        curl -sLO https://github.com/coverallsapp/coverage-reporter/releases/download/v0.6.9/coveralls-linux.tar.gz
        curl -sLO https://github.com/coverallsapp/coverage-reporter/releases/download/v0.6.9/coveralls-checksums.txt
        cat coveralls-checksums.txt | grep coveralls-linux.tar.gz | sha256sum --check
        tar -xzf coveralls-linux.tar.gz
        rm coveralls-checksums.txt
        cd ${CURDIR}
        ~/bin/coveralls report --debug --format python .coverage

[afinetooth]

Hi @set-soft. It looks like the issue might be with your last line.

That is to say, there is an issue with your last line that will cause an error: --format python. Right now, the direct parsing of pytest-cov (ie. .coverage files is not supported). So if you want to use the XML conversion step, coverage xml, you'll want to use --format cobertura, since the XML format is Cobertura XML.

(--format python will be supported soon, but it will require having coverage.py in your environment, so we can perform the conversion step for you.)

Let me know if you have any luck with 0.6.10, after converting your .coverage file to Cobertura XMl with coverage xml and using --format cobertura.

It's great that you're using the --debug flag. If the above doesn't work, please share the debug output from your CI build log and we'll go from there.

[set-soft]

Hi @set-soft. It looks like the issue might be with your last line.

That is to say, there is an issue with your last line that will cause an error: --format python. Right now, the direct parsing of pytest-cov (ie. .coverage files is not supported). So if you want to use the XML conversion step, coverage xml, you'll want to use --format cobertura, since the XML format is Cobertura XML.

I also tried this (before python), didn't work.

(--format python will be supported soon, but it will require having coverage.py in your environment, so we can perform the conversion step for you.)

Let me know if you have any luck with 0.6.10, after converting your .coverage file to Cobertura XMl with coverage xml and using --format cobertura.

When I tried using cobertura I got

  v0.6.10

✏️ Forced coverage format: cobertura
🔍 Detected coverage file: coverage.xml
🔍 Detected coverage file: .coverage
⭐️ Running in parallel mode. You must call the webhook after all jobs finish: `coveralls done --build-number 8752691638`
#<IO::Error:Error reading file: Is a directory>

You can see the run here

I also tried removing the .coverage file here

It's great that you're using the --debug flag. If the above doesn't work, please share the debug output from your CI build log and we'll go from there.

Lamentably I didn't have debug enabled when I tried cobertura.

[afinetooth]

Hi @set-soft. I submitted this as a potential issue with 0.6.10 not ignoring files while searching for file extensions (including .coverage in the context of a directory in your environment, .coverage/).

If you're concerned about getting it to work before we can have a look, I suggest renaming that directory if feasible.

I'll update you here when I hear back.

[afinetooth]

@set-soft. Follow-up clarification from my team member:

When you said:

I also tried removing the .coverage file here

Can I confirm that you mean you removed any and all .coverage files from your environment?

Specifically, did you remove the .coverage file but leave a .coverage/ directory in your environment?

Thanks 🙏

[set-soft]

Hi @afinetooth !

Take a look at the Upload Coverage optional (requisite) step, it does rm .coverage.

So .coverage was a file, not a directory, and was removed to avoid confusion.

You can see the output of the ls command at the end of the step Collect coverage ki8 it says:

-rw-r--r-- 1 root root   77824 Apr 19 11:47 .coverage
-rw-r--r-- 1 root root     371 Apr 19 11:42 .coveragerc
-rw-r--r-- 1 root root 1071161 Apr 19 11:47 coverage.xml

This step ran:

        python3-coverage combine
        python3-coverage report
        python3-coverage html -d output/htmlcov_${{ matrix.w_tests }}
        python3-coverage xml
        ls -la coverage* .coverage*

I never knew what directory was the problem.

I tried running the same docker image in my local machine and tried to reproduce the steps, I got:

 
⠀⠀⠀⠀⠀⠀⣿
⠀⠀⠀⠀⠀⣼⣿⣧⠀⠀⠀⠀⠀⠀⠀ ⣠⣶⣾⣿⡇⢀⣴⣾⣿⣷⣆ ⣿⣿⠀⣰⣿⡟⢸⣿⣿⣿⡇ ⣿⣿⣿⣷⣦⠀⠀⢠⣿⣿⣿⠀⠀⣿⣿⠁⠀⣼⣿⡇⠀⢀⣴⣾⣿⡷
⠶⣶⣶⣶⣾⣿⣿⣿⣷⣶⣶⣶⠶  ⣸⣿⡟ ⠀⢠⣿⣿⠃⠈⣿⣿⠀⣿⣿⢠⣿⡿⠀⣿⣿⣧⣤⠀⢸⣿⡇⣠⣿⡿⠀⢠⣿⡟⣿⣿⠀⢸⣿⡿⠀⠀⣿⣿⠃⠀⢸⣿⣧⣄
⠀⠀⠙⢻⣿⣿⣿⣿⣿⡟⠋⠁⠀⠀ ⣿⣿⡇⠀ ⢸⣿⣿⠀⣸⣿⡟⠀⣿⣿⣾⡿⠁ ⣿⣿⠛⠛⠀⣿⣿⢿⣿⣏⠀⢀⣿⣿⣁⣿⣿⠀⣾⣿⡇⠀⢸⣿⡿⠀⠀⡀⠙⣿⣿⡆
⠀⠀⢠⣿⣿⣿⠿⣿⣿⣿⡄⠀⠀⠀ ⠙⢿⣿⣿⠇⠈⠿⣿⣿⡿⠋⠀⠀⢿⣿⡿⠁⠀⢸⣿⣿⣿⡇⢸⣿⣿⠀⣿⣿⣄⣾⣿⠛⠛⣿⣿⢠⣿⣿⣿ ⣼⣿⣿⣿ ⣿⣿⡿⠋⠀
⠀⢀⣾⠟⠋⠀⠀⠀⠙⠻⣷⡀⠀⠀
 
  v0.6.10

✏️ Forced coverage format: cobertura
🔍 Detected coverage file: coverage.xml
⏱️ Report parsing: 00:00:00.355358404
⏱️ Report request: 00:00:00.005574598
Unhandled exception: Error reading file: Is a directory (IO::Error)
  from ???
  from ???
  from ???
  from ???
  from ???
  from ???
  from ???
  from ???
  from ???

I used coveralls report --format cobertura -r pp --debug

[altaurog]

I just encountered this. I was able to work around it by specifying --cov-report=lcov to pytest.

using lcov format also produced correct results, whereas with the default .coverage output I had observed a discrepancy in the reported coverage which seemed to stem from improper parsing of the python files.