Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug]: Generating a coverage report with --runInBand, collectCoverageFrom, and ts-jest can mask a failing exit code #4193

Open
slifty opened this issue Aug 11, 2023 · 0 comments

Comments

@slifty
Copy link

slifty commented Aug 11, 2023

Version

29.1.1

Steps to reproduce

  1. Clone my repo at https://github.com/slifty/ts-jest-bug-example
  2. npm i
  3. npx jest --runInBand --coverage
  4. echo $?
  5. You should see exit code 0 despite the errors

Expected behavior

I expect:

  1. the coverage report to generate and
  2. for the exit code to be 1.

Actual behavior

Exit code 0 is returned

Debug log

ts-jest.log

Additional context

Running npx jest --coverage without --runInBand does not cause this bug and instead renders output such as:

Running coverage on untested files...Failed to collect coverage from /Users/slifty/Maestral/Code/personal/jesttest/src/foo.ts
ERROR: Jest worker encountered 3 child process exceptions, exceeding retry limit
STACK: Error: Jest worker encountered 3 child process exceptions, exceeding retry limit
    at ChildProcessWorker.initialize (/Users/slifty/Maestral/Code/personal/jesttest/node_modules/jest-worker/build/workers/ChildProcessWorker.js:211:21)
    at ChildProcessWorker._onExit (/Users/slifty/Maestral/Code/personal/jesttest/node_modules/jest-worker/build/workers/ChildProcessWorker.js:396:12)
    at ChildProcess.emit (node:events:513:28)
    at Process.ChildProcess._handle.onexit (node:internal/child_process:291:12)
----------|---------|----------|---------|---------|-------------------
File      | % Stmts | % Branch | % Funcs | % Lines | Uncovered Line #s
----------|---------|----------|---------|---------|-------------------
All files |       0 |        0 |       0 |       0 |
----------|---------|----------|---------|---------|-------------------
Test Suites: 1 failed, 1 total
Tests:       0 total
Snapshots:   0 total
Time:        4 s
Ran all test suites.

Running npx jest --runInBand --coverage only renders:

Running coverage on untested files...

and then the process exits with code 0.

Some other interesting "alternative outcomes":

  • If collectCoverageFrom is not specified then coverage is generated as expected and the process returns correct exit codes.
  • If the tests are able to run then then coverage is STILL not generated and the exit code is always 1 regardless of test outcomes.

I had originally opened this issue in the jest repository but they pointed out this issue does NOT happen with babel-jest and so the problem might be a bug in ts-jest

Environment

System:
    OS: macOS 12.6
    CPU: (8) x64 Intel(R) Core(TM) i7-8559U CPU @ 2.70GHz
  Binaries:
    Node: 20.5.0 - ~/.nvm/versions/node/v20.5.0/bin/node
    Yarn: 1.22.19 - /usr/local/bin/yarn
    npm: 9.8.0 - ~/.nvm/versions/node/v20.5.0/bin/npm
  npmPackages:
    jest: ^29.6.2 => 29.6.2
slifty added a commit to PhilanthropyDataCommons/service that referenced this issue Aug 25, 2023
There is currently a bug in the `ts-jest` project [1] which causes
compile failures to result in a masked failing exit code *if test
coverage is being generated*.

The best solution would be to patch upstream, but in lieu of that we are
going to just run tests twice: once without coverage and once with
coverage.

Issue #457 npm run test:ci prints failures but exits success

[1] kulshekhar/ts-jest#4193
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant