Tests logging after teardown do not fail tests in non-watch mode

[NMinhNguyen]

🐛 Bug Report

#10728 sets the exit code to 1 when tests log after teardown. This correctly terminates the process if run in non-watch mode. However, if you do jest --watch then all tests are reported as passing, despite there being a warning:

Cannot log after tests are done. Did you forget to wait for something async in your test?

Additionally, I noticed that console.warn and console.error don't cause failures - is that intended? I'm also happy to open a dedicated issue if you prefer keeping things isolated.

To Reproduce

Steps to reproduce the behavior:

1. Given this test:

   // sum.test.js
   
   const delay = ms => new Promise(resolve => setTimeout(resolve, ms));
   
   test('should fail in watch mode', () => {
     async function foo() {
       await delay(100);
   
       console.log('this should fail tests in watch mode too');
     }
   
     foo();
     expect(true).toBe(true);
   });

2. Run yarn jest --watchAll (or --watch if running inside a git repo)

3. Observe that Jest reports a passing test, although it does log a warning:

   

4. Now if you just run yarn jest, you'll see the process exits with exit code 1.

Expected behavior

Jest should report a test failure.

Link to repl or repo (highly encouraged)

https://repl.it/@NMinhNguyen/jest-playground#sum.test.js

Note: you'll need to run it in watch mode: yarn jest --watchAll (--watchAll since it's not inside a git repo).

envinfo

npx envinfo --preset jest
npx: installed 1 in 3.502s

  System:
    OS: Linux 5.4 Debian GNU/Linux 9 (stretch) 9 (stretch)
    CPU: (4) x64 Intel(R) Xeon(R) CPU @ 2.30GHz
  Binaries:
    Node: 12.18.3 - /usr/local/bin/node
    Yarn: 1.22.4 - /usr/local/bin/yarn
    npm: 6.14.6 - /usr/local/bin/npm
  npmPackages:
    jest: 27.0.0-next.3 => 27.0.0-next.3 

[snitin315]

I will take a look and send a PR for this in near future.

[SimenB]

Thanks @snitin315!

[noobyogi0010]

@SimenB Can I take this if no one's working on it already?

[SimenB]

AFAIK this is still relevant - can't remember merging anything that should affect it at least 🙂

[regevbr]

I'm experiencing the same issue - when running on my machine, I get the Cannot log after tests are done. Did you forget to wait for something async in your test? errors but Jest still finishes with exit code 0. When running the same jest command, but with --runInBand Jest does exit with code 1 like it should. I took me hours to figure out why Jest failed in my CI machine and not locally :-(

[chrisgarber]

I also spent many hours trying to debug this. IMO the user experience of having no tests fail, but having the process exit with a non-0 status is extremely confusing. Is there any way to correlate logging with the test that may have issued the promise (or whatever) that caused the logging after cleanup? The other problem that I experienced is that it was quite difficult to definitively find which test was causing the issue.

[john-pimq]

I also meet the problem. The jest show all tests is passed, however it return the exit code to 1 so it make my CI machine failed (this situation is appear mostly on CI machine and difficult to reproduce when running test on local). Also the error log can not help me to find the cause of the problem, I have no ideal how to handle this occasional fail test situation on my CI machine.

P.S. The error message Cannot log after tests are done. Did you forget to wait for something async in your test? show occasionally make the exit code return 1 and fail the CI machine process when running test.

[ChristianSchwarz]

@snitin315 commented on Mar 1, 2021

I will take a look and send a PR for this in near future.

Thumbs up! May the "near future" not be long away.

[github-actions]

This issue is stale because it has been open for 1 year with no activity. Remove stale label or comment or this will be closed in 30 days.

[github-actions]

This issue was closed because it has been stalled for 30 days with no activity. Please open a new issue if the issue is still relevant, linking to this one.