Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

mocha --parallel sometimes fails silently in CI #6940

Closed
echuber2 opened this issue Jan 12, 2023 · 3 comments · May be fixed by mochajs/mocha#4959
Closed

mocha --parallel sometimes fails silently in CI #6940

echuber2 opened this issue Jan 12, 2023 · 3 comments · May be fixed by mochajs/mocha#4959
Labels
bug A bug or regression

Comments

@echuber2
Copy link
Collaborator

There's an issue with the invocations of mocha --parallel in docker/test_js.sh which is used by the CI workflow. It seems that on some systems, this can fail silently.

To reproduce:

  • Checkout 325b89b (prior to the chalk fix) and rebuild a container for testing.
  • In the container, make this edit to a few lines in Makefile to speed up the example:

Old:

test-js: test-prairielearn test-prairielib test-grader-host test-workspace-host test-packages
test-prairielearn: start-support
        @yarn mocha --parallel "tests/**/*.test.{js,mjs}"

Modified: (The text here doesn't reflect this, but you need to use tabs, not spaces.)

test-js: test-prairielearn
test-prairielearn: start-support
        @yarn mocha --parallel "tests/access.test.js" "tests/database.test.js"
        echo finished with no error but should have failed
  • Run: docker/test_js.sh. On a 4-core CPU, this seems to succeed with no error (but it should have failed).
  • Now try again without --parallel. It should fail as expected.

Similarly, I note this behavior:

  • Using --parallel --jobs 2 has the problem (test succeeds when it should not).
  • Using --parallel --jobs 1 does not have the problem (test fails as expected).
  • Using --bail --parallel --jobs 2 does not have the problem (test fails as expected).
  • (Accordingly:) Using --bail --parallel (no jobs limit) does not have the problem (test fails as expected). Maybe this is an acceptable workaround for now?

This may be related to a bug with mocha's parallel mode that was previously reported: mochajs/mocha#4559

@nwalters512
Copy link
Contributor

The bug reported upstream appears to only affect ESM files that import from mocha, neither of which applies to us. It's possible that this is a similar but unrelated issue?

@echuber2
Copy link
Collaborator Author

Yeah, I don't think it's exactly the same, but there could be a root cause they only bandaided over before.

@nwalters512
Copy link
Contributor

We haven't seen this issue occur again since this was opened. mochajs/mocha#4959 was opened to fix this upstream; I don't think there's anything else actionable for us to do at this point.

@nwalters512 nwalters512 closed this as not planned Won't fix, can't repro, duplicate, stale Jun 8, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug A bug or regression
Projects
None yet
2 participants