📝 Docs: Should let people know test file execution order can be controlled by import/require

[vassudanagunta]

The docs mention a number of ways that test file execution order can be controlled, but there is no mention of the most semantically powerful way: by importing or requiring testA.js in testB.js, testA.js will execute first.

This is incredibly powerful, as one is able to express that if both testA.js and testB.js tests fail, the failures should be examined in testA.js first, either because there is a dependency (e.g. testB.js depends on things tested by testA.js) or because testA.js tests something more fundamental.

See How to set execution order of mocha test cases in multiple files, in particular this answer.

[JoshuaKGoldberg]

🤔 importing from one test file to another is a pretty uncommon practice - and for good reason. Depending on how your tests are being run, it can result in describe() blocks getting executed multiple times. mocha --parallel will create a worker pool for tests. Each worker will execute all the describe()s that get imported in its files.

You can reproduce this by creating two test files:

// a.spec.js
require("./b.spec.js");
it("test a", () => {});

// b.spec.js
it("test b", () => {});

Running sequentially gets the nice ordering you're looking for:

$ npx mocha *.spec.js 

  ✔ test b
  ✔ test a

  2 passing (1ms)

...but with -parallel we get a duplicate test b:

$ npx mocha *.spec.js --parallel

  ✔ test b

  ✔ test b
  ✔ test a

  3 passing (74ms)

Other test frameworks such as Jest experience this as well. Which means it's also a problem for portability between frameworks.

An alternative way to control test file execution order is to change the order files are passed to the mocha command. In the above example, mocha b.spec.js a.spec.js runs test b before test a.

Closing as this isn't a recommendation we'll want to add to the docs. Thanks for filing though! +1 on the idea of filing SO answers as docs requests on the framework, that's a very good thing. 🤎

[vassudanagunta]

Thanks for your answer. I have zero objection to the wontfix.

An alternative way to control test file execution order is to change the order files are passed to the mocha command. In the above example, mocha b.spec.js a.spec.js runs test b before test a.

This doesn't scale very well, and the logical dependencies between tests aren't expressed where they should be, in the test that assumes functionality covered by another test works correctly.

Should I submit this as a feature request? Or do you think that's not gonna fly?

[JoshuaKGoldberg]

I wouldn't object to the filing, but note that per #5027 we're trying not to make any big splashy changes -including feature additions- that don't have a lot of demand from users. Given that this issue only had one 👍 I can tell you that it'd be a "no" from me and I'm guessing a "no" from the other maintainers. But I can't say for sure. Sorry to be the bearer of bad news. 😞