QUnit.module.only including sibling tests when used in the middle of a set

[sukima]

Tell us about your runtime:

• QUnit version: 2.16.0 (CLI and Browser)
• Which environment are you using? (e.g., browser, Node): Tested in both
• How are you running QUnit? (e.g., QUnit CLI, Grunt, Karma, manually in browser): QUnit CLI, and Manually in browser

What are you trying to do?

Code that reproduces the problem:

QUnit.module('Module A', () => {
    QUnit.test('Test A', assert => {
        assert.ok(0);
    });
    QUnit.module.only('Module B', () => {
        QUnit.test('Test B', assert => {
            assert.ok(1);
        });
    });
    QUnit.test('Test C', assert => {
        assert.ok(0);
    });
});

What did you expect to happen?

Expected only Module A > Module B > Test B to run.

What actually happened?

Both Module A > Module B > Test B and Module A > Test C ran evident by the failing assertion.

TAP version 13
ok 1 Module A > Module B > Test B
not ok 2 Module A > Test C
  ---
  message: failed, expected argument to be truthy, was: 0
  severity: failed
  actual  : 0
  expected: true
  stack: |
        at Object.<anonymous> (/Users/dweaver/source/foo/foo-test.js:11:16)
  ...
1..3
# pass 2
# skip 0
# todo 0
# fail 1

[sukima]

Possibly related to issue #1610

[Krinkle]

Issue #1610 was a recent regression where subsequent calls to QUnit.module() did not correctly take into account the only state of a preceeding module.

The issue you found is limited to QUnit.test() without further nesting in another sub-module. I've reproduced this in a CodePen and was able to reproduce the issue on all past versions where these methods exist (from 2.5.0 to 2.16.0), so I believe this never worked.

In thinking about why this has not come up before, I believe the reason is that developers usually do not not mix child modules between two child tests. The typical structure is to first define any tests belonging directly to the parent, and then define any child modules below that – in that order.

Internally, when module.only() is called, we clear the queue of all tests and modules, and then set a flag to remember to skip any future calls to module() unless they are a child of an "only"-marked module. We've never had logic in place to skip tests within the same parent.

Having said that, there is no reason for this not to work, and it seems perfectly valid to define them in a different order if you prefer that. I'm only mentioning the order as a way to get this to work today.

Thanks!