api: allow reuse of support code library between runs in the same process

[nicojs]

Describe the bug
When loading support code files, Cucumber 8 uses import(...) instead of require('..'). This is great, but without a way to reuse support library code, specified in #1711, it would break StrykerJS integration.

NOTE: you might already know this to be the case, but I personally would love this to be fixed before the v8 release via this API: #1711 (comment). Otherwise, StrykerJS will be unable to support cucumber v8

To Reproduce

This code example works for @cucumber/cucumber@7.3.2, but not for @cucumber/cucumber@8.0.0-rc2.

const { Cli } = require('@cucumber/cucumber');
const { DirectoryRequireCache } = require('@stryker-mutator/util');

const cli = new Cli({
  argv: ['', ''],
  cwd: process.cwd(),
  stdout: process.stdout,
  env: process.env,
});

const cache = new DirectoryRequireCache();

async function main() {
  const result = await cli.run();
  console.log(result);
  cache.record();
  cache.clear();
  const result2 = await cli.run();
  console.log(result2);
}

main().catch(console.error);

In this example, the DirectoryRequireCache class is used, which is a utility that clears all local files from the require.cache.

Expected behavior

.


4 scenarios (4 passed)
12 steps (12 passed)
0m00.046s (executing steps: 0m00.009s)
{ shouldExitImmediately: false, success: true }
.


4 scenarios (4 passed)
12 steps (12 passed)
0m00.044s (executing steps: 0m00.006s)
{ shouldExitImmediately: false, success: true }

Actual behavior:

.

4 scenarios (4 passed)
12 steps (12 passed)
0m00.032s (executing steps: 0m00.004s)
{
  shouldAdvertisePublish: false,
  shouldExitImmediately: false,
  success: true
}

U

Failures:

1) Scenario: easy maths # features/simple_math.feature:7
  ? Given a variable set to 1
       Undefined. Implement with the following snippet:
  [...]

Proposed solution

Implement the proposal mentioned here: #1711 (comment) before the v8 release.

Additional context

When you use import(...) to load a cjs module, nodejs will use the CommonJS loader initially to load that file. However, it places it in the "import cache" (for lack of a better word), which is the cache that we cannot clear. Any subsequent imports of that file will be reclaimed from that cache. This means that support code cannot be reloaded after it is loaded initially.

I didn't realize this myself, until jasmine v4 got released last week, which broke the StrykerJS integration.

See for a code example demonstrating the issue stryker-mutator/stryker-js#2922 (comment)

[davidjgoss]

Thanks for raising @nicojs. I'm definitely planning to tackle this issue before v8.

Aside from this problem with cached imports, do you see any other issues with StrykerJS adopting the runCucumber API? There's the issue of loading config (including from profiles) - any others?

[nicojs]

No other issues that I can think of, as long as the custom formatter API remains the same.
I can do tests with the new runCucumber API if you release it in an RC

[nicovak]

I tried v8-rc also and got the same issue. I also tried with the private objects directly but AFAIK Cucumber Cli is using runCucumber under the hood, so It's not usefull.