test-runner-module-mocking: bug when loaded normally in previous test before intercepted in next

[bashmish]

What I did

1. provided a minimal reproduction of the bug

We discussed a similar situation, but different, when the order of mocking and importing was relevant within a single test file.
But between 2 independent test files such problem wasn't discussed thoroughly, so I consider this to be a bug.
I hope my reproduction sheds some light on what goes wrong.

The biggest difference is that within a single test file you can make any order of loading modules you want, therefore this solution was OK.
But between 2 files I can't really guarantee this. In my repository I have tens of test files, many of them import reusable utilities. The module graph can be quite random depending on the unit test entry point.
The only solution I see is to have a special entry point for all unit tests which is guaranteed to be first loaded which will mock all necessary modules beforehand, but I don't think this is an acceptable solution, it scatters the test logic. We can as well just make a configuration for this then, but still same problem with scattering the test logic.

Fun fact: I was even able to use test-runner-module-mocking first without any issue, before smth changed in the order of test files loading, and caused this bug. Not so fun actually, as I spent a few hours debugging this and finding the root cause.

[changeset-bot]

⚠️ No Changeset found

Latest commit: 6555847

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

[tmsns]

I have seen this problem as well.
It seems that loaded modules are shared across runs of test files, across different wtr-sessions. This is weird, of course, as each session is normally running in its own context.

Do you have the issue when you disable concurrency? (ie set to 1) I remember this worked for me.

[bashmish]

I have seen this problem as well. It seems that loaded modules are shared across runs of test files, across different wtr-sessions. This is weird, of course, as each session is normally running in its own context.

Do you have the issue when you disable concurrency? (ie set to 1) I remember this worked for me.

Setting concurrency: 1 doesn't help, at least not in my case. It might help you, because it changes the order of executing the tests. In my case the order is still problematic, so in general it can't solve the problem.
I also tried to disable ETag plugin, to fully disable browser caching, it didn't help either.
Because it's not cached only in the browser, but WDS caches it first time when module is loaded, and if it's mocked after that, then it doesn't transform it again, resulting in mocking not having any effect.

I came across this today https://rollupjs.org/plugin-development/#shouldtransformcachedmodule, maybe we can look into invalidating the cache next time the same module is being requested, if it needs mocking. I don't know who exactly caches in WDS, maybe it's one of default plugins (rollup-based or just native WDS ones). ETag should then change too, so disabling the ETag plugin won't be needed either (I was just playing, in real life you don't want to disable ETag, especially if we end up having concurrency: 1 due to other bugs caused by inactive tab behavior in Chromium). Just throwing ideas here.