This repository explores a core-layer issue that vm.script and vm.runInContext have inconsistency v8 coverage from the different test-sequence when test by Jest.
Just 1 step, you can reproduce this issue.
This repo contains 2 test files, 2 events, and 1 common module.
To make the issue as easy-understanding as possible, here are 4 test cases in npm script to help you see the issue.
All of these cases print the function coverage from the v8 coverage result.
- Case1 shows the result from the 2 test files, and the testA.test.js test first.
- Case2 shows the result from the 2 test files, and the testB.test.js test first.
- Case3 shows the result from the testA.test.js only.
- Case3 shows the result from the testB.test.js only.
Please observe the result of testA.test.js.
You could see the difference in the "ranges" key.
Such as:
- Obtain 4 objects in ranges in Case1.
- Obtain 1 object in ranges in Case2.
But in Case3, there are 4 test objects same as the result from Case1.
(Case4 has no result from testA.test.js, therefore please ignore this case temporarily.)
The Jest coverage generated by merged v8 coverage result,
This issue would cause the Jest coverage report generages wrong count-numbers and wrong coverage-numbers.
And I do have one wrong report now, this repository is simplified from that case I have.
My team has no choice but to adopt vm.script and related method to figure out the problem.
However, I have no clue to track the root cause from this issue, neither in Jest, v8 engine nor other material.
I hope this post can make resonate or notice, to help us know the real reason.
At least, we know where the problem is, we have more knowledge to control the effect scale.
If there are some developer willing to figure this issue out, would be better than expection.