Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add a seed value to test runs #13400

Merged
merged 57 commits into from Oct 9, 2022
Merged
Show file tree
Hide file tree
Changes from 17 commits
Commits
Show all changes
57 commits
Select commit Hold shift + click to select a range
886a1e2
planned documentation
Joshua-Hwang Oct 6, 2022
d551da5
expose seed to runtime
Joshua-Hwang Oct 6, 2022
4e44bef
WIP documentation
Joshua-Hwang Oct 6, 2022
0877be4
integration tests for passing the seed to runtime
Joshua-Hwang Oct 6, 2022
d1f5311
seed is generated when not supplied
Joshua-Hwang Oct 6, 2022
49c060f
fixed other tests
Joshua-Hwang Oct 6, 2022
bc8b4eb
improved documentation
Joshua-Hwang Oct 6, 2022
c5ca4d7
Merge branch 'facebook:main' into feat-get-seed
Joshua-Hwang Oct 6, 2022
fecf106
added copyright
Joshua-Hwang Oct 6, 2022
c39e63f
added replace seed function
Joshua-Hwang Oct 7, 2022
923f556
added Seed to reporter
Joshua-Hwang Oct 7, 2022
07f7b55
fixed some e2e tests for new Seed report
Joshua-Hwang Oct 7, 2022
3987577
updated line number of e2e test
Joshua-Hwang Oct 7, 2022
dda334e
updated snapshots
Joshua-Hwang Oct 7, 2022
3c61bc2
Merge branch 'main' of https://github.com/facebook/jest into feat-get…
jhwang98 Oct 7, 2022
08e53ac
added changelog
jhwang98 Oct 7, 2022
e848382
fixed e2e tests
jhwang98 Oct 7, 2022
9424589
lint fix
Joshua-Hwang Oct 7, 2022
83f5e9a
updated further snapshots
Joshua-Hwang Oct 7, 2022
bfcc0b4
removed seed from initial options
Joshua-Hwang Oct 8, 2022
3e78642
added typing tests
Joshua-Hwang Oct 8, 2022
ba7f269
documentation and messages made more clear
Joshua-Hwang Oct 8, 2022
40047fe
added --showSeed arg
Joshua-Hwang Oct 8, 2022
6730725
moved seed from environment to ctor
Joshua-Hwang Oct 8, 2022
589d434
snapshots updated
Joshua-Hwang Oct 8, 2022
4eafd79
fixed up e2e tests
Joshua-Hwang Oct 8, 2022
ebdf7cb
added new e2e test for --showSeed
Joshua-Hwang Oct 8, 2022
116bdca
linting fix
Joshua-Hwang Oct 8, 2022
b7d5faa
Merge branch 'feat-get-seed' of github.com:jhwang98/jest into feat-ge…
Joshua-Hwang Oct 8, 2022
c7d196a
added showseed docs
Joshua-Hwang Oct 8, 2022
d2af3a6
showSeed is the only way to print the seed
Joshua-Hwang Oct 8, 2022
2ea7dc6
e2e tests related are updated
Joshua-Hwang Oct 8, 2022
b9b8abf
snapshots updated
Joshua-Hwang Oct 8, 2022
679521b
added showSeed to config
Joshua-Hwang Oct 8, 2022
adda4f7
added e2e test for showSeed config
Joshua-Hwang Oct 8, 2022
01ac6f3
documented showSeed config
Joshua-Hwang Oct 8, 2022
71c04c4
updated docs
Joshua-Hwang Oct 8, 2022
1fee0a7
globalConfig passed instead and error thrown when seed is not present
Joshua-Hwang Oct 8, 2022
7b97aab
lint fix
Joshua-Hwang Oct 8, 2022
076eb02
replace Date.now
Joshua-Hwang Oct 8, 2022
d0eed4a
added showSeed test
Joshua-Hwang Oct 9, 2022
523b995
use jest.useFakeTimer instead
Joshua-Hwang Oct 9, 2022
1328b7b
removed optional chaining
Joshua-Hwang Oct 9, 2022
1aae653
lint fix and error now has message
Joshua-Hwang Oct 9, 2022
d568629
clarified docs
Joshua-Hwang Oct 9, 2022
e246b8a
Changelog updated
Joshua-Hwang Oct 9, 2022
3efe6fc
tweak changelog
SimenB Oct 9, 2022
0fb3e79
tweak link
SimenB Oct 9, 2022
d023a21
run same integration test
SimenB Oct 9, 2022
535730b
do not use snapshots in tests
SimenB Oct 9, 2022
3cb272b
tweak doc
SimenB Oct 9, 2022
b8a70eb
test bounds of seed
SimenB Oct 9, 2022
b2021e5
no hooks
SimenB Oct 9, 2022
91212a2
remove unused optional
SimenB Oct 9, 2022
ed5f9c1
oops
SimenB Oct 9, 2022
157ea7e
Merge branch 'main' into feat-get-seed
SimenB Oct 9, 2022
cf0230c
prettier
SimenB Oct 9, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
Expand Up @@ -2,6 +2,7 @@

### Features

- `[@jest/cli, jest-config, @jest/environment, jest-environment-node, jest-environment-jsdom, @jest/reporters, jest-runtime, @jest/types]` Add `getSeed()` to the jest object available at runtime ([#13400](https://github.com/facebook/jest/pull/13400/))
SimenB marked this conversation as resolved.
Show resolved Hide resolved
- `[jest-config]` Add `readInitialConfig` utility function ([#13356](https://github.com/facebook/jest/pull/13356))
- `[jest-core]` Enable testResultsProcessor to be async ([#13343](https://github.com/facebook/jest/pull/13343))
- `[expect, @jest/expect-utils]` Allow `isA` utility to take a type argument ([#13355](https://github.com/facebook/jest/pull/13355))
Expand Down
11 changes: 11 additions & 0 deletions docs/CLI.md
Expand Up @@ -350,6 +350,17 @@ The default regex matching works fine on small runs, but becomes slow if provide

:::

### `--seed=<num>`

Provides a seed number that the tests have access to via `jest.getSeed()`.
If this option isn't specified Jest will randomly generate a seed.
SimenB marked this conversation as resolved.
Show resolved Hide resolved

The seed value must be between `-0x80000000` to `0x7fffffff` inclusive.
SimenB marked this conversation as resolved.
Show resolved Hide resolved

```bash
jest --seed=1324
```

### `--selectProjects <project1> ... <projectN>`

Run the tests of the specified projects. Jest uses the attribute `displayName` in the configuration to identify each project. If you use this option, you should provide a `displayName` to all your projects.
Expand Down
9 changes: 9 additions & 0 deletions docs/JestObjectAPI.md
Expand Up @@ -908,6 +908,15 @@ This function is not available when using legacy fake timers implementation.

## Misc

### `jest.getSeed()`
Every time Jest runs a seed value is randomly generated which you could use in a pseudorandom number generator or anywhere else.
SimenB marked this conversation as resolved.
Show resolved Hide resolved

:::tip

You can manually define this value with the CLI argument `--seed=<num>`.
SimenB marked this conversation as resolved.
Show resolved Hide resolved

:::

### `jest.setTimeout(timeout)`

Set the default timeout interval (in milliseconds) for all tests and before/after hooks in the test file. This only affects the test file from which this function is called. The default timeout interval is 5 seconds if this method is not called.
Expand Down
11 changes: 7 additions & 4 deletions e2e/Utils.ts
Expand Up @@ -157,6 +157,9 @@ export const copyDir = (src: string, dest: string) => {
}
};

export const replaceSeed = (str: string) =>
str.replace(/Seed: (-?\d+)/g, 'Seed: <<REPLACED>>');

export const replaceTime = (str: string) =>
str
.replace(/\d*\.?\d+ m?s\b/g, '<<REPLACED>>')
Expand Down Expand Up @@ -201,7 +204,7 @@ export const extractSummary = (stdout: string) => {
const match = stdout
.replace(/(?:\\[rn])+/g, '\n')
.match(
/Test Suites:.*\nTests.*\nSnapshots.*\nTime.*(\nRan all test suites)*.*\n*$/gm,
/Seed:.*\nTest Suites:.*\nTests.*\nSnapshots.*\nTime.*(\nRan all test suites)*.*\n*$/gm,
);
if (!match) {
throw new Error(dedent`
Expand All @@ -211,7 +214,7 @@ export const extractSummary = (stdout: string) => {
`);
}

const summary = replaceTime(match[0]);
const summary = replaceSeed(replaceTime(match[0]));

const rest = stdout
.replace(match[0], '')
Expand Down Expand Up @@ -245,7 +248,7 @@ const sortTests = (stdout: string) =>
export const extractSortedSummary = (stdout: string) => {
const {rest, summary} = extractSummary(stdout);
return {
rest: sortTests(replaceTime(rest)),
rest: sortTests(replaceSeed(replaceTime(rest))),
summary,
};
};
Expand All @@ -254,7 +257,7 @@ export const extractSummaries = (
stdout: string,
): Array<{rest: string; summary: string}> => {
const regex =
/Test Suites:.*\nTests.*\nSnapshots.*\nTime.*(\nRan all test suites)*.*\n*$/gm;
/Seed:.*\nTest Suites:.*\nTests.*\nSnapshots.*\nTime.*(\nRan all test suites)*.*\n*$/gm;

let match = regex.exec(stdout);
const matches: Array<RegExpExecArray> = [];
Expand Down
Expand Up @@ -9,7 +9,8 @@ Force exiting Jest: Have you considered using \`--detectOpenHandles\` to detect
`;

exports[`CLI accepts exact file names if matchers matched 2`] = `
"Test Suites: 1 passed, 1 total
"Seed: <<REPLACED>>
Test Suites: 1 passed, 1 total
Tests: 1 passed, 1 total
Snapshots: 0 total
Time: <<REPLACED>>
Expand Down
21 changes: 14 additions & 7 deletions e2e/__tests__/__snapshots__/console.test.ts.snap
Expand Up @@ -41,7 +41,8 @@ exports[`console printing 1`] = `
`;

exports[`console printing 2`] = `
"Test Suites: 1 passed, 1 total
"Seed: <<REPLACED>>
Test Suites: 1 passed, 1 total
Tests: 1 passed, 1 total
Snapshots: 0 total
Time: <<REPLACED>>
Expand Down Expand Up @@ -92,7 +93,8 @@ exports[`console printing with --verbose 2`] = `
`;

exports[`console printing with --verbose 3`] = `
"Test Suites: 1 passed, 1 total
"Seed: <<REPLACED>>
Test Suites: 1 passed, 1 total
Tests: 1 passed, 1 total
Snapshots: 0 total
Time: <<REPLACED>>
Expand Down Expand Up @@ -127,7 +129,8 @@ exports[`does not error out when using winston 2`] = `
`;

exports[`does not error out when using winston 3`] = `
"Test Suites: 1 passed, 1 total
"Seed: <<REPLACED>>
Test Suites: 1 passed, 1 total
Tests: 1 passed, 1 total
Snapshots: 0 total
Time: <<REPLACED>>
Expand All @@ -139,7 +142,8 @@ exports[`does not print to console with --silent 1`] = `""`;
exports[`does not print to console with --silent 2`] = `"PASS __tests__/console.test.js"`;

exports[`does not print to console with --silent 3`] = `
"Test Suites: 1 passed, 1 total
"Seed: <<REPLACED>>
Test Suites: 1 passed, 1 total
Tests: 1 passed, 1 total
Snapshots: 0 total
Time: <<REPLACED>>"
Expand Down Expand Up @@ -174,7 +178,8 @@ exports[`respects --noStackTrace 2`] = `
`;

exports[`respects --noStackTrace 3`] = `
"Test Suites: 1 passed, 1 total
"Seed: <<REPLACED>>
Test Suites: 1 passed, 1 total
Tests: 1 passed, 1 total
Snapshots: 0 total
Time: <<REPLACED>>
Expand Down Expand Up @@ -210,7 +215,8 @@ exports[`respects noStackTrace in config 2`] = `
`;

exports[`respects noStackTrace in config 3`] = `
"Test Suites: 1 passed, 1 total
"Seed: <<REPLACED>>
Test Suites: 1 passed, 1 total
Tests: 1 passed, 1 total
Snapshots: 0 total
Time: <<REPLACED>>
Expand All @@ -225,7 +231,8 @@ exports[`the jsdom console is the same as the test console 2`] = `
`;

exports[`the jsdom console is the same as the test console 3`] = `
"Test Suites: 1 passed, 1 total
"Seed: <<REPLACED>>
Test Suites: 1 passed, 1 total
Tests: 1 passed, 1 total
Snapshots: 0 total
Time: <<REPLACED>>
Expand Down
3 changes: 2 additions & 1 deletion e2e/__tests__/__snapshots__/consoleDebugging.test.ts.snap
Expand Up @@ -14,7 +14,8 @@ exports[`console debugging with --verbose 2`] = `
`;

exports[`console debugging with --verbose 3`] = `
"Test Suites: 1 passed, 1 total
"Seed: <<REPLACED>>
Test Suites: 1 passed, 1 total
Tests: 1 passed, 1 total
Snapshots: 1 passed, 1 total
Time: <<REPLACED>>
Expand Down
Expand Up @@ -9,7 +9,8 @@ Force exiting Jest: Have you considered using \`--detectOpenHandles\` to detect
`;

exports[`prints console.logs when run with forceExit 2`] = `
"Test Suites: 1 passed, 1 total
"Seed: <<REPLACED>>
Test Suites: 1 passed, 1 total
Tests: 1 passed, 1 total
Snapshots: 0 total
Time: <<REPLACED>>
Expand Down
3 changes: 2 additions & 1 deletion e2e/__tests__/__snapshots__/coverageReport.test.ts.snap
Expand Up @@ -61,7 +61,8 @@ All files | 60 | 0 | 50 | 60 |
`;

exports[`json reporter printing with --coverage 1`] = `
"Test Suites: 1 failed, 1 total
"Seed: <<REPLACED>>
Test Suites: 1 failed, 1 total
Tests: 1 failed, 1 skipped, 2 passed, 4 total
Snapshots: 0 total
Time: <<REPLACED>>
Expand Down
12 changes: 8 additions & 4 deletions e2e/__tests__/__snapshots__/coverageThreshold.test.ts.snap
Expand Up @@ -8,7 +8,8 @@ Jest: "global" coverage threshold for lines (100%) not met: 0%"
`;

exports[`excludes tests matched by path threshold groups from global group 2`] = `
"Test Suites: 1 passed, 1 total
"Seed: <<REPLACED>>
Test Suites: 1 passed, 1 total
Tests: 1 passed, 1 total
Snapshots: 0 total
Time: <<REPLACED>>
Expand Down Expand Up @@ -42,7 +43,8 @@ Jest: "global" coverage threshold for lines (90%) not met: 50%"
`;

exports[`exits with 1 if coverage threshold is not met 2`] = `
"Test Suites: 1 passed, 1 total
"Seed: <<REPLACED>>
Test Suites: 1 passed, 1 total
Tests: 1 passed, 1 total
Snapshots: 0 total
Time: <<REPLACED>>
Expand All @@ -66,7 +68,8 @@ Jest: Coverage data for apple.js was not found."
`;

exports[`exits with 1 if path threshold group is not found in coverage data 2`] = `
"Test Suites: 1 passed, 1 total
"Seed: <<REPLACED>>
Test Suites: 1 passed, 1 total
Tests: 1 passed, 1 total
Snapshots: 0 total
Time: <<REPLACED>>
Expand All @@ -92,7 +95,8 @@ Jest: "./banana.js" coverage threshold for lines (100%) not met: 50%"
`;

exports[`file is matched by all path and glob threshold groups 2`] = `
"Test Suites: 1 passed, 1 total
"Seed: <<REPLACED>>
Test Suites: 1 passed, 1 total
Tests: 1 passed, 1 total
Snapshots: 0 total
Time: <<REPLACED>>
Expand Down
3 changes: 2 additions & 1 deletion e2e/__tests__/__snapshots__/customReporters.test.ts.snap
Expand Up @@ -78,7 +78,8 @@ exports[`Custom Reporters Integration default reporters enabled 1`] = `
`;

exports[`Custom Reporters Integration default reporters enabled 2`] = `
"Test Suites: 1 passed, 1 total
"Seed: <<REPLACED>>
Test Suites: 1 passed, 1 total
Tests: 1 passed, 1 total
Snapshots: 0 total
Time: <<REPLACED>>
Expand Down
1 change: 1 addition & 0 deletions e2e/__tests__/__snapshots__/domDiffing.test.ts.snap
Expand Up @@ -43,6 +43,7 @@ exports[`should work without error 1`] = `

at Object.toBe (__tests__/dom.test.js:18:41)

Seed: <<REPLACED>>
Test Suites: 1 failed, 1 total
Tests: 2 failed, 2 total
Snapshots: 0 total
Expand Down
12 changes: 8 additions & 4 deletions e2e/__tests__/__snapshots__/emptyDescribeWithHooks.test.ts.snap
Expand Up @@ -3,7 +3,8 @@
exports[`hook in describe with skipped test 1`] = `
Object {
"rest": "",
"summary": "Test Suites: 1 skipped, 0 of 1 total
"summary": "Seed: <<REPLACED>>
Test Suites: 1 skipped, 0 of 1 total
Tests: 1 skipped, 1 total
Snapshots: 0 total
Time: <<REPLACED>>
Expand All @@ -30,7 +31,8 @@ Object {

at beforeEach (__tests__/hookInEmptyDescribe.test.js:9:3)
at Object.describe (__tests__/hookInEmptyDescribe.test.js:8:1)",
"summary": "Test Suites: 1 failed, 1 total
"summary": "Seed: <<REPLACED>>
Test Suites: 1 failed, 1 total
Tests: 1 passed, 1 total
Snapshots: 0 total
Time: <<REPLACED>>
Expand All @@ -57,7 +59,8 @@ Object {

at beforeEach (__tests__/hookInEmptyNestedDescribe.test.js:9:3)
at Object.describe (__tests__/hookInEmptyNestedDescribe.test.js:8:1)",
"summary": "Test Suites: 1 failed, 1 total
"summary": "Seed: <<REPLACED>>
Test Suites: 1 failed, 1 total
Tests: 1 passed, 1 total
Snapshots: 0 total
Time: <<REPLACED>>
Expand Down Expand Up @@ -129,7 +132,8 @@ Object {

at beforeAll (__tests__/multipleHooksInEmptyDescribe.test.js:12:3)
at Object.describe (__tests__/multipleHooksInEmptyDescribe.test.js:8:1)",
"summary": "Test Suites: 1 failed, 1 total
"summary": "Seed: <<REPLACED>>
Test Suites: 1 failed, 1 total
Tests: 1 passed, 1 total
Snapshots: 0 total
Time: <<REPLACED>>
Expand Down
9 changes: 6 additions & 3 deletions e2e/__tests__/__snapshots__/findRelatedFiles.test.ts.snap
@@ -1,7 +1,8 @@
// Jest Snapshot v1, https://goo.gl/fbAQLP

exports[`--findRelatedTests flag coverage configuration is applied correctly 1`] = `
"Test Suites: 1 passed, 1 total
"Seed: <<REPLACED>>
Test Suites: 1 passed, 1 total
Tests: 1 passed, 1 total
Snapshots: 0 total
Time: <<REPLACED>>
Expand All @@ -23,7 +24,8 @@ All files | 100 | 100 | 100 | 100 |
`;

exports[`--findRelatedTests flag generates coverage report for filename 1`] = `
"Test Suites: 2 passed, 2 total
"Seed: <<REPLACED>>
Test Suites: 2 passed, 2 total
Tests: 2 passed, 2 total
Snapshots: 0 total
Time: <<REPLACED>>
Expand All @@ -46,7 +48,8 @@ All files | 100 | 100 | 100 | 100 |
`;

exports[`--findRelatedTests flag generates coverage report for filename 4`] = `
"Test Suites: 1 passed, 1 total
"Seed: <<REPLACED>>
Test Suites: 1 passed, 1 total
Tests: 1 passed, 1 total
Snapshots: 0 total
Time: <<REPLACED>>
Expand Down