Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(jest-worker): add JestWorkerFarm helper type #12753

Merged
merged 3 commits into from Apr 27, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
2 changes: 2 additions & 0 deletions CHANGELOG.md
Expand Up @@ -2,6 +2,8 @@

### Features

- `[jest-worker]` Add `JestWorkerFarm` helper type ([#12753](https://github.com/facebook/jest/pull/12753))

### Fixes

### Chore & Maintenance
Expand Down
30 changes: 16 additions & 14 deletions packages/jest-haste-map/src/index.ts
Expand Up @@ -16,7 +16,7 @@ import {Stats, readFileSync, writeFileSync} from 'graceful-fs';
import type {Config} from '@jest/types';
import {escapePathForRegex} from 'jest-regex-util';
import {requireOrImportModule} from 'jest-util';
import {Worker} from 'jest-worker';
import {JestWorkerFarm, Worker} from 'jest-worker';
import HasteFS from './HasteFS';
import HasteModuleMap from './ModuleMap';
import H from './constants';
Expand Down Expand Up @@ -108,13 +108,16 @@ type Watcher = {
close(): Promise<void>;
};

type WorkerInterface = {worker: typeof worker; getSha1: typeof getSha1};
type HasteWorker = typeof import('./worker');

export {default as ModuleMap} from './ModuleMap';
export type {SerializableModuleMap} from './types';
export type {IModuleMap} from './types';
export type {default as FS} from './HasteFS';
export type {ChangeEvent, HasteMap as HasteMapObject} from './types';
export {default as ModuleMap} from './ModuleMap';
export type {
ChangeEvent,
HasteMap as HasteMapObject,
IModuleMap,
SerializableModuleMap,
} from './types';
Comment on lines +115 to +120
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just cleaned up type exports.


const CHANGE_INTERVAL = 30;
const MAX_WAIT_TIME = 240000;
Expand Down Expand Up @@ -216,7 +219,7 @@ export default class HasteMap extends EventEmitter {
private _isWatchmanInstalledPromise: Promise<boolean> | null = null;
private _options: InternalOptions;
private _watchers: Array<Watcher>;
private _worker: WorkerInterface | null;
private _worker: JestWorkerFarm<HasteWorker> | HasteWorker | null;

static getStatic(config: Config.ProjectConfig): HasteMapStatic {
if (config.haste.hasteMapModulePath) {
Expand Down Expand Up @@ -690,7 +693,7 @@ export default class HasteMap extends EventEmitter {
this._recoverDuplicates(hasteMap, relativeFilePath, fileMetadata[H.ID]);
}

const promises = [];
const promises: Array<Promise<void>> = [];
for (const relativeFilePath of filesToProcess.keys()) {
if (
this._options.skipPackageJson &&
Expand Down Expand Up @@ -726,9 +729,7 @@ export default class HasteMap extends EventEmitter {
private _cleanup() {
const worker = this._worker;

// @ts-expect-error
if (worker && typeof worker.end === 'function') {
// @ts-expect-error
if (worker && 'end' in worker && typeof worker.end === 'function') {
worker.end();
}

Expand All @@ -745,19 +746,20 @@ export default class HasteMap extends EventEmitter {
/**
* Creates workers or parses files and extracts metadata in-process.
*/
private _getWorker(options?: {forceInBand: boolean}): WorkerInterface {
private _getWorker(options?: {
forceInBand: boolean;
}): JestWorkerFarm<HasteWorker> | HasteWorker {
if (!this._worker) {
if ((options && options.forceInBand) || this._options.maxWorkers <= 1) {
this._worker = {getSha1, worker};
} else {
// @ts-expect-error: assignment of a worker with custom properties.
this._worker = new Worker(require.resolve('./worker'), {
exposedMethods: ['getSha1', 'worker'],
// @ts-expect-error: option does not exist on the node 12 types
forkOptions: {serialization: 'json'},
maxRetries: 3,
numWorkers: this._options.maxWorkers,
}) as WorkerInterface;
}) as JestWorkerFarm<HasteWorker>;
}
}

Expand Down
12 changes: 8 additions & 4 deletions packages/jest-reporters/src/CoverageReporter.ts
Expand Up @@ -26,10 +26,12 @@ import type {
} from '@jest/test-result';
import type {Config} from '@jest/types';
import {clearLine, isInteractive} from 'jest-util';
import {Worker} from 'jest-worker';
import {JestWorkerFarm, Worker} from 'jest-worker';
import BaseReporter from './BaseReporter';
import getWatermarks from './getWatermarks';
import type {CoverageWorker, ReporterContext} from './types';
import type {ReporterContext} from './types';

type CoverageWorker = typeof import('./CoverageWorker');

const FAIL_COLOR = chalk.bold.red;
const RUNNING_TEST_COLOR = chalk.bold.dim;
Expand Down Expand Up @@ -137,7 +139,9 @@ export default class CoverageReporter extends BaseReporter {
);
}

let worker: CoverageWorker | Worker;
let worker:
| JestWorkerFarm<CoverageWorker>
| typeof import('./CoverageWorker');

if (this._globalConfig.maxWorkers <= 1) {
worker = require('./CoverageWorker');
Expand All @@ -148,7 +152,7 @@ export default class CoverageReporter extends BaseReporter {
forkOptions: {serialization: 'json'},
maxRetries: 2,
numWorkers: this._globalConfig.maxWorkers,
});
}) as JestWorkerFarm<CoverageWorker>;
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Type casting is needed for now. Will work without casting after implementing createWorkerFarm().

}

const instrumentation = files.map(async fileObj => {
Expand Down
3 changes: 0 additions & 3 deletions packages/jest-reporters/src/types.ts
Expand Up @@ -13,15 +13,12 @@ import type {
TestResult,
} from '@jest/test-result';
import type {Config} from '@jest/types';
import type {worker} from './CoverageWorker';

export type ReporterOnStartOptions = {
estimatedTime: number;
showStatus: boolean;
};

export type CoverageWorker = {worker: typeof worker};

export interface Reporter {
readonly onTestResult?: (
test: Test,
Expand Down
16 changes: 6 additions & 10 deletions packages/jest-runner/src/index.ts
Expand Up @@ -16,17 +16,11 @@ import type {
} from '@jest/test-result';
import {deepCyclicCopy} from 'jest-util';
import type {TestWatcher} from 'jest-watcher';
import {PromiseWithCustomMessage, Worker} from 'jest-worker';
import {JestWorkerFarm, PromiseWithCustomMessage, Worker} from 'jest-worker';
import runTest from './runTest';
import type {SerializableResolver, worker} from './testWorker';
import type {SerializableResolver} from './testWorker';
import {EmittingTestRunner, TestRunnerOptions, UnsubscribeFn} from './types';

const TEST_WORKER_PATH = require.resolve('./testWorker');

interface WorkerInterface extends Worker {
worker: typeof worker;
}

export type {Test, TestEvents} from '@jest/test-result';
export type {Config} from '@jest/types';
export type {TestWatcher} from 'jest-watcher';
Expand All @@ -43,6 +37,8 @@ export type {
UnsubscribeFn,
} from './types';

type TestWorker = typeof import('./testWorker');

export default class TestRunner extends EmittingTestRunner {
readonly #eventEmitter = new Emittery<TestEvents>();

Expand Down Expand Up @@ -108,14 +104,14 @@ export default class TestRunner extends EmittingTestRunner {
}
}

const worker = new Worker(TEST_WORKER_PATH, {
const worker = new Worker(require.resolve('./testWorker'), {
exposedMethods: ['worker'],
// @ts-expect-error: option does not exist on the node 12 types
forkOptions: {serialization: 'json', stdio: 'pipe'},
maxRetries: 3,
numWorkers: this._globalConfig.maxWorkers,
setupArgs: [{serializableResolvers: Array.from(resolvers.values())}],
}) as WorkerInterface;
}) as JestWorkerFarm<TestWorker>;

if (worker.getStdout()) worker.getStdout().pipe(process.stdout);
if (worker.getStderr()) worker.getStderr().pipe(process.stderr);
Expand Down
80 changes: 80 additions & 0 deletions packages/jest-worker/__typetests__/jest-worker.test.ts
@@ -0,0 +1,80 @@
/**
* Copyright (c) Facebook, Inc. and its affiliates. All Rights Reserved.
*
* This source code is licensed under the MIT license found in the
* LICENSE file in the root directory of this source tree.
*/

import {expectError, expectType} from 'tsd-lite';
import type {JestWorkerFarm} from 'jest-worker';
import type * as testWorker from './testWorker';

type TestWorker = {
runTest: () => void;
isResult: boolean;
end: () => void; // reserved keys should be excluded from returned type
getStderr: () => string;
getStdout: () => string;
setup: () => void;
teardown: () => void;
};

// unknown JestWorkerFarm

declare const unknownWorkerFarm: JestWorkerFarm<Record<string, unknown>>;

expectError(unknownWorkerFarm.runTest());
expectError(unknownWorkerFarm.runTestAsync());

expectError(unknownWorkerFarm.getResult());
expectError(unknownWorkerFarm.isResult);

expectError(unknownWorkerFarm.setup());
expectError(unknownWorkerFarm.teardown());

expectType<Promise<{forceExited: boolean}>>(unknownWorkerFarm.end());
expectType<NodeJS.ReadableStream>(unknownWorkerFarm.getStderr());
expectType<NodeJS.ReadableStream>(unknownWorkerFarm.getStdout());

// detected JestWorkerFarm

declare const detectedWorkerFarm: JestWorkerFarm<typeof testWorker>;

expectType<Promise<void>>(detectedWorkerFarm.runTest());
expectType<Promise<void>>(detectedWorkerFarm.runTestAsync());

expectError(detectedWorkerFarm.getResult());
expectError(detectedWorkerFarm.isResult);

expectError(detectedWorkerFarm.setup());
expectError(detectedWorkerFarm.teardown());

expectError<Promise<void>>(detectedWorkerFarm.end());
expectType<Promise<{forceExited: boolean}>>(detectedWorkerFarm.end());

expectError<Promise<string>>(detectedWorkerFarm.getStderr());
expectType<NodeJS.ReadableStream>(detectedWorkerFarm.getStderr());

expectError<Promise<string>>(detectedWorkerFarm.getStdout());
expectType<NodeJS.ReadableStream>(detectedWorkerFarm.getStdout());

// typed JestWorkerFarm

declare const typedWorkerFarm: JestWorkerFarm<TestWorker>;

expectType<Promise<void>>(typedWorkerFarm.runTest());

expectError(typedWorkerFarm.isResult);
expectError(typedWorkerFarm.runTestAsync());

expectError(typedWorkerFarm.setup());
expectError(typedWorkerFarm.teardown());

expectError<Promise<void>>(typedWorkerFarm.end());
expectType<Promise<{forceExited: boolean}>>(typedWorkerFarm.end());

expectError<Promise<string>>(typedWorkerFarm.getStderr());
expectType<NodeJS.ReadableStream>(typedWorkerFarm.getStderr());

expectError<Promise<string>>(typedWorkerFarm.getStdout());
expectType<NodeJS.ReadableStream>(typedWorkerFarm.getStdout());
26 changes: 26 additions & 0 deletions packages/jest-worker/__typetests__/testWorker.ts
@@ -0,0 +1,26 @@
/**
* Copyright (c) Facebook, Inc. and its affiliates. All Rights Reserved.
*
* This source code is licensed under the MIT license found in the
* LICENSE file in the root directory of this source tree.
*/

export function runTest(): void {}
export async function runTestAsync(): Promise<void> {}

function getResult(): string {
return 'result';
}
export const isResult = true;

// reserved keys should be excluded from returned type

export function end(): void {}
export function getStderr(): string {
return 'get-err';
}
export function getStdout(): string {
return 'get-out';
}
export function setup(): void {}
export function teardown(): void {}
12 changes: 12 additions & 0 deletions packages/jest-worker/__typetests__/tsconfig.json
@@ -0,0 +1,12 @@
{
"extends": "../../../tsconfig.json",
"compilerOptions": {
"composite": false,
"noUnusedLocals": false,
"noUnusedParameters": false,
"skipLibCheck": true,

"types": ["node"]
},
"include": ["./**/*"]
}
2 changes: 2 additions & 0 deletions packages/jest-worker/package.json
Expand Up @@ -22,10 +22,12 @@
"supports-color": "^8.0.0"
},
"devDependencies": {
"@tsd/typescript": "~4.6.2",
"@types/merge-stream": "^1.1.2",
"@types/supports-color": "^8.1.0",
"get-stream": "^6.0.0",
"jest-leak-detector": "^28.0.1",
"tsd-lite": "^0.5.1",
"worker-farm": "^1.6.0"
},
"engines": {
Expand Down
6 changes: 3 additions & 3 deletions packages/jest-worker/src/Farm.ts
Expand Up @@ -9,20 +9,20 @@ import FifoQueue from './FifoQueue';
import {
CHILD_MESSAGE_CALL,
ChildMessage,
FarmOptions,
OnCustomMessage,
OnEnd,
OnStart,
PromiseWithCustomMessage,
QueueChildMessage,
TaskQueue,
WorkerCallback,
WorkerFarmOptions,
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This rename is not a breaking change, because the type was not exposed before.

WorkerInterface,
WorkerSchedulingPolicy,
} from './types';

export default class Farm {
private readonly _computeWorkerKey: FarmOptions['computeWorkerKey'];
private readonly _computeWorkerKey: WorkerFarmOptions['computeWorkerKey'];
private readonly _workerSchedulingPolicy: WorkerSchedulingPolicy;
private readonly _cacheKeys: Record<string, WorkerInterface> =
Object.create(null);
Expand All @@ -33,7 +33,7 @@ export default class Farm {
constructor(
private _numOfWorkers: number,
private _callback: WorkerCallback,
options: FarmOptions = {},
options: WorkerFarmOptions = {},
) {
this._computeWorkerKey = options.computeWorkerKey;
this._workerSchedulingPolicy =
Expand Down