Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(jest-cli, jest-config, @jest/core, jest-haste-map, @jest/reporters, jest-runner, jest-runtime, @jest/types): add workerThreads configuration option #13939

Merged
Merged
1 change: 1 addition & 0 deletions CHANGELOG.md
Expand Up @@ -2,6 +2,7 @@

### Features

- `[jest-cli, jest-config, @jest/core, jest-haste-map, @jest/reporters, jest-runner, jest-runtime, @jest/types]` Add `workerThreads` configuration option to allow using [worker threads](https://nodejs.org/dist/latest/docs/api/worker_threads.html) for parallelization ([#13939](https://github.com/facebook/jest/pull/13939))
- `[jest-worker]` Add `start` method to worker farms ([#13937](https://github.com/facebook/jest/pull/13937))

### Fixes
Expand Down
4 changes: 4 additions & 0 deletions docs/CLI.md
Expand Up @@ -506,3 +506,7 @@ In most CI environments, this is automatically handled for you.
### `--watchman`

Whether to use [`watchman`](https://facebook.github.io/watchman/) for file crawling. Defaults to `true`. Disable using `--no-watchman`.

### `--workerThreads`

Whether to use [worker threads](https://nodejs.org/dist/latest/docs/api/worker_threads.html) for parallelization. [Child processes](https://nodejs.org/dist/latest/docs/api/child_process.html) are used by default.
4 changes: 4 additions & 0 deletions docs/Configuration.md
Expand Up @@ -2375,3 +2375,7 @@ This option allows comments in `package.json`. Include the comment text as the v
}
}
```

### `workerThreads`
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alternatively could be named: threads, useThreads, useWorkerThreads. Or it could implemented similar to coverageProvider or testRunner, e.g. parallelization: 'childProcesses' | 'workerThreads'.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is fine. there won't ever be any other alternatives


Whether to use [worker threads](https://nodejs.org/dist/latest/docs/api/worker_threads.html) for parallelization. [Child processes](https://nodejs.org/dist/latest/docs/api/child_process.html) are used by default.
6 changes: 6 additions & 0 deletions packages/jest-cli/src/args.ts
Expand Up @@ -709,4 +709,10 @@ export const options: {[key: string]: Options} = {
'--no-watchman.',
type: 'boolean',
},
workerThreads: {
description:
'Whether to use worker threads for parallelization. Child processes ' +
'are used by default.',
type: 'boolean',
},
};
1 change: 1 addition & 0 deletions packages/jest-config/src/ValidConfig.ts
Expand Up @@ -186,6 +186,7 @@ export const initialOptions: Config.InitialOptions = {
],
watchman: true,
workerIdleMemoryLimit: multipleValidOptions(0.2, '50%'),
workerThreads: true,
};

export const initialProjectOptions: Config.InitialProjectOptions = {
Expand Down
1 change: 1 addition & 0 deletions packages/jest-config/src/index.ts
Expand Up @@ -140,6 +140,7 @@ const groupOptions = (
watchPlugins: options.watchPlugins,
watchman: options.watchman,
workerIdleMemoryLimit: options.workerIdleMemoryLimit,
workerThreads: options.workerThreads,
}),
projectConfig: Object.freeze({
automock: options.automock,
Expand Down
1 change: 1 addition & 0 deletions packages/jest-config/src/normalize.ts
Expand Up @@ -936,6 +936,7 @@ export default async function normalize(
case 'watch':
case 'watchAll':
case 'watchman':
case 'workerThreads':
value = oldOptions[key];
break;
case 'workerIdleMemoryLimit':
Expand Down
1 change: 1 addition & 0 deletions packages/jest-core/src/cli/index.ts
Expand Up @@ -157,6 +157,7 @@ const buildContextsAndHasteMaps = async (
resetCache: !config.cache,
watch: globalConfig.watch || globalConfig.watchAll,
watchman: globalConfig.watchman,
workerThreads: globalConfig.workerThreads,
});
hasteMapInstances[index] = hasteMapInstance;
return createContext(config, await hasteMapInstance.build());
Expand Down
4 changes: 4 additions & 0 deletions packages/jest-haste-map/src/index.ts
Expand Up @@ -79,6 +79,7 @@ type Options = {
throwOnModuleCollision?: boolean;
useWatchman?: boolean;
watch?: boolean;
workerThreads?: boolean;
};

type InternalOptions = {
Expand All @@ -103,6 +104,7 @@ type InternalOptions = {
throwOnModuleCollision: boolean;
useWatchman: boolean;
watch: boolean;
workerThreads?: boolean;
};

type Watcher = {
Expand Down Expand Up @@ -267,6 +269,7 @@ class HasteMap extends EventEmitter implements IHasteMap {
throwOnModuleCollision: !!options.throwOnModuleCollision,
useWatchman: options.useWatchman ?? true,
watch: !!options.watch,
workerThreads: options.workerThreads,
};
this._console = options.console || globalThis.console;

Expand Down Expand Up @@ -748,6 +751,7 @@ class HasteMap extends EventEmitter implements IHasteMap {
this._worker = {getSha1, worker};
} else {
this._worker = new Worker(require.resolve('./worker'), {
enableWorkerThreads: this._options.workerThreads,
exposedMethods: ['getSha1', 'worker'],
forkOptions: {serialization: 'json'},
maxRetries: 3,
Expand Down
1 change: 1 addition & 0 deletions packages/jest-reporters/src/CoverageReporter.ts
Expand Up @@ -147,6 +147,7 @@ export default class CoverageReporter extends BaseReporter {
worker = require('./CoverageWorker');
} else {
worker = new Worker(require.resolve('./CoverageWorker'), {
enableWorkerThreads: this._globalConfig.workerThreads,
exposedMethods: ['worker'],
forkOptions: {serialization: 'json'},
maxRetries: 2,
Expand Down
1 change: 1 addition & 0 deletions packages/jest-runner/src/index.ts
Expand Up @@ -105,6 +105,7 @@ export default class TestRunner extends EmittingTestRunner {
}

const worker = new Worker(require.resolve('./testWorker'), {
enableWorkerThreads: this._globalConfig.workerThreads,
exposedMethods: ['worker'],
forkOptions: {serialization: 'json', stdio: 'pipe'},
// The workerIdleMemoryLimit should've been converted to a number during
Expand Down
2 changes: 2 additions & 0 deletions packages/jest-runtime/src/index.ts
Expand Up @@ -79,6 +79,7 @@ type HasteMapOptions = {
resetCache: boolean;
watch?: boolean;
watchman: boolean;
workerThreads?: boolean;
};

interface InternalModuleOptions extends Required<CallerTransformOptions> {
Expand Down Expand Up @@ -370,6 +371,7 @@ export default class Runtime {
throwOnModuleCollision: config.haste.throwOnModuleCollision,
useWatchman: options?.watchman,
watch: options?.watch,
workerThreads: options?.workerThreads,
});
}

Expand Down
3 changes: 3 additions & 0 deletions packages/jest-types/src/Config.ts
Expand Up @@ -326,6 +326,7 @@ export type InitialOptions = Partial<{
watchman: boolean;
watchPlugins: Array<string | [string, Record<string, unknown>]>;
workerIdleMemoryLimit: number | string;
workerThreads: boolean;
}>;

export type SnapshotUpdateState = 'all' | 'new' | 'none';
Expand Down Expand Up @@ -419,6 +420,7 @@ export type GlobalConfig = {
config: Record<string, unknown>;
}> | null;
workerIdleMemoryLimit?: number;
workerThreads?: boolean;
mrazauskas marked this conversation as resolved.
Show resolved Hide resolved
};

export type ProjectConfig = {
Expand Down Expand Up @@ -574,5 +576,6 @@ export type Argv = Arguments<
watchman: boolean;
watchPathIgnorePatterns: Array<string>;
workerIdleMemoryLimit: number | string;
workerThreads: boolean;
}>
>;