Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improved watcher hooks #3841

Merged
merged 8 commits into from Oct 31, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
19 changes: 16 additions & 3 deletions docs/05-plugin-development.md
Expand Up @@ -72,7 +72,9 @@ To interact with the build process, your plugin object includes 'hooks'. Hooks a
* `sequential`: If several plugins implement this hook, all of them will be run in the specified plugin order. If a hook is async, subsequent hooks of this kind will wait until the current hook is resolved.
* `parallel`: If several plugins implement this hook, all of them will be run in the specified plugin order. If a hook is async, subsequent hooks of this kind will be run in parallel and not wait for the current hook.

Build hooks are run during the build phase, which is triggered by `rollup.rollup(inputOptions)`. They are mainly concerned with locating, providing and transforming input files before they are processed by Rollup. The first hook of the build phase is [options](guide/en/#options), the last one is always [buildEnd](guide/en/#buildend). Additionally in watch mode, the [watchChange](guide/en/#watchchange) hook can be triggered at any time to notify a new run will be triggered once the current run has generated its outputs.
Build hooks are run during the build phase, which is triggered by `rollup.rollup(inputOptions)`. They are mainly concerned with locating, providing and transforming input files before they are processed by Rollup. The first hook of the build phase is [options](guide/en/#options), the last one is always [buildEnd](guide/en/#buildend).

Additionally, in watch mode the [watchChange](guide/en/#watchchange) hook can be triggered at any time to notify a new run will be triggered once the current run has generated its outputs. Also, when watcher closes, the [closeWatcher](guide/en/#closewatcher) hook will be triggered.

See [Output Generation Hooks](guide/en/#output-generation-hooks) for hooks that run during the output generation phase to modify the generated output.

Expand All @@ -92,6 +94,13 @@ Next Hook: [`resolveId`](guide/en/#resolveid) to resolve each entry point in par

Called on each `rollup.rollup` build. This is the recommended hook to use when you need access to the options passed to `rollup.rollup()` as it takes the transformations by all [`options`](guide/en/#options) hooks into account and also contains the right default values for unset options.

#### `closeWatcher`
Type: `() => void`<br>
Kind: `sync, sequential`<br>
Previous/Next Hook: This hook can be triggered at any time both during the build and the output generation phases. If that is the case, the current build will still proceed but no new [`watchChange`](guide/en/#watchChange) events will be triggered ever.

Notifies a plugin when watcher process closes and all open resources should be closed too. This hook cannot be used by output plugins.

#### `load`
Type: `(id: string) => string | null | {code: string, map?: string | SourceMap, ast? : ESTree.Program, moduleSideEffects?: boolean | "no-treeshake" | null, syntheticNamedExports?: boolean | string | null, meta?: {[plugin: string]: any} | null}`<br>
Kind: `async, first`<br>
Expand Down Expand Up @@ -236,11 +245,11 @@ See [custom module meta-data](guide/en/#custom-module-meta-data) for how to use
You can use [`this.getModuleInfo`](guide/en/#thisgetmoduleinfomoduleid-string--moduleinfo--null) to find out the previous values of `moduleSideEffects`, `syntheticNamedExports` and `meta` inside this hook.

#### `watchChange`
Type: `(id: string) => void`<br>
Type: `watchChange: (id: string, change: {event: 'create' | 'update' | 'delete'}) => void`<br>
Kind: `sync, sequential`<br>
Previous/Next Hook: This hook can be triggered at any time both during the build and the output generation phases. If that is the case, the current build will still proceed but a new build will be scheduled to start once the current build has completed, starting again with [`options`](guide/en/#options).

Notifies a plugin whenever rollup has detected a change to a monitored file in `--watch` mode. This hook cannot be used by output plugins.
Notifies a plugin whenever rollup has detected a change to a monitored file in `--watch` mode. This hook cannot be used by output plugins. Second argument contains additional details of change event.

### Output Generation Hooks

Expand Down Expand Up @@ -655,6 +664,10 @@ During the build, this object represents currently available information about t

Returns `null` if the module id cannot be found.

#### `this.getWatchFiles() => string[]`

Get ids of the files which has been watched previously. Include both files added by plugins with `this.addWatchFile` and files added implicitly by rollup during the build.

#### `this.meta: {rollupVersion: string, watchMode: boolean}`

An object containing potentially useful Rollup metadata:
Expand Down
8 changes: 6 additions & 2 deletions src/Graph.ts
Expand Up @@ -10,7 +10,8 @@ import {
NormalizedInputOptions,
RollupCache,
RollupWatcher,
SerializablePluginCache
SerializablePluginCache,
WatchChangeHook
} from './rollup/types';
import { BuildPhase } from './utils/buildPhase';
import { errImplicitDependantIsNotIncluded, error } from './utils/error';
Expand Down Expand Up @@ -84,10 +85,13 @@ export default class Graph {

if (watcher) {
this.watchMode = true;
const handleChange = (id: string) => this.pluginDriver.hookSeqSync('watchChange', [id]);
const handleChange: WatchChangeHook = (...args) => this.pluginDriver.hookSeqSync('watchChange', args);
const handleClose = () => this.pluginDriver.hookSeqSync('closeWatcher', []);
watcher.on('change', handleChange);
watcher.on('close', handleClose);
watcher.once('restart', () => {
watcher.removeListener('change', handleChange);
watcher.removeListener('close', handleClose);
});
}
this.pluginDriver = new PluginDriver(this, options, options.plugins, this.pluginCache);
Expand Down
19 changes: 13 additions & 6 deletions src/rollup/types.d.ts
Expand Up @@ -195,6 +195,7 @@ export interface PluginContext extends MinimalPluginContext {
getFileName: (fileReferenceId: string) => string;
getModuleIds: () => IterableIterator<string>;
getModuleInfo: GetModuleInfo;
getWatchFiles: () => string[];
/** @deprecated Use `this.resolve` instead */
isExternal: IsExternal;
/** @deprecated Use `this.getModuleIds` instead */
Expand Down Expand Up @@ -318,6 +319,9 @@ export type ResolveFileUrlHook = (
export type AddonHookFunction = (this: PluginContext) => string | Promise<string>;
export type AddonHook = string | AddonHookFunction;

export type ChangeEvent = 'create' | 'update' | 'delete'
export type WatchChangeHook = (this: PluginContext, id: string, change: {event: ChangeEvent}) => void

/**
* use this type for plugin annotation
* @example
Expand Down Expand Up @@ -345,6 +349,7 @@ export interface OutputBundleWithPlaceholders {
export interface PluginHooks extends OutputPluginHooks {
buildEnd: (this: PluginContext, err?: Error) => Promise<void> | void;
buildStart: (this: PluginContext, options: NormalizedInputOptions) => Promise<void> | void;
closeWatcher: (this: PluginContext) => void;
load: LoadHook;
moduleParsed: ModuleParsedHook;
options: (
Expand All @@ -354,7 +359,7 @@ export interface PluginHooks extends OutputPluginHooks {
resolveDynamicImport: ResolveDynamicImportHook;
resolveId: ResolveIdHook;
transform: TransformHook;
watchChange: (id: string) => void;
watchChange: WatchChangeHook;
}

interface OutputPluginHooks {
Expand Down Expand Up @@ -423,6 +428,7 @@ export type FirstPluginHooks =

export type SequentialPluginHooks =
| 'augmentChunkHash'
| 'closeWatcher'
| 'generateBundle'
| 'options'
| 'outputOptions'
Expand Down Expand Up @@ -787,9 +793,9 @@ export interface RollupWatchOptions extends InputOptions {
watch?: WatcherOptions | false;
}

interface TypedEventEmitter<T> {
interface TypedEventEmitter<T extends {[event: string]: (...args: any) => any}> {
addListener<K extends keyof T>(event: K, listener: T[K]): this;
emit<K extends keyof T>(event: K, ...args: any[]): boolean;
emit<K extends keyof T>(event: K, ...args: Parameters<T[K]>): boolean;
lukastaegert marked this conversation as resolved.
Show resolved Hide resolved
eventNames(): Array<keyof T>;
getMaxListeners(): number;
listenerCount(type: keyof T): number;
Expand All @@ -807,11 +813,11 @@ interface TypedEventEmitter<T> {

export type RollupWatcherEvent =
| { code: 'START' }
| { code: 'BUNDLE_START'; input: InputOption; output: readonly string[] }
| { code: 'BUNDLE_START'; input?: InputOption; output: readonly string[] }
| {
code: 'BUNDLE_END';
duration: number;
input: InputOption;
input?: InputOption;
output: readonly string[];
result: RollupBuild;
}
Expand All @@ -820,7 +826,8 @@ export type RollupWatcherEvent =

export interface RollupWatcher
extends TypedEventEmitter<{
change: (id: string) => void;
change: (id: string, change: {event: ChangeEvent}) => void;
close: () => void;
event: (event: RollupWatcherEvent) => void;
restart: () => void;
}> {
Expand Down
1 change: 1 addition & 0 deletions src/utils/PluginContext.ts
Expand Up @@ -126,6 +126,7 @@ export function getPluginContexts(
getFileName: fileEmitter.getFileName,
getModuleIds: () => graph.modulesById.keys(),
getModuleInfo: graph.getModuleInfo,
getWatchFiles: () => Object.keys(graph.watchFiles),
isExternal: getDeprecatedContextHandler(
(id: string, parentId: string | undefined, isResolved = false) =>
options.external(id, parentId, isResolved),
Expand Down
1 change: 1 addition & 0 deletions src/utils/PluginDriver.ts
Expand Up @@ -47,6 +47,7 @@ const inputHookNames: {
} = {
buildEnd: 1,
buildStart: 1,
closeWatcher: 1,
load: 1,
moduleParsed: 1,
options: 1,
Expand Down
12 changes: 6 additions & 6 deletions src/watch/fileWatcher.ts
@@ -1,6 +1,6 @@
import chokidar, { FSWatcher } from 'chokidar';
import { platform } from 'os';
import { ChokidarOptions } from '../rollup/types';
import { ChangeEvent, ChokidarOptions } from '../rollup/types';
import { Task } from './watch';

export class FileWatcher {
Expand Down Expand Up @@ -45,7 +45,7 @@ export class FileWatcher {
const task = this.task;
const isLinux = platform() === 'linux';
const isTransformDependency = transformWatcherId !== null;
const handleChange = (id: string) => {
const handleChange = (id: string, event: ChangeEvent) => {
const changedId = transformWatcherId || id;
if (isLinux) {
// unwatching and watching fixes an issue with chokidar where on certain systems,
Expand All @@ -54,13 +54,13 @@ export class FileWatcher {
watcher.unwatch(changedId);
watcher.add(changedId);
}
task.invalidate(changedId, isTransformDependency);
task.invalidate(changedId, {isTransformDependency, event});
};
const watcher = chokidar
.watch([], this.chokidarOptions)
.on('add', handleChange)
.on('change', handleChange)
.on('unlink', handleChange);
.on('add', id => handleChange(id, 'create'))
.on('change', id => handleChange(id, 'update'))
.on('unlink', id => handleChange(id, 'delete'));
return watcher;
}
}
64 changes: 46 additions & 18 deletions src/watch/watch.ts
Expand Up @@ -2,6 +2,7 @@ import * as path from 'path';
import createFilter from 'rollup-pluginutils/src/createFilter';
import { rollupInternal } from '../rollup/rollup';
import {
ChangeEvent,
MergedRollupOptions,
OutputOptions,
RollupBuild,
Expand All @@ -13,12 +14,30 @@ import { mergeOptions } from '../utils/options/mergeOptions';
import { GenericConfigObject } from '../utils/options/options';
import { FileWatcher } from './fileWatcher';

const eventsRewrites: Record<ChangeEvent, Record<ChangeEvent, ChangeEvent | 'buggy' | null>> = {
create: {
create: 'buggy',
delete: null, //delete file from map
update: 'create',
},
delete: {
create: 'update',
delete: 'buggy',
update: 'buggy',
},
update: {
create: 'buggy',
delete: 'delete',
update: 'update',
}
}

export class Watcher {
emitter: RollupWatcher;

private buildDelay = 0;
private buildTimeout: NodeJS.Timer | null = null;
private invalidatedIds: Set<string> = new Set();
private invalidatedIds: Map<string, ChangeEvent> = new Map();
private rerun = false;
private running: boolean;
private tasks: Task[];
Expand All @@ -43,16 +62,25 @@ export class Watcher {
for (const task of this.tasks) {
task.close();
}
this.emitter.emit('close');
this.emitter.removeAllListeners();
}

emit(event: string, value?: any) {
lukastaegert marked this conversation as resolved.
Show resolved Hide resolved
this.emitter.emit(event as any, value);
}
invalidate(file?: {event: ChangeEvent, id: string}) {
if (file) {
const prevEvent = this.invalidatedIds.get(file.id);
const event = prevEvent
? eventsRewrites[prevEvent][file.event]
: file.event;

invalidate(id?: string) {
if (id) {
this.invalidatedIds.add(id);
if (event === 'buggy') {
//TODO: throws or warn? Currently just ignore, uses new event
this.invalidatedIds.set(file.id, file.event);
} else if (event === null) {
this.invalidatedIds.delete(file.id);
} else {
this.invalidatedIds.set(file.id, event);
}
}
if (this.running) {
this.rerun = true;
Expand All @@ -63,19 +91,19 @@ export class Watcher {

this.buildTimeout = setTimeout(() => {
this.buildTimeout = null;
for (const id of this.invalidatedIds) {
this.emit('change', id);
for (const [id, event] of this.invalidatedIds.entries()) {
this.emitter.emit('change', id, {event});
}
this.invalidatedIds.clear();
this.emit('restart');
this.emitter.emit('restart');
this.run();
}, this.buildDelay);
}

private async run() {
this.running = true;

this.emit('event', {
this.emitter.emit('event', {
code: 'START'
});

Expand All @@ -84,12 +112,12 @@ export class Watcher {
await task.run();
}
this.running = false;
this.emit('event', {
this.emitter.emit('event', {
code: 'END'
});
} catch (error) {
this.running = false;
this.emit('event', {
this.emitter.emit('event', {
code: 'ERROR',
error
});
Expand Down Expand Up @@ -144,16 +172,16 @@ export class Task {
this.fileWatcher.close();
}

invalidate(id: string, isTransformDependency: boolean | undefined) {
invalidate(id: string, details: {event: ChangeEvent, isTransformDependency?: boolean}) {
this.invalidated = true;
if (isTransformDependency) {
if (details.isTransformDependency) {
for (const module of this.cache.modules) {
if (module.transformDependencies.indexOf(id) === -1) continue;
// effective invalidation
module.originalCode = null as any;
}
}
this.watcher.invalidate(id);
this.watcher.invalidate({id, event: details.event});
}

async run() {
Expand All @@ -167,7 +195,7 @@ export class Task {

const start = Date.now();

this.watcher.emit('event', {
this.watcher.emitter.emit('event', {
code: 'BUNDLE_START',
input: this.options.input,
output: this.outputFiles
Expand All @@ -180,7 +208,7 @@ export class Task {
}
this.updateWatchedFiles(result);
this.skipWrite || (await Promise.all(this.outputs.map(output => result.write(output))));
this.watcher.emit('event', {
this.watcher.emitter.emit('event', {
code: 'BUNDLE_END',
duration: Date.now() - start,
input: this.options.input,
Expand Down