initial implementation of global setup/teardown

- adds a generic "plugin loader" which handles root hooks, global setup/teardown, future
- renamed `validatePlugin` to `validateLegacyPlugin`. reporters/interfaces should eventually be refactored to use this plugin system
- should maybe decouple plugins from the `Mocha` prototype
- new events emitted by `Runner` for global setup/teardown
- need to add unit tests, need to support parallel mode

upgrade eslint-config-semistandard

more implementation, tests

added more tests, implementation. needs #4366 for browser compat

- removed dead code in `lib/cli/run-helpers.js`
- be sure not to send certain options to the worker processes
- fix root hook bug in worker
- send global fixtures to `Runner` constructor
- removed debugs
- **add `Process.unhandledRejection` listeners** which might be bad
- fix some tests while I'm fixing tests

reorganize plugin fixtures

add "to contain once" mocha result output assertion

(to ensure some output is not duplicated)

add runGlobalSetup, runGlobalTeardown to Mocha

revert changes to Runner

`Mocha` will handle global fixtures
no events will be emitted

remove test/integration/options/require.spec.js which is now in test/integration/plugins/

revert change to ParallelBufferedRunner which invoked global fixtures

pull helper code out of test/integration/options/watch.spec.js into integration test helpers

- because it's nice to be able to test watching elsewhere
- also renamed `runMochaWatch` to `runMochaWatchJSON` and provide `runMochaWatch` for plain output

move tests from test/integration/options/require.spec.js into test/integration/plugins/root-hooks.spec.js

Mocha#run now runs global fixtures; add watch mode support

- add `enableGlobalSetup()` and `enableGlobalTeardown()` methods on `Mocha`
- update `globalSetup()` and `globalTeardown()` methods
- watch mode runs global teardown upon `SIGINT`. is that right?

add command prop to runMochaJSON return value

if a fixture run with `runMochaJSON()` fails, the return value now contains a `command` prop (like the result of `runMocha()`) for easier debugging

remove unused garbage from RUnner

fix bug where mocha would never exit with a non-zero code

- I think this is understandable, because `done()` is weirdly _not_ an error-first callback
- remove try/catch guards around fixtures and let `process` uncaught/unhandled listeners deal w/ it
- also fix `unit/mocha.spec.js` test

update package-lock.json

move plugin.spec.js into node tests for now

remove unused 'watchify', add 'touch'

Signed-off-by: Christopher Hiller <boneskull@boneskull.com>

remove noisy debugs in Runner

fix flaky test in test/unit/mocha.spec.js

rename assertion types

- `JSONRunResult` -> `JSONResult`
- `RawRunResult` -> `SummarizedResult`
- `RawResult` remains the same
- tweak identification functions

integration test helper improvements

- better organization
- renamed stuff for consistency
  - `runMochaWatch` -> `runMochaWatchAsync`
  - `splitRegExp` -> `SPLIT_DOT_REPORTER_REGEXP`
  - rename parameters, add and fix docstrings
- modernize some of it
- add various typedefs to align with assertion types
- remove `invokeSubMocha` as it is no longer needed now that `mocha` will not fork a process if no `node` options are provided
- add special case for _forking_ mocha on win32 in watch mode, as that's the only way we can cleanup cleanly (parallel runs will need this treatment too, but I think it demands a more generalized solution)
- fix potential bug in `resolveFixturePath()`
- in `runMochaWatchAsync`, add `sleepMs` option; defaults to 2s
- extract `createTempDir()` from various test files and expose on helper
- prefer `rimraf` over `fs-extra.remove` since the former will retry when dir is locked on win32
- expose `touchFile()` on helper; do not use wonky handrolled touching algorithms and use [touch](https://npm.im/touch) instead
- update tests to use new helpers and renamed helpers

watch improvements and refactors

- print "waiting" msg to stderr (_should_ be ok)
- add "cleaning up" message upon ctrl-c
- remove needless `afterRun`  option from various places
- only run teardown fixtures if they exist
- add win32 fix for testing
- add `Mocha#hasGlobalSetupFixtures()` and `Mocha#hasGlobalTeardownFixtures()`

more tests

Signed-off-by: Christopher Hiller <boneskull@boneskull.com>

upgrade rewiremock

Signed-off-by: Christopher Hiller <boneskull@boneskull.com>

rename lib/plugin => lib/plugin-loader

- rename `createInvalidPluginError` => `createInvalidLegacyPluginError` and soft-deprecate
- add `createInvalidPluginDefinitionError` and `createInvalidPluginImplementationError` w/ constants; use them
- move some typedefs into `lib/mocha.js` as they are referenced via public aforementioned error factories
- remove TS docstrings
- better coverage
- move `plugin-loader` test back into `test/unit` (removed rewiremock from it)

lib/cli/run-helpers.js

@@ -12,10 +12,10 @@ const path = require('path');
 const debug = require('debug')('mocha:cli:run:helpers');
 const {watchRun, watchParallelRun} = require('./watch-run');
 const collectFiles = require('./collect-files');
-const {type} = require('../utils');
 const {format} = require('util');
-const {createInvalidPluginError, createUnsupportedError} = require('../errors');
+const {createInvalidLegacyPluginError} = require('../errors');
 const {requireOrImport} = require('../esm-utils');
+const PluginLoader = require('../plugin-loader');
 
 /**
  * Exits Mocha when tests + code under test has finished execution (default)
@@ -79,62 +79,31 @@ exports.list = str =>
  *
  * Returns array of `mochaHooks` exports, if any.
  * @param {string[]} requires - Modules to require
- * @returns {Promise<MochaRootHookObject|MochaRootHookFunction>} Any root hooks
+ * @returns {Promise<object>} Plugin implementations
  * @private
  */
 exports.handleRequires = async (requires = []) => {
-  const acc = [];
-  for (const mod of requires) {
+  const pluginLoader = PluginLoader.create();
+  for await (const mod of requires) {
     let modpath = mod;
     // this is relative to cwd
     if (fs.existsSync(mod) || fs.existsSync(`${mod}.js`)) {
       modpath = path.resolve(mod);
       debug('resolved required file %s to %s', mod, modpath);
     }
     const requiredModule = await requireOrImport(modpath);
-    if (
-      requiredModule &&
-      typeof requiredModule === 'object' &&
-      requiredModule.mochaHooks
-    ) {
-      const mochaHooksType = type(requiredModule.mochaHooks);
-      if (/function$/.test(mochaHooksType) || mochaHooksType === 'object') {
-        debug('found root hooks in required file %s', mod);
-        acc.push(requiredModule.mochaHooks);
-      } else {
-        throw createUnsupportedError(
-          'mochaHooks must be an object or a function returning (or fulfilling with) an object'
-        );
+    if (requiredModule && typeof requiredModule === 'object') {
+      if (pluginLoader.load(requiredModule)) {
+        debug('found one or more plugin implementations in %s', modpath);
       }
     }
     debug('loaded required module "%s"', mod);
   }
-  return acc;
-};
-
-/**
- * Loads root hooks as exported via `mochaHooks` from required files.
- * These can be sync/async functions returning objects, or just objects.
- * Flattens to a single object.
- * @param {Array<MochaRootHookObject|MochaRootHookFunction>} rootHooks - Array of root hooks
- * @private
- * @returns {MochaRootHookObject}
- */
-exports.loadRootHooks = async rootHooks => {
-  const rootHookObjects = await Promise.all(
-    rootHooks.map(async hook => (/function$/.test(type(hook)) ? hook() : hook))
-  );
-
-  return rootHookObjects.reduce(
-    (acc, hook) => {
-      acc.beforeAll = acc.beforeAll.concat(hook.beforeAll || []);
-      acc.beforeEach = acc.beforeEach.concat(hook.beforeEach || []);
-      acc.afterAll = acc.afterAll.concat(hook.afterAll || []);
-      acc.afterEach = acc.afterEach.concat(hook.afterEach || []);
-      return acc;
-    },
-    {beforeAll: [], beforeEach: [], afterAll: [], afterEach: []}
-  );
+  const plugins = await pluginLoader.finalize();
+  if (Object.keys(plugins).length) {
+    debug('finalized plugin implementations: %O', plugins);
+  }
+  return plugins;
 };
 
 /**
@@ -236,7 +205,7 @@ exports.runMocha = async (mocha, options) => {
  * name
  * @private
  */
-exports.validatePlugin = (opts, pluginType, map = {}) => {
+exports.validateLegacyPlugin = (opts, pluginType, map = {}) => {
   /**
    * This should be a unique identifier; either a string (present in `map`),
    * or a resolvable (via `require.resolve`) module ID/path.
@@ -245,14 +214,14 @@ exports.validatePlugin = (opts, pluginType, map = {}) => {
   const pluginId = opts[pluginType];
 
   if (Array.isArray(pluginId)) {
-    throw createInvalidPluginError(
+    throw createInvalidLegacyPluginError(
       `"--${pluginType}" can only be specified once`,
       pluginType
     );
   }
 
-  const unknownError = err =>
-    createInvalidPluginError(
+  const createUnknownError = err =>
+    createInvalidLegacyPluginError(
       format('Could not load %s "%s":\n\n %O', pluginType, pluginId, err),
       pluginType,
       pluginId
@@ -268,10 +237,10 @@ exports.validatePlugin = (opts, pluginType, map = {}) => {
         try {
           opts[pluginType] = require(path.resolve(pluginId));
         } catch (err) {
-          throw unknownError(err);
+          throw createUnknownError(err);
         }
       } else {
-        throw unknownError(err);
+        throw createUnknownError(err);
       }
     }
   }

lib/cli/run.js

@@ -19,8 +19,7 @@ const {
 const {
   list,
   handleRequires,
-  validatePlugin,
-  loadRootHooks,
+  validateLegacyPlugin,
   runMocha
 } = require('./run-helpers');
 const {ONE_AND_DONES, ONE_AND_DONE_ARGS} = require('./one-and-dones');
@@ -339,13 +338,10 @@ exports.builder = yargs =>
       // currently a failing middleware does not work nicely with yargs' `fail()`.
       try {
         // load requires first, because it can impact "plugin" validation
-        const rawRootHooks = await handleRequires(argv.require);
-        validatePlugin(argv, 'reporter', Mocha.reporters);
-        validatePlugin(argv, 'ui', Mocha.interfaces);
-
-        if (rawRootHooks && rawRootHooks.length) {
-          argv.rootHooks = await loadRootHooks(rawRootHooks);
-        }
+        const plugins = await handleRequires(argv.require);
+        validateLegacyPlugin(argv, 'reporter', Mocha.reporters);
+        validateLegacyPlugin(argv, 'ui', Mocha.interfaces);
+        Object.assign(argv, plugins);
       } catch (err) {
         // this could be a bad --require, bad reporter, ui, etc.
         console.error(`\n${symbols.error} ${ansi.red('ERROR:')}`, err);

lib/cli/watch-run.js

@@ -1,5 +1,6 @@
 'use strict';
 
+const logSymbols = require('log-symbols');
 const debug = require('debug')('mocha:cli:watch');
 const path = require('path');
 const chokidar = require('chokidar');
@@ -32,6 +33,7 @@ exports.watchParallelRun = (
   fileCollectParams
 ) => {
   debug('creating parallel watcher');
+
   return createWatcher(mocha, {
     watchFiles,
     watchIgnore,
@@ -68,9 +70,6 @@ exports.watchParallelRun = (
       newMocha.lazyLoadFiles(true);
       return newMocha;
     },
-    afterRun({watcher}) {
-      blastCache(watcher);
-    },
     fileCollectParams
   });
 };
@@ -91,7 +90,6 @@ exports.watchParallelRun = (
  */
 exports.watchRun = (mocha, {watchFiles, watchIgnore}, fileCollectParams) => {
   debug('creating serial watcher');
-  // list of all test files
 
   return createWatcher(mocha, {
     watchFiles,
@@ -128,9 +126,6 @@ exports.watchRun = (mocha, {watchFiles, watchIgnore}, fileCollectParams) => {
 
       return newMocha;
     },
-    afterRun({watcher}) {
-      blastCache(watcher);
-    },
     fileCollectParams
   });
 };
@@ -141,7 +136,6 @@ exports.watchRun = (mocha, {watchFiles, watchIgnore}, fileCollectParams) => {
  * @param {Object} opts
  * @param {BeforeWatchRun} [opts.beforeRun] - Function to call before
  * `mocha.run()`
- * @param {AfterWatchRun} [opts.afterRun] - Function to call after `mocha.run()`
  * @param {string[]} [opts.watchFiles] - List of paths and patterns to watch. If
  *   not provided all files with an extension included in
  *   `fileCollectionParams.extension` are watched. See first argument of
@@ -155,25 +149,32 @@ exports.watchRun = (mocha, {watchFiles, watchIgnore}, fileCollectParams) => {
  */
 const createWatcher = (
   mocha,
-  {watchFiles, watchIgnore, beforeRun, afterRun, fileCollectParams}
+  {watchFiles, watchIgnore, beforeRun, fileCollectParams}
 ) => {
   if (!watchFiles) {
     watchFiles = fileCollectParams.extension.map(ext => `**/*.${ext}`);
   }
 
   debug('ignoring files matching: %s', watchIgnore);
+  let globalFixtureContext;
+
+  // we handle global fixtures manually
+  mocha.enableGlobalSetup(false).enableGlobalTeardown(false);
 
   const watcher = chokidar.watch(watchFiles, {
     ignored: watchIgnore,
     ignoreInitial: true
   });
 
   const rerunner = createRerunner(mocha, watcher, {
-    beforeRun,
-    afterRun
+    beforeRun
   });
 
-  watcher.on('ready', () => {
+  watcher.on('ready', async () => {
+    if (!globalFixtureContext) {
+      debug('triggering global setup');
+      globalFixtureContext = await mocha.runGlobalSetup();
+    }
     rerunner.run();
   });
 
@@ -185,10 +186,39 @@ const createWatcher = (
   process.on('exit', () => {
     showCursor();
   });
-  process.on('SIGINT', () => {
+
+  // this is for testing.
+  // win32 cannot gracefully shutdown via a signal from a parent
+  // process; a `SIGINT` from a parent will cause the process
+  // to immediately exit.  during normal course of operation, a user
+  // will type Ctrl-C and the listener will be invoked, but this
+  // is not possible in automated testing.
+  // there may be another way to solve this, but it too will be a hack.
+  // for our watch tests on win32 we must _fork_ mocha with an IPC channel
+  if (process.connected) {
+    process.on('message', msg => {
+      if (msg === 'SIGINT') {
+        process.emit('SIGINT');
+      }
+    });
+  }
+
+  let exiting = false;
+  process.on('SIGINT', async () => {
     showCursor();
-    console.log('\n');
-    process.exit(128 + 2);
+    console.error(`${logSymbols.warning} [mocha] cleaning up, please wait...`);
+    if (!exiting) {
+      exiting = true;
+      if (mocha.hasGlobalTeardownFixtures()) {
+        debug('running global teardown');
+        try {
+          await mocha.runGlobalTeardown(globalFixtureContext);
+        } catch (err) {
+          console.error(err);
+        }
+      }
+      process.exit(130);
+    }
   });
 
   // Keyboard shortcut for restarting when "rs\n" is typed (ala Nodemon)
@@ -212,12 +242,11 @@ const createWatcher = (
  * @param {FSWatcher} watcher - chokidar `FSWatcher` instance
  * @param {Object} [opts] - Options!
  * @param {BeforeWatchRun} [opts.beforeRun] - Function to call before `mocha.run()`
- * @param {AfterWatchRun} [opts.afterRun] - Function to call after `mocha.run()`
  * @returns {Rerunner}
  * @ignore
  * @private
  */
-const createRerunner = (mocha, watcher, {beforeRun, afterRun} = {}) => {
+const createRerunner = (mocha, watcher, {beforeRun} = {}) => {
   // Set to a `Runner` when mocha is running. Set to `null` when mocha is not
   // running.
   let runner = null;
@@ -226,16 +255,15 @@ const createRerunner = (mocha, watcher, {beforeRun, afterRun} = {}) => {
   let rerunScheduled = false;
 
   const run = () => {
-    mocha = beforeRun ? beforeRun({mocha, watcher}) : mocha;
-
+    mocha = beforeRun ? beforeRun({mocha, watcher}) || mocha : mocha;
     runner = mocha.run(() => {
       debug('finished watch run');
       runner = null;
-      afterRun && afterRun({mocha, watcher});
+      blastCache(watcher);
       if (rerunScheduled) {
         rerun();
       } else {
-        debug('waiting for changes...');
+        console.error(`${logSymbols.info} [mocha] waiting for changes...`);
       }
     });
   };
@@ -333,15 +361,6 @@ const blastCache = watcher => {
  * @returns {Mocha}
  */
 
-/**
- * Callback to be run after `mocha.run()` completes.  Typically used to clear
- * require cache.
- * @callback AfterWatchRun
- * @private
- * @param {{mocha: Mocha, watcher: FSWatcher}} options
- * @returns {void}
- */
-
 /**
  * Object containing run control methods
  * @typedef {Object} Rerunner