From 78c90afa2c4da20f14d34739e08fc59e40459f42 Mon Sep 17 00:00:00 2001 From: Jason Kurian Date: Sat, 27 Apr 2019 23:29:23 -0400 Subject: [PATCH 01/12] docs(readme): reorganize and optimize for configuration options [skip ci] remove outdated docs and point more to the preset configs --- README.md | 267 ++++++++++++++++-------------------------------------- 1 file changed, 76 insertions(+), 191 deletions(-) diff --git a/README.md b/README.md index cd3fc5bf9..1d9e41fe8 100644 --- a/README.md +++ b/README.md @@ -52,148 +52,114 @@ and a `text-lcov` coverage report. nyc --reporter=lcov --reporter=text-lcov npm test ``` -### Accurate stack traces using source-maps - -When `produce-source-map` is set to true, then the instrumented source files will -include inline source maps for the instrumenter transform. When combined with -[source-map-support](https://github.com/evanw/node-source-map-support), -stack traces for instrumented code will reflect their original lines. - -### Support for custom require hooks (babel, typescript, etc.) - -nyc supports custom require hooks like [`@babel/register`]. nyc can load -the hooks for you, [using the `--require` flag](#require-additional-modules). - -Source maps are used to map coverage information back to the appropriate lines -of the pre-transpiled code. You'll have to configure your custom require hook -to inline the source-map in the transpiled code. For Babel that means setting -the `sourceMaps` option to `inline`. - -### Source-Map support for pre-instrumented codebases - -If you opt to pre-instrument your source-code (rather than using a just-in-time -transpiler like [`@babel/register`]) nyc supports both inline source-maps and -`.map` files. - -_Important: If you are using nyc with a project that pre-instruments its code, -run nyc with the configuration option `--exclude-after-remap` set to `false`. -Otherwise nyc's reports will exclude any files that source-maps remap to folders -covered under exclude rules._ - -## Use with `babel-plugin-istanbul` for Babel Support - -We recommend using [`babel-plugin-istanbul`] if your project uses the babel tool chain: - -1. enable the `babel-plugin-istanbul` plugin: - - ```json - { - "babel": { - "presets": ["@babel/preset-env"], - "env": { - "test": { - "plugins": ["istanbul"] - } - } - } - } - ``` - - Note: With this configuration, the Istanbul instrumentation will only be active when `NODE_ENV` or `BABEL_ENV` is `test` unless the environment is a valid entry in `"env"` within the `.babelrc` file. - - We recommend using the [`cross-env`](https://npmjs.com/package/cross-env) package to set these environment variables - in your `package.json` scripts in a way that works cross-platform. - -2. disable nyc's instrumentation and source-maps, e.g. in `package.json`: - - ```json - { - "nyc": { - "require": [ - "@babel/register" - ], - "sourceMap": false, - "instrument": false - }, - "scripts": { - "test": "cross-env NODE_ENV=test nyc mocha" - } - } - ``` +## Configuring `nyc` -That's all there is to it, better ES2015+ syntax highlighting awaits: +### Babel projects - +Please start with the pre-configured [@istanbuljs/nyc-config-babel preset](https://www.npmjs.com/package/@istanbuljs/nyc-config-babel). +You can add your custom configuration options as shown below. -## Support for alternate file extensions (.jsx, .mjs) +### TypeScript projects -Supporting file extensions can be configured through either the configuration arguments or with the `nyc` config section in `package.json`. +Please start with the pre-configured [@istanbuljs/nyc-config-typescript preset](https://www.npmjs.com/package/@istanbuljs/nyc-config-typescript). -```shell -nyc --extension .jsx --extension .mjs npm test -``` +#### Adding your overrides ```json { "nyc": { - "extension": [ - ".jsx", - ".mjs" - ] + "extends": "@istanbuljs/nyc-config-typescript", + "all": true } } ``` +### Configuration files -## Checking coverage +Any configuration options that can be set via the command line can also be specified in the `nyc` stanza of your package.json, or within a seperate configuration file - a variety of flavors are available: -nyc can fail tests if coverage falls below a threshold. -After running your tests with nyc, simply run: +| File name | File Association | +|-----------------|------------------| +| `.nycrc` | JSON | +| `.nycrc.json` | JSON | +| `.nycrc.yaml` | YAML | +| `.nycrc.yml` | YAML | +| `nyc.config.js` | CommonJS export | -```shell -nyc check-coverage --lines 95 --functions 95 --branches 95 -``` +### Common Configuration Options -nyc also accepts a `--check-coverage` shorthand, which can be used to -both run tests and check that coverage falls within the threshold provided: +See `npx nyc --help` for all options available. +You can set these in any of the files listed above, or from the command line. +This table is a quick TLDR for the rest of this readme and there are more advanced docs available. -```shell -nyc --check-coverage --lines 100 npm test -``` +| Option name | Description | Type | Default | +| ----------- | ----------- | ---- | ------- | +| `all` | whether or not to instrument all files (not just the ones touched by your test suite) | `Boolean` | `false` | +| `check-coverage` | check whether coverage is within thresholds, fail if not | `Boolean` | `false` | +| `extension` | a list of extensions that nyc should attempt to handle in addition to `.js` | `Array` | `['.js']` | +| `include` | [globs of files to be included from instrumentation](#selecting-files-for-coverage) | `Array` | `['**']`| +| `exclude` | See [selecting files for coverage for more info](#selecting-files-for-coverage) | `Array` | [list](https://github.com/istanbuljs/istanbuljs/blob/master/packages/test-exclude/index.js#L176-L184) | +| `reporter` | [coverage reporters to use](https://istanbul.js.org/docs/advanced/alternative-reporters/) | `Array` | `['text']` | +| `report-dir` | where to put the coverage report files | `String` | `./coverage` | +| `temp-dir` | directory to output raw coverage information to | `String` | `./.nyc_output` | -The above check fails if coverage falls below 100%. +Configuration can also be provided by `nyc.config.js` if programmed logic is required: +```js +'use strict'; +const {defaultExclude} = require('test-exclude'); +const isWindows = require('is-windows'); -To check thresholds on a per-file basis run: +let platformExclude = [ + isWindows() ? 'lib/posix.js' : 'lib/win32.js' +]; -```shell -nyc check-coverage --lines 95 --per-file +module.exports = { + exclude: platformExclude.concat(defaultExclude) +}; ``` -## Running reports +### Publish and reuse your nyc configuration -Once you've run your tests with nyc, simply run: +nyc allows you to inherit other configurations using the key `extends`. +As an example, an alternative way to configure nyc for `babel-plugin-istanbul` would be to use the [@istanbuljs/nyc-config-babel preset](https://www.npmjs.com/package/@istanbuljs/nyc-config-babel): -```bash -nyc report +```json +{ + "nyc": { + "extends": "@istanbuljs/nyc-config-babel" + //add your custom overrides here + } +} ``` -To view your coverage report: +To publish and reuse your own `nyc` configuration, simply create an npm module that exports an `index.json` with your `nyc` config. - +### Accurate stack traces using source-maps -You can use [any reporters that are supported by `istanbul`](https://github.com/istanbuljs/istanbuljs/tree/master/packages/istanbul-reports/lib): `clover`, `cobertura`, `html`, `json-summary`, `json`, `lcov`, `lcovonly`, `none`, `teamcity`, `text-lcov`, `text-summary`, `text`. +When `produce-source-map` is set to true, then the instrumented source files will +include inline source maps for the instrumenter transform. When combined with +[source-map-support](https://github.com/evanw/node-source-map-support), +stack traces for instrumented code will reflect their original lines. -```bash -nyc report --reporter=lcov -``` +### Support for custom require hooks (babel, typescript, etc.) -You can find examples of the output for various reporters [here](https://istanbul.js.org/docs/advanced/alternative-reporters). +nyc supports custom require hooks like [`@babel/register`]. nyc can load +the hooks for you, [using the `--require` flag](#require-additional-modules). -You also have the choice of using a [custom reporter](https://github.com/pedrocarrico/istanbul-reporter-aws-cloudwatch-metrics). -Install custom reporters as a development dependency and you can use the `--reporter` flag to load and view them: +Source maps are used to map coverage information back to the appropriate lines +of the pre-transpiled code. You'll have to configure your custom require hook +to inline the source-map in the transpiled code. For Babel that means setting +the `sourceMaps` option to `inline`. -```bash -nyc report --reporter= -``` +### Source-Map support for pre-instrumented codebases + +If you opt to pre-instrument your source-code (rather than using a just-in-time +transpiler like [`@babel/register`]) nyc supports both inline source-maps and +`.map` files. + +_Important: If you are using nyc with a project that pre-instruments its code, +run nyc with the configuration option `--exclude-after-remap` set to `false`. +Otherwise nyc's reports will exclude any files that source-maps remap to folders +covered under exclude rules._ ## [Producing instrumented source](./docs/instrument.md) @@ -322,87 +288,6 @@ modules should be required in the subprocess collecting coverage: `nyc`'s default behavior is to cache instrumented files to disk to prevent instrumenting source files multiple times, and speed `nyc` execution times. You can disable this behavior by running `nyc` with the `--cache false` flag. You can also change the default cache directory from `./node_modules/.cache/nyc` by setting the `--cache-dir` flag. -## Configuring `nyc` - -Any configuration options that can be set via the command line can also be specified in the `nyc` stanza of your package.json, or within a seperate configuration file - a variety of flavors are available: - -| File name | File Association | -|-----------------|------------------| -| `.nycrc` | JSON | -| `.nycrc.json` | JSON | -| `.nycrc.yaml` | YAML | -| `.nycrc.yml` | YAML | -| `nyc.config.js` | CommonJS export | - -**package.json:** - -```json -{ - "description": "These are just examples for demonstration, nothing prescriptive", - "nyc": { - "check-coverage": true, - "per-file": true, - "lines": 99, - "statements": 99, - "functions": 99, - "branches": 99, - "include": [ - "src/**/*.js" - ], - "exclude": [ - "src/**/*.spec.js" - ], - "ignore-class-method": "methodToIgnore", - "reporter": [ - "lcov", - "text-summary" - ], - "require": [ - "./test/helpers/some-helper.js" - ], - "extension": [ - ".jsx" - ], - "cache": true, - "all": true, - "temp-dir": "./alternative-tmp", - "report-dir": "./alternative" - } -} -``` - -Configuration can also be provided by `nyc.config.js` if programmed logic is required: -```js -'use strict'; -const {defaultExclude} = require('test-exclude'); -const isWindows = require('is-windows'); - -let platformExclude = [ - isWindows() ? 'lib/posix.js' : 'lib/win32.js' -]; - -module.exports = { - exclude: platformExclude.concat(defaultExclude) -}; -``` - -### Publish, and reuse, your nyc configuration - -nyc allows you to inherit other configurations using the key `extends`. As an example, -an alternative way to configure nyc for `babel-plugin-istanbul` would be to use the -[@istanbuljs/nyc-config-babel preset](https://www.npmjs.com/package/@istanbuljs/nyc-config-babel): - -```json -{ - "nyc": { - "extends": "@istanbuljs/nyc-config-babel" - } -} -``` - -To publish and resuse your own `nyc` configuration, simply create an npm module that -exports an `index.json` with your `nyc` config. - ## High and low watermarks Several of the coverage reporters supported by nyc display special information From 5c77bf9b09f82df295fab44e802cc081736dd684 Mon Sep 17 00:00:00 2001 From: Jason Kurian Date: Sat, 27 Apr 2019 23:55:59 -0400 Subject: [PATCH 02/12] docs(readme): more reorganization [skip ci] add a Coverage thresholds section add the skip-full option to the table because it's interesting move nyc instrument link closer to the advanced features --- README.md | 117 +++++++++++++++++++++++++++++++----------------------- 1 file changed, 68 insertions(+), 49 deletions(-) diff --git a/README.md b/README.md index 1d9e41fe8..1c97f8634 100644 --- a/README.md +++ b/README.md @@ -100,6 +100,7 @@ This table is a quick TLDR for the rest of this readme and there are more advanc | `exclude` | See [selecting files for coverage for more info](#selecting-files-for-coverage) | `Array` | [list](https://github.com/istanbuljs/istanbuljs/blob/master/packages/test-exclude/index.js#L176-L184) | | `reporter` | [coverage reporters to use](https://istanbul.js.org/docs/advanced/alternative-reporters/) | `Array` | `['text']` | | `report-dir` | where to put the coverage report files | `String` | `./coverage` | +| `skip-full` | don't show files with 100% statement, branch, and function coverage | `Boolean` | `false` | | `temp-dir` | directory to output raw coverage information to | `String` | `./.nyc_output` | Configuration can also be provided by `nyc.config.js` if programmed logic is required: @@ -133,54 +134,6 @@ As an example, an alternative way to configure nyc for `babel-plugin-istanbul` w To publish and reuse your own `nyc` configuration, simply create an npm module that exports an `index.json` with your `nyc` config. -### Accurate stack traces using source-maps - -When `produce-source-map` is set to true, then the instrumented source files will -include inline source maps for the instrumenter transform. When combined with -[source-map-support](https://github.com/evanw/node-source-map-support), -stack traces for instrumented code will reflect their original lines. - -### Support for custom require hooks (babel, typescript, etc.) - -nyc supports custom require hooks like [`@babel/register`]. nyc can load -the hooks for you, [using the `--require` flag](#require-additional-modules). - -Source maps are used to map coverage information back to the appropriate lines -of the pre-transpiled code. You'll have to configure your custom require hook -to inline the source-map in the transpiled code. For Babel that means setting -the `sourceMaps` option to `inline`. - -### Source-Map support for pre-instrumented codebases - -If you opt to pre-instrument your source-code (rather than using a just-in-time -transpiler like [`@babel/register`]) nyc supports both inline source-maps and -`.map` files. - -_Important: If you are using nyc with a project that pre-instruments its code, -run nyc with the configuration option `--exclude-after-remap` set to `false`. -Otherwise nyc's reports will exclude any files that source-maps remap to folders -covered under exclude rules._ - -## [Producing instrumented source](./docs/instrument.md) - -## Setting the project root directory - -nyc runs a lot of file system operations relative to the project root directory. -During startup nyc will look for the *default* project root directory. -The *default* project root directory is the first directory found that contains a `package.json` file when searching from the current working directory up. -If nyc fails to find a directory containing a `package.json` file, it will use the current working directory as the *default* project root directory. -You can change the project root directory with the `--cwd` option. - -nyc uses the project root directory when: - * looking for source files to instrument - * creating globs for include and exclude rules during file selection - * loading custom require hooks from the `require` array - -nyc may create artefact directories within the project root, such as: - * the report directory, `/coverage` - * the cache directory, `/node_modules/.cache/nyc` - * the temp directory, `/.nyc_output` - ## Selecting files for coverage By default, nyc only collects coverage for source files that are visited during a test. @@ -277,6 +230,52 @@ The exclude rules then prevent nyc instrumenting anything in a `test` folder and } ``` +## Accurate stack traces using source-maps + +When `produce-source-map` is set to true, then the instrumented source files will +include inline source maps for the instrumenter transform. When combined with +[source-map-support](https://github.com/evanw/node-source-map-support), +stack traces for instrumented code will reflect their original lines. + +### Support for custom require hooks (babel, typescript, etc.) + +nyc supports custom require hooks like [`@babel/register`]. nyc can load +the hooks for you, [using the `--require` flag](#require-additional-modules). + +Source maps are used to map coverage information back to the appropriate lines +of the pre-transpiled code. You'll have to configure your custom require hook +to inline the source-map in the transpiled code. For Babel that means setting +the `sourceMaps` option to `inline`. + +### Source-Map support for pre-instrumented codebases + +If you opt to pre-instrument your source-code (rather than using a just-in-time +transpiler like [`@babel/register`]) nyc supports both inline source-maps and +`.map` files. + +_Important: If you are using nyc with a project that pre-instruments its code, +run nyc with the configuration option `--exclude-after-remap` set to `false`. +Otherwise nyc's reports will exclude any files that source-maps remap to folders +covered under exclude rules._ + +## Setting the project root directory + +nyc runs a lot of file system operations relative to the project root directory. +During startup nyc will look for the *default* project root directory. +The *default* project root directory is the first directory found that contains a `package.json` file when searching from the current working directory up. +If nyc fails to find a directory containing a `package.json` file, it will use the current working directory as the *default* project root directory. +You can change the project root directory with the `--cwd` option. + +nyc uses the project root directory when: + * looking for source files to instrument + * creating globs for include and exclude rules during file selection + * loading custom require hooks from the `require` array + +nyc may create artefact directories within the project root, such as: + * the report directory, `/coverage` + * the cache directory, `/node_modules/.cache/nyc` + * the temp directory, `/.nyc_output` + ## Require additional modules The `--require` flag can be provided to `nyc` to indicate that additional @@ -288,7 +287,25 @@ modules should be required in the subprocess collecting coverage: `nyc`'s default behavior is to cache instrumented files to disk to prevent instrumenting source files multiple times, and speed `nyc` execution times. You can disable this behavior by running `nyc` with the `--cache false` flag. You can also change the default cache directory from `./node_modules/.cache/nyc` by setting the `--cache-dir` flag. -## High and low watermarks +## Coverage thresholds + +You can set custom coverage thresholds that will fail if `check-coverage` is set to `true` and your coverage drops below those thresholds. +For example, in the following configuration, dropping below 80% branch, line, functions, or statements coverage would fail the build (you can have any combination of these): + +```json +{ + "nyc": { + "branches": 80, + "lines": 80, + "functions": 80, + "statements": 80 + } +} +``` + +To do this check on a per-file basis (as opposed to in aggregate), set the `per-file` option to `true`. + +### High and low watermarks Several of the coverage reporters supported by nyc display special information for high and low watermarks: @@ -343,6 +360,8 @@ rather than having to ignore every instance of that method: ## [Integrating with codecov](./docs/setup-codecov.md) +## [Producing instrumented source](./docs/instrument.md) + ## Integrating with TAP formatters Many testing frameworks (Mocha, Tape, Tap, etc.) can produce [TAP](https://en.wikipedia.org/wiki/Test_Anything_Protocol) output. [tap-nyc](https://github.com/MegaArman/tap-nyc) is a TAP formatter designed to look nice with nyc. From d23fc704ffdc273698c23062e7e1ff408bf8f698 Mon Sep 17 00:00:00 2001 From: Jason Kurian Date: Sun, 28 Apr 2019 02:23:07 -0400 Subject: [PATCH 03/12] docs(readme): rework installation section [skip ci] --- README.md | 35 ++++++++++------------------------- 1 file changed, 10 insertions(+), 25 deletions(-) diff --git a/README.md b/README.md index 1c97f8634..4e4be7757 100644 --- a/README.md +++ b/README.md @@ -13,47 +13,32 @@ Istanbul's state of the art command line interface, with support for: * applications that spawn subprocesses. * ES2015 transforms, via [`babel-plugin-istanbul`], or source-maps. -## Instrumenting your code +## Installation & Usage -You can install nyc as a development dependency and add it to the test stanza -in your package.json. - -```shell -npm i nyc --save-dev -``` +You can use `npx` instead of installing nyc as a dependency. +Simply add `nyc` before your test runner is invoked (replace `mocha` with your test runner everywhere you see it): ```json { "scripts": { - "test": "nyc mocha" + "test": "npx nyc mocha" } } ``` -Alternatively, you can install nyc globally and use it to execute `npm test`: - -```shell -npm i nyc -g -``` +Or use your package manager to add a dev dependency: `npm i -D nyc` or `yarn add -D nyc`. -```shell -nyc npm test -``` +## Configuring `nyc` -nyc accepts a wide variety of configuration arguments, run `nyc --help` for -thorough documentation. +nyc accepts a wide variety of configuration arguments, run `npx nyc --help` for thorough documentation. -Configuration arguments should be provided prior to the program that nyc -is executing. As an example, the following command executes `npm test`, -and indicates to nyc that it should output both an `lcov` -and a `text-lcov` coverage report. +Configuration arguments on the command-line should be provided prior to the program that nyc is executing. +As an example, the following command executes `npm test`, and indicates to nyc that it should output both an `lcov` and a `text-summary` coverage report. ```shell -nyc --reporter=lcov --reporter=text-lcov npm test +npx nyc --reporter=lcov --reporter=text-summary npm test ``` -## Configuring `nyc` - ### Babel projects Please start with the pre-configured [@istanbuljs/nyc-config-babel preset](https://www.npmjs.com/package/@istanbuljs/nyc-config-babel). From 79024a355fb00b3afb0a29ff7590967fa7e08e20 Mon Sep 17 00:00:00 2001 From: Jason Kurian Date: Sun, 28 Apr 2019 02:26:47 -0400 Subject: [PATCH 04/12] docs(readme): move Source map info towards the bottom [skip ci] --- README.md | 56 +++++++++++++++++++++++++++---------------------------- 1 file changed, 28 insertions(+), 28 deletions(-) diff --git a/README.md b/README.md index 4e4be7757..8654386d1 100644 --- a/README.md +++ b/README.md @@ -215,34 +215,6 @@ The exclude rules then prevent nyc instrumenting anything in a `test` folder and } ``` -## Accurate stack traces using source-maps - -When `produce-source-map` is set to true, then the instrumented source files will -include inline source maps for the instrumenter transform. When combined with -[source-map-support](https://github.com/evanw/node-source-map-support), -stack traces for instrumented code will reflect their original lines. - -### Support for custom require hooks (babel, typescript, etc.) - -nyc supports custom require hooks like [`@babel/register`]. nyc can load -the hooks for you, [using the `--require` flag](#require-additional-modules). - -Source maps are used to map coverage information back to the appropriate lines -of the pre-transpiled code. You'll have to configure your custom require hook -to inline the source-map in the transpiled code. For Babel that means setting -the `sourceMaps` option to `inline`. - -### Source-Map support for pre-instrumented codebases - -If you opt to pre-instrument your source-code (rather than using a just-in-time -transpiler like [`@babel/register`]) nyc supports both inline source-maps and -`.map` files. - -_Important: If you are using nyc with a project that pre-instruments its code, -run nyc with the configuration option `--exclude-after-remap` set to `false`. -Otherwise nyc's reports will exclude any files that source-maps remap to folders -covered under exclude rules._ - ## Setting the project root directory nyc runs a lot of file system operations relative to the project root directory. @@ -341,6 +313,34 @@ rather than having to ignore every instance of that method: } ``` +## Accurate stack traces using source-maps + +When `produce-source-map` is set to true, then the instrumented source files will +include inline source maps for the instrumenter transform. When combined with +[source-map-support](https://github.com/evanw/node-source-map-support), +stack traces for instrumented code will reflect their original lines. + +### Support for custom require hooks (babel, typescript, etc.) + +nyc supports custom require hooks like [`@babel/register`]. nyc can load +the hooks for you, [using the `--require` flag](#require-additional-modules). + +Source maps are used to map coverage information back to the appropriate lines +of the pre-transpiled code. You'll have to configure your custom require hook +to inline the source-map in the transpiled code. For Babel that means setting +the `sourceMaps` option to `inline`. + +### Source-Map support for pre-instrumented codebases + +If you opt to pre-instrument your source-code (rather than using a just-in-time +transpiler like [`@babel/register`]) nyc supports both inline source-maps and +`.map` files. + +_Important: If you are using nyc with a project that pre-instruments its code, +run nyc with the configuration option `--exclude-after-remap` set to `false`. +Otherwise nyc's reports will exclude any files that source-maps remap to folders +covered under exclude rules._ + ## [Integrating with coveralls](./docs/setup-coveralls.md) ## [Integrating with codecov](./docs/setup-codecov.md) From 6323afd0ed493b259a1ac867cf52ab9aba10a311 Mon Sep 17 00:00:00 2001 From: Jason Kurian Date: Tue, 30 Apr 2019 02:25:41 -0400 Subject: [PATCH 05/12] remove incorrect sections and formatting [skip ci] --- README.md | 31 ++++++++++++------------------- 1 file changed, 12 insertions(+), 19 deletions(-) diff --git a/README.md b/README.md index 8654386d1..55f1a3b62 100644 --- a/README.md +++ b/README.md @@ -235,14 +235,17 @@ nyc may create artefact directories within the project root, such as: ## Require additional modules -The `--require` flag can be provided to `nyc` to indicate that additional -modules should be required in the subprocess collecting coverage: +The `--require` flag can be provided to `nyc` to indicate that additional modules should be required in the subprocess collecting coverage: -`nyc --require @babel/register --require @babel/polyfill mocha` +``` +nyc --require @babel/register --require @babel/polyfill mocha +``` ## Caching -`nyc`'s default behavior is to cache instrumented files to disk to prevent instrumenting source files multiple times, and speed `nyc` execution times. You can disable this behavior by running `nyc` with the `--cache false` flag. You can also change the default cache directory from `./node_modules/.cache/nyc` by setting the `--cache-dir` flag. +`nyc`'s default behavior is to cache instrumented files to disk to prevent instrumenting source files multiple times, and speed `nyc` execution times. +You can disable this behavior by running `nyc` with the `--cache false` flag. +You can also change the default cache directory from `./node_modules/.cache/nyc` by setting the `--cache-dir` flag. ## Coverage thresholds @@ -264,13 +267,10 @@ To do this check on a per-file basis (as opposed to in aggregate), set the `per- ### High and low watermarks -Several of the coverage reporters supported by nyc display special information -for high and low watermarks: +Several of the coverage reporters supported by nyc display special information for high and low watermarks: -* high-watermarks represent healthy test coverage (in many reports - this is represented with green highlighting). -* low-watermarks represent sub-optimal coverage levels (in many reports - this is represented with red highlighting). +* high-watermarks represent healthy test coverage (in many reports this is represented with green highlighting). +* low-watermarks represent sub-optimal coverage levels (in many reports this is represented with red highlighting). You can specify custom high and low watermarks in nyc's configuration: @@ -313,14 +313,7 @@ rather than having to ignore every instance of that method: } ``` -## Accurate stack traces using source-maps - -When `produce-source-map` is set to true, then the instrumented source files will -include inline source maps for the instrumenter transform. When combined with -[source-map-support](https://github.com/evanw/node-source-map-support), -stack traces for instrumented code will reflect their original lines. - -### Support for custom require hooks (babel, typescript, etc.) +## Support for custom require hooks (babel, typescript, etc.) nyc supports custom require hooks like [`@babel/register`]. nyc can load the hooks for you, [using the `--require` flag](#require-additional-modules). @@ -330,7 +323,7 @@ of the pre-transpiled code. You'll have to configure your custom require hook to inline the source-map in the transpiled code. For Babel that means setting the `sourceMaps` option to `inline`. -### Source-Map support for pre-instrumented codebases +## Source-Map support for pre-instrumented codebases If you opt to pre-instrument your source-code (rather than using a just-in-time transpiler like [`@babel/register`]) nyc supports both inline source-maps and From 6fb718116a80710701828e3b6f342314808f6967 Mon Sep 17 00:00:00 2001 From: Jason Kurian Date: Tue, 30 Apr 2019 12:20:45 -0400 Subject: [PATCH 06/12] review comments [skip ci] --- README.md | 21 +++++++++++---------- 1 file changed, 11 insertions(+), 10 deletions(-) diff --git a/README.md b/README.md index 55f1a3b62..9ff479537 100644 --- a/README.md +++ b/README.md @@ -78,15 +78,15 @@ This table is a quick TLDR for the rest of this readme and there are more advanc | Option name | Description | Type | Default | | ----------- | ----------- | ---- | ------- | -| `all` | whether or not to instrument all files (not just the ones touched by your test suite) | `Boolean` | `false` | -| `check-coverage` | check whether coverage is within thresholds, fail if not | `Boolean` | `false` | -| `extension` | a list of extensions that nyc should attempt to handle in addition to `.js` | `Array` | `['.js']` | -| `include` | [globs of files to be included from instrumentation](#selecting-files-for-coverage) | `Array` | `['**']`| -| `exclude` | See [selecting files for coverage for more info](#selecting-files-for-coverage) | `Array` | [list](https://github.com/istanbuljs/istanbuljs/blob/master/packages/test-exclude/index.js#L176-L184) | -| `reporter` | [coverage reporters to use](https://istanbul.js.org/docs/advanced/alternative-reporters/) | `Array` | `['text']` | -| `report-dir` | where to put the coverage report files | `String` | `./coverage` | -| `skip-full` | don't show files with 100% statement, branch, and function coverage | `Boolean` | `false` | -| `temp-dir` | directory to output raw coverage information to | `String` | `./.nyc_output` | +| `all` | Whether or not to instrument all files (not just the ones touched by your test suite) | `Boolean` | `false` | +| `check-coverage` | Check whether coverage is within thresholds, fail if not | `Boolean` | `false` | +| `extension` | List of extensions that nyc should attempt to handle in addition to `.js` | `Array` | `['.js']` | +| `include` | See [selecting files for coverage] for more info | `Array` | `['**']`| +| `exclude` | See [selecting files for coverage] for more info | `Array` | [list](https://github.com/istanbuljs/istanbuljs/blob/master/packages/test-exclude/index.js#L176-L184) | +| `reporter` | [Coverage reporters to use](https://istanbul.js.org/docs/advanced/alternative-reporters/) | `Array` | `['text']` | +| `report-dir` | Where to put the coverage report files | `String` | `./coverage` | +| `skip-full` | Don't show files with 100% statement, branch, and function coverage | `Boolean` | `false` | +| `temp-dir` | Directory to output raw coverage information to | `String` | `./.nyc_output` | Configuration can also be provided by `nyc.config.js` if programmed logic is required: ```js @@ -117,7 +117,7 @@ As an example, an alternative way to configure nyc for `babel-plugin-istanbul` w } ``` -To publish and reuse your own `nyc` configuration, simply create an npm module that exports an `index.json` with your `nyc` config. +To publish and reuse your own `nyc` configuration, simply create an npm module that exports your JSON config (via [`index.json`](https://github.com/istanbuljs/istanbuljs/blob/master/packages/nyc-config-typescript/) or a CJS [`index.js`](https://github.com/istanbuljs/istanbuljs/blob/master/packages/nyc-config-hook-run-in-this-context/)). ## Selecting files for coverage @@ -354,3 +354,4 @@ Take a look at http://istanbul.js.org/docs/advanced/ and please feel free to [co [`@babel/register`]: https://www.npmjs.com/package/@babel/register [`babel-plugin-istanbul`]: https://github.com/istanbuljs/babel-plugin-istanbul +[selecting files for coverage]: #selecting-files-for-coverage From e1567a75f7098e920cb7bfb0080d28f3a9bcebe0 Mon Sep 17 00:00:00 2001 From: Jason Kurian Date: Wed, 1 May 2019 17:18:53 -0400 Subject: [PATCH 07/12] add more examples and clarifications [skip ci] --- .github/ISSUE_TEMPLATE.md | 4 ++- README.md | 52 ++++++++++++++++++++++++++------------- 2 files changed, 38 insertions(+), 18 deletions(-) diff --git a/.github/ISSUE_TEMPLATE.md b/.github/ISSUE_TEMPLATE.md index 749671df0..58d559499 100644 --- a/.github/ISSUE_TEMPLATE.md +++ b/.github/ISSUE_TEMPLATE.md @@ -18,7 +18,9 @@ create a basic `nyc-bug-demo` repository and link to that please [mandatory]. ## Environment Information ``` # paste the output here diff --git a/README.md b/README.md index 9ff479537..4d215a2a2 100644 --- a/README.md +++ b/README.md @@ -16,17 +16,30 @@ Istanbul's state of the art command line interface, with support for: ## Installation & Usage You can use `npx` instead of installing nyc as a dependency. -Simply add `nyc` before your test runner is invoked (replace `mocha` with your test runner everywhere you see it): +Simply add `nyc@latest` before your test runner is invoked (replace `mocha` with your test runner everywhere you see it): ```json { "scripts": { - "test": "npx nyc mocha" + "test": "npx nyc@latest mocha" } } ``` -Or use your package manager to add a dev dependency: `npm i -D nyc` or `yarn add -D nyc`. +Or use your package manager to add it as a dev dependency: `npm i -D nyc` or `yarn add -D nyc`. +You can also use nyc to call npm scripts (assuming they don't already have nyc executed in them), like so: +```json +{ + "scripts": { + "test": "mocha", + "coverage": "nyc npm run test" + } +} +``` + +**Note**: If you use [`jest`](https://npm.im/jest) or [`tap`](https://www.node-tap.org/), you do not need to install `nyc`. +Those runners already have the IstanbulJS libraries to provide coverage for you. +Follow their documentation to enable and configure coverage reporting. ## Configuring `nyc` @@ -50,14 +63,18 @@ Please start with the pre-configured [@istanbuljs/nyc-config-typescript preset]( #### Adding your overrides +nyc allows you to inherit other configurations using the key `extends`. +You can then add the specific configuration options you want that aren't in that particular shared config, e.g. ```json { "nyc": { "extends": "@istanbuljs/nyc-config-typescript", - "all": true + "all": true, + "check-coverage": true } } ``` + ### Configuration files Any configuration options that can be set via the command line can also be specified in the `nyc` stanza of your package.json, or within a seperate configuration file - a variety of flavors are available: @@ -103,21 +120,22 @@ module.exports = { }; ``` -### Publish and reuse your nyc configuration +### Publish and reuse your nyc configuration(s) -nyc allows you to inherit other configurations using the key `extends`. -As an example, an alternative way to configure nyc for `babel-plugin-istanbul` would be to use the [@istanbuljs/nyc-config-babel preset](https://www.npmjs.com/package/@istanbuljs/nyc-config-babel): +To publish and reuse your own `nyc` configuration, simply create an npm module that exports your JSON config (via [`index.json`](https://github.com/istanbuljs/istanbuljs/blob/master/packages/nyc-config-typescript/) or a CJS [`index.js`](https://github.com/istanbuljs/istanbuljs/blob/master/packages/nyc-config-hook-run-in-this-context/)). -```json -{ - "nyc": { - "extends": "@istanbuljs/nyc-config-babel" - //add your custom overrides here - } -} -``` +A more advanced use case would be to combine multiple shared in a `nyc.config.js` file: +```js +const babelConfig = require('@istanbuljs/nyc-config-babel'); +const hookRunInThisContextConfig = require('@istanbuljs/nyc-config-hook-run-in-this-context'); -To publish and reuse your own `nyc` configuration, simply create an npm module that exports your JSON config (via [`index.json`](https://github.com/istanbuljs/istanbuljs/blob/master/packages/nyc-config-typescript/) or a CJS [`index.js`](https://github.com/istanbuljs/istanbuljs/blob/master/packages/nyc-config-hook-run-in-this-context/)). +module.exports = { + ...babelConfig, + ...hookRunInThisContextConfig, + all: true, + 'check-coverage': true +}; +``` ## Selecting files for coverage @@ -153,7 +171,7 @@ You can also specify negated paths in the `exclude` array, by prefixing them wit Negated paths can restore paths that have been already been excluded in the `exclude` array. Multiple `exclude` globs can be specified on the command line, each must follow a `--exclude`, `-x` switch. -The `exclude` option has the following defaults settings: +The `exclude` option has the following defaults: ```js [ 'coverage/**', From 4621789d333e9ac16eadfaaf5a6a2aa91c01843c Mon Sep 17 00:00:00 2001 From: Jason Kurian Date: Wed, 1 May 2019 17:23:37 -0400 Subject: [PATCH 08/12] fix examples [skip ci] --- README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index 4d215a2a2..43ff8fb8f 100644 --- a/README.md +++ b/README.md @@ -124,7 +124,7 @@ module.exports = { To publish and reuse your own `nyc` configuration, simply create an npm module that exports your JSON config (via [`index.json`](https://github.com/istanbuljs/istanbuljs/blob/master/packages/nyc-config-typescript/) or a CJS [`index.js`](https://github.com/istanbuljs/istanbuljs/blob/master/packages/nyc-config-hook-run-in-this-context/)). -A more advanced use case would be to combine multiple shared in a `nyc.config.js` file: +A more advanced use case would be to combine multiple shared configs in a `nyc.config.js` file: ```js const babelConfig = require('@istanbuljs/nyc-config-babel'); const hookRunInThisContextConfig = require('@istanbuljs/nyc-config-hook-run-in-this-context'); @@ -256,7 +256,7 @@ nyc may create artefact directories within the project root, such as: The `--require` flag can be provided to `nyc` to indicate that additional modules should be required in the subprocess collecting coverage: ``` -nyc --require @babel/register --require @babel/polyfill mocha +nyc --require esm mocha ``` ## Caching From 3e0624e26ba75274df22df05fba00f03e9e9724f Mon Sep 17 00:00:00 2001 From: Jason Kurian Date: Wed, 1 May 2019 17:32:43 -0400 Subject: [PATCH 09/12] documents interaction between --require and --all [skip ci] --- README.md | 14 +++++++++++--- 1 file changed, 11 insertions(+), 3 deletions(-) diff --git a/README.md b/README.md index 43ff8fb8f..25089ea04 100644 --- a/README.md +++ b/README.md @@ -54,16 +54,16 @@ npx nyc --reporter=lcov --reporter=text-summary npm test ### Babel projects -Please start with the pre-configured [@istanbuljs/nyc-config-babel preset](https://www.npmjs.com/package/@istanbuljs/nyc-config-babel). +Please start with the pre-configured [`@istanbuljs/nyc-config-babel`] preset. You can add your custom configuration options as shown below. ### TypeScript projects -Please start with the pre-configured [@istanbuljs/nyc-config-typescript preset](https://www.npmjs.com/package/@istanbuljs/nyc-config-typescript). +Please start with the pre-configured [`@istanbuljs/nyc-config-typescript`](https://www.npmjs.com/package/@istanbuljs/nyc-config-typescript) preset. #### Adding your overrides -nyc allows you to inherit other configurations using the key `extends`. +nyc allows you to inherit other configurations using the key `extends` in the `package.json` stanza, `.nycrc`, or YAML files. You can then add the specific configuration options you want that aren't in that particular shared config, e.g. ```json { @@ -259,6 +259,13 @@ The `--require` flag can be provided to `nyc` to indicate that additional module nyc --require esm mocha ``` +### Interaction with `--all` flag + +The `--require` flag also operates on the main nyc process for use by `--all`. +For example, in situations with `nyc --all --instrument false` and babel-plugin-istanbul setup the `--all` option only works if `--require @babel/register` is passed to nyc. +Passing it to mocha would cause the tests to be instrumented but unloaded sources would not be seen. +The [`@istanbuljs/nyc-config-babel`] package handles this for you! + ## Caching `nyc`'s default behavior is to cache instrumented files to disk to prevent instrumenting source files multiple times, and speed `nyc` execution times. @@ -372,4 +379,5 @@ Take a look at http://istanbul.js.org/docs/advanced/ and please feel free to [co [`@babel/register`]: https://www.npmjs.com/package/@babel/register [`babel-plugin-istanbul`]: https://github.com/istanbuljs/babel-plugin-istanbul +[`@istanbuljs/nyc-config-babel`]: https://www.npmjs.com/package/@istanbuljs/nyc-config-babel [selecting files for coverage]: #selecting-files-for-coverage From a66a6c1e436c34a120c45fc3fd1b039c7645f017 Mon Sep 17 00:00:00 2001 From: Jason Kurian Date: Wed, 1 May 2019 21:01:43 -0400 Subject: [PATCH 10/12] add Combining multiple runs section [skip ci] remove old custom require hooks section that has been updated above --- README.md | 44 +++++++++++++++++++++++++++++--------------- 1 file changed, 29 insertions(+), 15 deletions(-) diff --git a/README.md b/README.md index 25089ea04..2021efcfc 100644 --- a/README.md +++ b/README.md @@ -246,7 +246,7 @@ nyc uses the project root directory when: * creating globs for include and exclude rules during file selection * loading custom require hooks from the `require` array -nyc may create artefact directories within the project root, such as: +nyc may create artefact directories within the project root, with these defaults: * the report directory, `/coverage` * the cache directory, `/node_modules/.cache/nyc` * the temp directory, `/.nyc_output` @@ -338,26 +338,40 @@ rather than having to ignore every instance of that method: } ``` -## Support for custom require hooks (babel, typescript, etc.) +## Combining reports from multiple runs +If for whatever reason you have different test runners in your project or a different series of test runs for different kinds of tests, nyc will automatically combine the coverage report for you if configured correctly with the `--no-clean` flag and the `report` command. +Originally inspired by @janiukjf in #1001, here's an example, where the `test:*` scripts (not shown) invoke only your test runner(s) and not nyc: -nyc supports custom require hooks like [`@babel/register`]. nyc can load -the hooks for you, [using the `--require` flag](#require-additional-modules). +```json +{ + "scripts": { + "cover": "npm run cover:unit && npm run cover:integration && npm run cover:report", + "cover:unit": "nyc --silent npm run test:unit", + "cover:integration": "nyc --silent --no-clean npm run test:integration", + "cover:report": "nyc report --reporter=lcov --reporter=text" + } +} +``` + +### What about `nyc merge`? -Source maps are used to map coverage information back to the appropriate lines -of the pre-transpiled code. You'll have to configure your custom require hook -to inline the source-map in the transpiled code. For Babel that means setting -the `sourceMaps` option to `inline`. +The `nyc merge` command is for producing one _raw coverage output file_ that combines the results from many test runs. +So if you had the above setup and needed to produce a single `coverage.json` for some external tool, you could do: + +```json +{ + "scripts": { + "cover:merge": "npm run cover:unit && npm run cover:integration && nyc merge .nyc_output coverage.json" + } +} +``` ## Source-Map support for pre-instrumented codebases -If you opt to pre-instrument your source-code (rather than using a just-in-time -transpiler like [`@babel/register`]) nyc supports both inline source-maps and -`.map` files. +If you opt to pre-instrument your source-code (rather than using a just-in-time transpiler like [`@babel/register`]) nyc supports both inline source-maps and `.map` files. -_Important: If you are using nyc with a project that pre-instruments its code, -run nyc with the configuration option `--exclude-after-remap` set to `false`. -Otherwise nyc's reports will exclude any files that source-maps remap to folders -covered under exclude rules._ +_Important: If you are using nyc with a project that pre-instruments its code, run nyc with the configuration option `--exclude-after-remap` set to `false`. +Otherwise nyc's reports will exclude any files that source-maps remap to folders covered under exclude rules._ ## [Integrating with coveralls](./docs/setup-coveralls.md) From 82a6dd59d217299d01b7aea3f4caa1a242f1dac0 Mon Sep 17 00:00:00 2001 From: Jason Kurian Date: Thu, 2 May 2019 10:30:41 -0400 Subject: [PATCH 11/12] fix installation section and others [skip ci] --- .github/ISSUE_TEMPLATE.md | 1 - README.md | 28 +++++++++++++++------------- 2 files changed, 15 insertions(+), 14 deletions(-) diff --git a/.github/ISSUE_TEMPLATE.md b/.github/ISSUE_TEMPLATE.md index 58d559499..001abc2c7 100644 --- a/.github/ISSUE_TEMPLATE.md +++ b/.github/ISSUE_TEMPLATE.md @@ -20,7 +20,6 @@ create a basic `nyc-bug-demo` repository and link to that please [mandatory]. ``` # paste the output here diff --git a/README.md b/README.md index 2021efcfc..729a36ca1 100644 --- a/README.md +++ b/README.md @@ -1,7 +1,7 @@ # nyc [![Build Status](https://travis-ci.org/istanbuljs/nyc.svg?branch=master)](https://travis-ci.org/istanbuljs/nyc) -[![Coverage Status](https://coveralls.io/repos/bcoe/nyc/badge.svg?branch=)](https://coveralls.io/r/bcoe/nyc?branch=master) +[![Coverage Status](https://coveralls.io/repos/istanbuljs/nyc/badge.svg?branch=)](https://coveralls.io/r/istanbuljs/nyc?branch=master) [![NPM version](https://img.shields.io/npm/v/nyc.svg)](https://www.npmjs.com/package/nyc) [![Conventional Commits](https://img.shields.io/badge/Conventional%20Commits-1.0.0-yellow.svg)](https://conventionalcommits.org) [![community slack](http://devtoolscommunity.herokuapp.com/badge.svg)](http://devtoolscommunity.herokuapp.com) @@ -11,32 +11,34 @@ _Having problems? want to contribute? join our [community slack](http://devtools Istanbul's state of the art command line interface, with support for: * applications that spawn subprocesses. -* ES2015 transforms, via [`babel-plugin-istanbul`], or source-maps. +* source mapped coverage of Babel and TypeScript projects ## Installation & Usage -You can use `npx` instead of installing nyc as a dependency. -Simply add `nyc@latest` before your test runner is invoked (replace `mocha` with your test runner everywhere you see it): - +Use your package manager to add it as a dev dependency: `npm i -D nyc` or `yarn add -D nyc`. +You can use nyc to call npm scripts (assuming they don't already have nyc executed in them), like so (replace `mocha` with your test runner everywhere you see it): ```json { "scripts": { - "test": "npx nyc@latest mocha" + "test": "mocha", + "coverage": "nyc npm run test" } } ``` -Or use your package manager to add it as a dev dependency: `npm i -D nyc` or `yarn add -D nyc`. -You can also use nyc to call npm scripts (assuming they don't already have nyc executed in them), like so: +You can use also `npx` instead of installing nyc as a dependency, but you might get updates you are not ready for; to get around this, pin to a specific major version by specifying, e.g. `nyc@14`. +In general, simply add `nyc` before your test runner is invoked: + ```json { "scripts": { - "test": "mocha", - "coverage": "nyc npm run test" + "test": "npx nyc@latest mocha" } } ``` +This is a good way of testing upcoming releases of nyc, usually on the `next` tag. + **Note**: If you use [`jest`](https://npm.im/jest) or [`tap`](https://www.node-tap.org/), you do not need to install `nyc`. Those runners already have the IstanbulJS libraries to provide coverage for you. Follow their documentation to enable and configure coverage reporting. @@ -49,7 +51,7 @@ Configuration arguments on the command-line should be provided prior to the prog As an example, the following command executes `npm test`, and indicates to nyc that it should output both an `lcov` and a `text-summary` coverage report. ```shell -npx nyc --reporter=lcov --reporter=text-summary npm test +nyc --reporter=lcov --reporter=text-summary npm test ``` ### Babel projects @@ -89,7 +91,7 @@ Any configuration options that can be set via the command line can also be speci ### Common Configuration Options -See `npx nyc --help` for all options available. +See `nyc --help` for all options available. You can set these in any of the files listed above, or from the command line. This table is a quick TLDR for the rest of this readme and there are more advanced docs available. @@ -262,7 +264,7 @@ nyc --require esm mocha ### Interaction with `--all` flag The `--require` flag also operates on the main nyc process for use by `--all`. -For example, in situations with `nyc --all --instrument false` and babel-plugin-istanbul setup the `--all` option only works if `--require @babel/register` is passed to nyc. +For example, in situations with `nyc --all --instrument false` and [`babel-plugin-istanbul`] setup the `--all` option only works if `--require @babel/register` is passed to nyc. Passing it to mocha would cause the tests to be instrumented but unloaded sources would not be seen. The [`@istanbuljs/nyc-config-babel`] package handles this for you! From 970a26bb9ec936de2872b73cf063bed78329856c Mon Sep 17 00:00:00 2001 From: Jason Kurian Date: Thu, 2 May 2019 11:13:26 -0400 Subject: [PATCH 12/12] fixes examples [skip ci] --- README.md | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/README.md b/README.md index 729a36ca1..026a69a56 100644 --- a/README.md +++ b/README.md @@ -27,7 +27,6 @@ You can use nyc to call npm scripts (assuming they don't already have nyc execut ``` You can use also `npx` instead of installing nyc as a dependency, but you might get updates you are not ready for; to get around this, pin to a specific major version by specifying, e.g. `nyc@14`. -In general, simply add `nyc` before your test runner is invoked: ```json { @@ -48,10 +47,10 @@ Follow their documentation to enable and configure coverage reporting. nyc accepts a wide variety of configuration arguments, run `npx nyc --help` for thorough documentation. Configuration arguments on the command-line should be provided prior to the program that nyc is executing. -As an example, the following command executes `npm test`, and indicates to nyc that it should output both an `lcov` and a `text-summary` coverage report. +As an example, the following command executes `ava`, and indicates to nyc that it should output both an `lcov` (`lcov.info` + html report) and a `text-summary` coverage report. ```shell -nyc --reporter=lcov --reporter=text-summary npm test +nyc --reporter=lcov --reporter=text-summary ava ``` ### Babel projects