regression: Cypress 9.1.0 throws when code coverage is enabled, failing every test

[cellog]

Current behavior

every test fails with a crash

Desired behavior

Test code to reproduce

upgrade cypress to 9.1.0 with a project that has followed instructions for enabling instrumenting and code coverage. Failure is in both cypress open and cypress open-ct

Cypress Version

9.1.0

Other

Tested with Cypress 8.3.1 -> 9.1.0. Every version of Cypress except 9.1.0 successfully runs tests with code coverage enabled, version 9.1.0 fails

  "optionalDependencies": {
    "@cypress/code-coverage": "3.9.10",
    "@cypress/react": "5.9.4",
    "@cypress/webpack-dev-server": "1.4.0",
    "@cypress/webpack-preprocessor": "5.9.1",
    "@istanbuljs/nyc-config-typescript": "1.0.1",
    "@jsdevtools/coverage-istanbul-loader": "3.0.5",
    "@testing-library/cypress": "8.0.1",
    "@types/cypress-cucumber-preprocessor": "4.0.1",
    "babel-plugin-istanbul": "6.0.0",
    "cypress": "9.1.0",
    "cypress-cucumber-preprocessor": "4.2.0",
    "cypress-file-upload": "5.0.8",
    "cypress-iframe": "1.0.1",
    "cypress-real-events": "1.5.1"
  }

cypress.json:

{
  "baseUrl": "http://localhost:6007",
  "integrationFolder": "../src",
  "testFiles": "**/integration_tests/**/*.{test.tsx,test.ts,feature}",
  "video": false,
  "component": {
    "componentFolder": "../../",
    "testFiles": "**/component_tests/**/*.{test.tsx,test.ts}"
  }
}

code coverage-related plugins code (it's all standard stuff):

plugins/index.js

/* eslint-disable @typescript-eslint/no-var-requires */
/// <reference types="cypress" />
// ***********************************************************
// This example plugins/index.js can be used to load plugins
//
// You can change the location of this file or turn off loading
// the plugins file with the 'pluginsFile' configuration option.
//
// You can read more here:
// https://on.cypress.io/plugins-guide
// ***********************************************************

const resolve = require("resolve");
const path = require("path");

// This function is called when a project is opened or re-opened (e.g. due to
// the project's config changing)
const preprocessor = require("@cypress/webpack-preprocessor");
const codeCoverage = require("@cypress/code-coverage/task");
const cucumber = require("cypress-cucumber-preprocessor").default;
const browserify = require("@cypress/browserify-preprocessor");
const webpackOptions = require("./webpack.config");
const tsConfig = require("../../../../../tsconfig.common.json");

/**
 * @type {Cypress.PluginConfig}
 */
module.exports = (on, config) => {
  const cucumberOptions = {
    ...browserify.defaultOptions,
    typescript: resolve.sync("typescript", { baseDir: config.projectRoot }),
  };

  cucumberOptions.browserifyOptions.transform.unshift([
    // There is absolutely a better way to share these resources, but this is the best
    // I can do given the constraints of git, file links, and the monorepo we live with
    path.resolve("..", "..", "..", "e2e", "cypress", "plugins", "aliaser.js"),
    {
      aliases: tsConfig.compilerOptions.paths,
      baseDir: path.resolve("..", "..", ".."),
    },
  ]);

  if (config.testingType === "component") {
    const { startDevServer } = require("@cypress/webpack-dev-server");

    on("dev-server:start", (options) => startDevServer({ options, webpackConfig: webpackOptions }));
  } else {
    on("file:preprocessor", (file) =>
      file.filePath.includes(".feature") ? cucumber(cucumberOptions)(file) : preprocessor({ webpackOptions })(file)
    );
  }
  codeCoverage(on, config);
  return config;
};

support/index.js

// ***********************************************************
// This example support/index.js is processed and
// loaded automatically before your test files.
//
// This is a great place to put global configuration and
// behavior that modifies Cypress.
//
// You can change the location of this file or turn off
// automatically serving support files with the
// 'supportFile' configuration option.
//
// You can read more here:
// https://on.cypress.io/configuration
// ***********************************************************

// Import commands.js using ES2015 syntax:
import "./commands";
// Alternatively you can use CommonJS syntax:
// require('./commands')
import "@cypress/code-coverage/support";
import "cypress-real-events/support";

[cellog]

note: also fails with @cypress/code-coverage version 3.9.11

[cellog]

also fails with latest babel-plugin-istanbul

[Narretz]

Shouldn't this be reported here: https://github.com/cypress-io/code-coverage/issues?

[cellog]

Cypress broke backwards compatibility with the code coverage extension and didn't mention it in the release notes. Also, it wasn't a breaking version (10.x), so is most likely a regression. Otherwise I would have posted it there. Again, Cypress 9.0.0 is just fine, it's 9.1.0 that breaks, @Narretz .

[Narretz]

Maybe this is also fixed by #19239?

There weren't many changes in 9.1 that could have caused this.

[jennifer-shehane]

Please try out 9.1.1 and see if this issue is fixed. We were able to track down many of the bugs that the patch fix addresses, but didn’t have time to verify everything reported. Let us know!

[cellog]

I'll try on Monday, thanks so much for the response!

[tmieulet]

9.1.1 fixed this issue.
Thanks