Set CSP on karma to prevent evalError regression #4706

outsideris · 2021-07-29T12:57:50Z

This karma config set CSP header for browser test to prevent evalError regression in future.

When I test it with mocha@8.3.0, EvalError occurred as I expected.

$ npm start test.browser

> mocha@8.3.0 start
> nps "test.browser"

nps is executing `test.browser` : nps clean build test.browser.unit test.browser.bdd test.browser.tdd test.browser.qunit test.browser.esm test.browser.requirejs test.browser.webpack
nps is executing `clean` : rimraf mocha.js
nps is executing `build` : rollup -c

./browser-entry.js → ./mocha.js...
created ./mocha.js in 7.7s
nps is executing `test.browser.unit` : cross-env NODE_PATH=. karma start --single-run --colors

START:
wrote bundle to /Users/outsider/Dropbox/projects/github/mocha/.karma/Edward.kornet/26f394cf-2f4b-442f-b68d-8aafd4fe616e.rollup.js
29 07 2021 21:51:30.548:INFO [karma-server]: Karma v5.2.3 server started at http://localhost:9876/
29 07 2021 21:51:30.551:INFO [launcher]: Launching browsers ChromeHeadless with concurrency unlimited
29 07 2021 21:51:30.557:INFO [launcher]: Starting browser ChromeHeadless
29 07 2021 21:51:32.127:INFO [Chrome Headless 92.0.4515.107 (Mac OS 10.15.7)]: Connected on socket WA07RiE5vXntQyXyAAAA with id 92272476
Chrome Headless 92.0.4515.107 (Mac OS 10.15.7) ERROR
  Uncaught EvalError: Refused to evaluate a string as JavaScript because 'unsafe-eval' is not an allowed source of script in the following Content Security Policy directive: "script-src https: 'self' 'unsafe-inline'".

  at mocha.js:13554:11

  EvalError: Refused to evaluate a string as JavaScript because 'unsafe-eval' is not an allowed source of script in the following Content Security Policy directive: "script-src https: 'self' 'unsafe-inline'".

      at Function (<anonymous>)
      at mocha.js:13554:11
      at createCommonjsModule (mocha.js:16:6)
      at mocha.js:12845:2
      at mocha.js:4:92
      at mocha.js:5:2

Finished in 0.891 secs / 0 secs @ 21:51:33 GMT+0900 (대한민국 표준시)

SUMMARY:
✔ 0 tests completed
Chrome Headless 92.0.4515.107 (Mac OS 10.15.7) ERROR
  Uncaught TypeError: Cannot read property 'setup' of undefined
  at node_modules/karma-mocha/lib/adapter.js:255:16

  TypeError: Cannot read property 'setup' of undefined
      at node_modules/karma-mocha/lib/adapter.js:255:16
      at node_modules/karma-mocha/lib/adapter.js:256:3

✖ Error while running the tests! Exit code: 1

Chrome Headless 92.0.4515.107 (Mac OS 10.15.7) ERROR
  Uncaught ReferenceError: describe is not defined
  at .karma/Edward.kornet/26f394cf-2f4b-442f-b68d-8aafd4fe616e.rollup.js:13645:2

  ReferenceError: describe is not defined
      at .karma/Edward.kornet/26f394cf-2f4b-442f-b68d-8aafd4fe616e.rollup.js:13645:2
      at .karma/Edward.kornet/26f394cf-2f4b-442f-b68d-8aafd4fe616e.rollup.js:4:77
      at .karma/Edward.kornet/26f394cf-2f4b-442f-b68d-8aafd4fe616e.rollup.js:5:2

✖ Error while running the tests! Exit code: 1

The script called "test.browser.unit" which runs "cross-env NODE_PATH=. karma start --single-run --colors" failed with exit code 1 https://github.com/sezna/nps/blob/master/other/ERRORS_AND_WARNINGS.md#failed-with-exit-code
The script called "test.browser" which runs "nps clean build test.browser.unit test.browser.bdd test.browser.tdd test.browser.qunit test.browser.esm test.browser.requirejs test.browser.webpack" failed with exit code 1 https://github.com/sezna/nps/blob/master/other/ERRORS_AND_WARNINGS.md#failed-with-exit-code

And it is fine with the current master which is version-pinned devDependencies.

However, when I removed version-pinned devDependencies, it is passed as well. So, I'm not sure this config is correct to prevent evalError regression.

outsideris · 2021-07-29T12:58:12Z

@juergba Do you have any idea?

coveralls · 2021-07-29T13:08:27Z

Coverage remained the same at 94.375% when pulling 60ddda6 on outsideris:add-eval-error-test into 02bf13d on mochajs:master.

juergba · 2021-07-29T13:25:31Z

I just got this test from a helpful user:

<!DOCTYPE html>
<html>
  <head>
    <meta http-equiv="Content-Security-Policy" content="script-src http: https: file: 'unsafe-inline'">
    <link rel="stylesheet" href="https://unpkg.com/mocha@9.0.0/mocha.css" />
  </head>
  <body>
    <div id="mocha"></div>
    <!script src="https://unpkg.com/mocha@9.0.3/mocha.js"></script>
    <script src="C:/Users/juerg/Desktop/mocha.js"></script>
    <script>
      mocha.setup('bdd');
      it('works', () => {});
      mocha.run();
    </script>
  </body>
</html>

I don't know this karma configuration, I'm sorry. Can't you just include above test, without changing any karma configuration?

juergba · 2021-07-29T13:34:05Z

However, when I removed version-pinned devDependencies, it is passed as well. So, I'm not sure this config is correct to prevent evalError regression.

Have you created a new browser bundle, after removing the devDependencies?

juergba · 2021-07-30T05:52:26Z

@outsideris I guess you haven't removed the pinned versions correctly.
If I remember well, npm test will fail when no versions have been pinned.
You can check package-lock.json with npm list -all regenerator-runtime.

Signed-off-by: Outsider <outsideris@gmail.com>

outsideris · 2021-08-04T18:41:26Z

I rebased this PR on current master including #4707

I passed browser tests.

And I confirmed npm start test.browser failed without regenerator-runtime version-pinned.

Chrome Headless 92.0.4515.107 (Mac OS 10.15.7) ERROR
  Uncaught EvalError: Refused to evaluate a string as JavaScript because 'unsafe-eval' is not an allowed source of script in the following Content Security Policy directive: "script-src https: 'self' 'unsafe-inline'".

  at mocha.js:13646:11

  EvalError: Refused to evaluate a string as JavaScript because 'unsafe-eval' is not an allowed source of script in the following Content Security Policy directive: "script-src https: 'self' 'unsafe-inline'".

      at Function (<anonymous>)
      at mocha.js:13646:11
      at createCommonjsModule (mocha.js:19:6)
      at mocha.js:12937:2
      at mocha.js:5:92
      at mocha.js:6:2

$ mocha@9.0.3 /Users/outsider/mocha
├─┬ @babel/preset-env@7.14.9
│ └─┬ @babel/plugin-transform-regenerator@7.14.5
│   └─┬ regenerator-transform@0.14.5
│     └─┬ @babel/runtime@7.14.8
│       └── regenerator-runtime@0.13.9
└─┬ rewiremock@3.14.3
  └─┬ babel-runtime@6.26.0
    └── regenerator-runtime@0.11.1

outsideris · 2021-08-04T18:42:14Z

Can't you just include above test, without changing any karma configuration?

Why don't you want to change karma configuration?

Signed-off-by: Outsider <outsideris@gmail.com>

outsideris added the area: browser browser-specific label Jul 29, 2021

outsideris requested a review from a team July 29, 2021 12:57

outsideris added the run-browser-test run browser tests on forked PR if code is safe label Jul 29, 2021

github-actions bot removed the run-browser-test run browser tests on forked PR if code is safe label Jul 29, 2021

add test for evalError with CSP

60ddda6

Signed-off-by: Outsider <outsideris@gmail.com>

outsideris force-pushed the add-eval-error-test branch from 1a21cd7 to 60ddda6 Compare August 4, 2021 17:56

outsideris added the run-browser-test run browser tests on forked PR if code is safe label Aug 4, 2021

github-actions bot removed the run-browser-test run browser tests on forked PR if code is safe label Aug 4, 2021

outsideris marked this pull request as ready for review August 4, 2021 18:42

juergba approved these changes Aug 5, 2021

View reviewed changes

juergba merged commit 09ffc30 into mochajs:master Aug 5, 2021

juergba added qa semver-patch implementation requires increase of "patch" version number; "bug fixes" labels Aug 5, 2021

juergba added this to the next milestone Aug 5, 2021

dhuang612 pushed a commit to dhuang612/mocha that referenced this pull request Aug 10, 2021

Set CSP on karma to prevent 'evalError' regression (mochajs#4706)

c1e7e2a

Signed-off-by: Outsider <outsideris@gmail.com>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Set CSP on karma to prevent evalError regression #4706

Set CSP on karma to prevent evalError regression #4706

outsideris commented Jul 29, 2021

outsideris commented Jul 29, 2021

coveralls commented Jul 29, 2021 •

edited

juergba commented Jul 29, 2021 •

edited

juergba commented Jul 29, 2021

juergba commented Jul 30, 2021

outsideris commented Aug 4, 2021

outsideris commented Aug 4, 2021

Set CSP on karma to prevent evalError regression #4706

Set CSP on karma to prevent evalError regression #4706

Conversation

outsideris commented Jul 29, 2021

outsideris commented Jul 29, 2021

coveralls commented Jul 29, 2021 • edited

juergba commented Jul 29, 2021 • edited

juergba commented Jul 29, 2021

juergba commented Jul 30, 2021

outsideris commented Aug 4, 2021

outsideris commented Aug 4, 2021

coveralls commented Jul 29, 2021 •

edited

juergba commented Jul 29, 2021 •

edited