Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable mutation testing again on Stryker's own code #2426

Closed
bartekleon opened this issue Aug 26, 2020 · 5 comments · Fixed by #2475
Closed

Enable mutation testing again on Stryker's own code #2426

bartekleon opened this issue Aug 26, 2020 · 5 comments · Fixed by #2475
Milestone

Comments

@bartekleon
Copy link
Member

replacing stuff in github actions i realised that you can't run mutation testing anymore... it seems there are some problems with configurations:

@stryker-mutator/core: 14:33:11 (2884) INFO ConfigReader Using stryker.conf.js
@stryker-mutator/core: 14:33:11 (2884) INFO ConfigReader Stryker can help you setup a `stryker.conf` file for your project.
@stryker-mutator/core: 14:33:11 (2884) INFO ConfigReader Please execute `stryker init` in your project's root directory.
@stryker-mutator/core: 14:33:11 (2884) ERROR Stryker Invalid config file. Inner error: TypeError: Cannot set property 'module' of undefined
@stryker-mutator/core: TypeError: Cannot set property 'module' of undefined
@stryker-mutator/core:     at Object.<anonymous> (/home/runner/work/stryker/stryker/packages/core/stryker.conf.js:4:27)
@stryker-mutator/core:     at Module._compile (internal/modules/cjs/loader.js:1137:30)
@stryker-mutator/core:     at Object.Module._extensions..js (internal/modules/cjs/loader.js:1157:10)
@stryker-mutator/core:     at Module.load (internal/modules/cjs/loader.js:985:32)
@stryker-mutator/core:     at Function.Module._load (internal/modules/cjs/loader.js:878:14)
@stryker-mutator/core:     at Module.require (internal/modules/cjs/loader.js:1025:19)
@stryker-mutator/core:     at require (internal/modules/cjs/helpers.js:72:18)
@stryker-mutator/core:     at ConfigReader.loadConfigModule (/home/runner/work/stryker/stryker/packages/core/src/config/ConfigReader.js:60:32)
@stryker-mutator/core:     at ConfigReader.readConfig (/home/runner/work/stryker/stryker/packages/core/src/config/ConfigReader.js:26:35)
@stryker-mutator/core:     at readConfig (/home/runner/work/stryker/stryker/packages/core/src/config/readConfig.js:7:25)
@bartekleon bartekleon added the 🐛 Bug Something isn't working label Aug 26, 2020
@bartekleon
Copy link
Member Author

bartekleon commented Aug 26, 2020

This part could be fixed by changing config file name or adding a parameter to CI. We handle stryker.conf... but ours is stryker.parent.conf so it doesn't see it.
Then there is a problem with dashboard option in core.
it could be solved by

settings.dashboard ? settings.dashboard.module = moduleName : settings.dashboard = { module: moduleName };

then problem with build...

settings.buildCommand = 'tsc -b && lerna run build';

then with tests - i can't actually solve this one...
error messages:

@stryker-mutator/core: > @stryker-mutator/core@4.0.0-beta.3 stryker C:\Users\groszek\Desktop\GitHub\stryker\packages\core
@stryker-mutator/core: > node bin/stryker run
@stryker-mutator/core: 17:44:25 (4396) INFO ConfigReader Using stryker.conf.js
@stryker-mutator/core: 17:44:25 (4396) INFO BroadcastReporter Detected that current console does not support the "progress" reporter, downgrading to "progress-append-only" reporter
@stryker-mutator/core: 17:44:25 (4396) WARN InputFileResolver Globbing expression "typings/**/*.ts" did not result in any files.
@stryker-mutator/core: 17:44:25 (4396) INFO InputFileResolver Found 103 of 188 file(s) to be mutated.
@stryker-mutator/core: 17:44:27 (4396) INFO Instrumenter Instrumented 103 source file(s) with 2817 mutant(s)
@stryker-mutator/core: 17:44:27 (4396) INFO ConcurrencyTokenProvider Creating 2 checker process(es) and 2 test runner process(es).
@stryker-mutator/core: 17:44:41 (4396) INFO Sandbox Running build command "tsc -b && lerna run build" in the sandbox at "C:\Users\groszek\Desktop\GitHub\stryker\packages\core\.stryker-tmp\sandbox8835757".
@stryker-mutator/core: 17:44:41 (4396) INFO DryRunExecutor Starting initial test run. This may take a while.
@stryker-mutator/core: 17:44:42 (4396) INFO DryRunExecutor Initial test run succeeded. Ran 0 tests in 17 seconds (net 0 ms, overhead 5 ms).
@stryker-mutator/core: 17:44:42 (4396) ERROR Stryker No tests were executed. Stryker will exit prematurely. Please check your configuration.

@nicojs nicojs added 🎨 Refactoring and removed 🐛 Bug Something isn't working labels Aug 27, 2020
@nicojs nicojs added this to the 4.0 milestone Aug 27, 2020
@nicojs nicojs changed the title can't run mutation tests on epic/mutation-switching Enable mutation testing again on Stryker's own code Aug 27, 2020
@nicojs
Copy link
Member

nicojs commented Aug 27, 2020

Yeah, we didn't update the config for Running Stryker on Stryker. I've changed the title (because it's not an issue with Stryker itself).

@nicojs
Copy link
Member

nicojs commented Aug 27, 2020

@stryker-mutator/core: 17:44:41 (4396) INFO Sandbox Running build command "tsc -b && lerna run build" in the sandbox at "C:\Users\groszek\Desktop\GitHub\stryker\packages\core.stryker-tmp\sandbox8835757".

That command actually failed, but because you're running on windows and nicojs/node-link-parent-bin#20 you don't notice it.

I think "tsc -b" should be the build command of a sub package.

@bartekleon
Copy link
Member Author

@nicojs if you were able to fix it I would be grateful, these reports would be very handy in refactoring and optimising code :)

@nicojs
Copy link
Member

nicojs commented Aug 29, 2020

I'm running into this issue:

nicojs@nicoj03:~/stryker/packages/core$ npm run stryker

> @stryker-mutator/core@4.0.0-beta.4 stryker /home/nicojs/stryker/packages/core
> node bin/stryker run

17:54:44 (15197) INFO ConfigReader Using stryker.conf.js
17:54:45 (15197) WARN InputFileResolver Globbing expression "typings/**/*.ts" did not result in any files.
17:54:45 (15197) INFO InputFileResolver Found 104 of 189 file(s) to be mutated.
17:54:49 (15197) INFO Instrumenter Instrumented 104 source file(s) with 2830 mutant(s)
17:54:50 (15197) INFO ConcurrencyTokenProvider Creating 2 checker process(es) and 2 test runner process(es).
17:54:52 (15197) INFO Sandbox Running build command "tsc -b" in the sandbox at "/home/nicojs/stryker/packages/core/.stryker-tmp/sandbox8668860".
17:55:16 (15197) ERROR Stryker an error occurred Error: Command failed with exit code 1: tsc -b
src/config/OptionsValidator.ts(108,13): error TS2578: Unused '@ts-expect-error' directive.
src/config/OptionsValidator.ts(120,312): error TS2578: Unused '@ts-expect-error' directive.
    at makeError (/home/nicojs/stryker/packages/core/node_modules/execa/lib/error.js:59:11)
    at handlePromise (/home/nicojs/stryker/packages/core/node_modules/execa/index.js:114:26)

We're getting the "unused @ts-expect-error" because the strip-comments dependency isn't stripping the comments for some reason. See problem 3 discussed in #2438

We need to find another way to deal with typescript compile errors.

nicojs added a commit that referenced this issue Sep 13, 2020
Enable mutation testing in the workflow again for the packages that work again 🎉

Still pending: mocha-runner,  jasmine-runner (#2473) and karma-runner (#2474).

Closes #2426
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants