[ng test] When code-coverage is on, sourcemaps for non spec files show JS content instead of TS

[Meligy]

Bug Report or Feature Request (mark with an x)

- [x] bug report -> please search issues before submitting
- [ ] feature request

Versions.

@angular/cli: 1.2.3
node: 8.1.4
os: darwin x64
@angular/animations: 4.3.1
@angular/common: 4.3.1
@angular/compiler: 4.3.1
@angular/core: 4.3.1
@angular/forms: 4.3.1
@angular/http: 4.3.1
@angular/platform-browser: 4.3.1
@angular/platform-browser-dynamic: 4.3.1
@angular/router: 4.3.1
@angular/cli: 1.2.3
@angular/compiler-cli: 4.3.1
@angular/language-service: 4.3.1

(Also tested on Windows 8.1 with Node 6)

Repro steps.

ng new test-app
cd test-app

Go to app.component.ts, and a constructor that looks like:

  constructor() {
    debugger;
  }

Then run ng test --sourcemap

Switch to Chrome test browser, click "Debug", open the devtools, refresh.

Observe how the breakpoint shows right (you see the class that is AppComponent)

Now, stop the test runner, and re-run the command adding --code-coverage to it.
Like ng test --sourcemap --code-coverage

Again, switch to Chrome test browser, click "Debug", open the devtools, refresh.

Observe how the breakpoint shows differently this time:

(shown in the next section)

The log given by the failure.

In the test browser devtools, we see compiled JavaScript as the content of the TypeScript file, like:

Desired functionality.

I'd like to be able to see the correct content of the files in sourcemaps, that is the TypeScript code, not the JavaScript code. Similar to the first screenshot when not using --code-coverage. I'd like --code-coverage to not cause this to break.

Mention any other details that might be useful.

• Note how the filename at the top of the source file tab and the sourcemaps comment at the end of the file content point to the TypeScript file not the JavaScript file!!
  There are 2 sourcemaps comments in the file though, and only happening in the file where the result is wrong.

• This problem does NOT happen in the spec files!!
  I confirmed by adding : AppComponent to one of the tests. It's preserved (plus lambdas etc)
  

• This was working in CLI 1.0.6 (the one that a project I was helping upgrade happened to be upgrading from, signaling the issue, not sure about other versions)

[Meligy]

There's one solution I thought might work, and it didn't. Adding it here for the record anyway.

I went to the src/test.ts file, and changed the line

const context = require.context('./', true, /\.spec\.ts$/);

to be this line, which is primarily needed for proper code coverage results:

const context = require.context('./app', true, /\.ts$/);

I was hoping that would somehow fix the issue as a side effect, but it still didn't change the behavior explained in the main issue.

I thought maybe it was broken because the application file (in the report above it's app.component.ts) was not directly covered by the test.ts line (although it actually was, via import from app.component.spec.ts). But changing the line had no effect on the result.

It might be something to do with how the ng test handles the spec and app tsconfig files maybe. I really don't know.

[Meligy]

Retested with CLI v1.3.0-rc.3.

Same results.

ng version output for last test:

@angular/cli: 1.3.0-rc.3
node: 8.1.4
os: darwin x64
@angular/animations: 4.3.2
@angular/common: 4.3.2
@angular/compiler: 4.3.2
@angular/core: 4.3.2
@angular/forms: 4.3.2
@angular/http: 4.3.2
@angular/platform-browser: 4.3.2
@angular/platform-browser-dynamic: 4.3.2
@angular/router: 4.3.2
@angular/cli: 1.3.0-rc.3
@angular/compiler-cli: 4.3.2
@angular/language-service: 4.3.2

[filipesilva]

Thanks for the report @Meligy, this is unintended and will investigate.

[filipesilva]

I looked into this a bit today but didn't get very far. As far as I can tell, we're passing in the sourcemaps correctly into istanbul-instrumenter-loader, but the sourcemaps that get back from it refer to transpiled code. Need to look into it further.

[filipesilva]

I did some more digging around today, and opened an issue at webpack-contrib/istanbul-instrumenter-loader#73.

[lmssrinivas]

@Meligy @filipesilva I am also facing the same issue, Couldn't have both source-maps and code-coverage at same time. Any work around to fix this?

[Meligy]

The issue does not break the sourcemaps in code coverage report (the HTML files it generates).

For now my team's flow is to run tests without coverage while developing / writing tests, and then before committing their code they run the tests with code coverage once and open the code coverage page to check results. If less than 100%, they run the normal tests again and ensure they cover everything missing.

[csentis]

@hansl Thanks, Hans, for picking this up - highly appreciated.

[escalonn]

It's been a year and webpack-contrib/istanbul-instrumenter-loader#73 is still not addressed - this blocks getting coverage reports in SonarQube for Angular (line numbers in LCOV report must match TS source), which is a very basic requirement.

Is there any workaround, or maybe a different code coverage reporter which can be plugged into Karma? If this is never addressed I think there is a case to move away from Istanbul, or at least this instrumenter, in the basic Angular template. At minimum, to provide an alternative alongside it in the karma.conf.

[Meligy]

This might be too crazy for just a sourcemaps workaround, but if your tests do not use a lot of Karma and/or specific features, you might try the several Jest schematics available for the CLI, since the broken tool is specific to Karma. But again, this is probably going too far too.

[hansl]

Since this only happens while debugging AND using code coverage, because it is a bug with an upstream dependency and webpack has been silent on it, and because it's been happening for a while, we think the issue is not as high a priority as previously triaged. We are going to take a look at it when webpack updates their loaders.

Thanks.

[escalonn]

only happens while debugging AND using code coverage

does this mean it is possible to configure karma to somehow skip "debugging" while running the tests and generating the coverage report via istanbul?

if there is some way to get coverage with correct source maps with jasmine+karma I would love to know about it.

[Meligy]

@escalonn I think what @hansl is saying, is that you'd normally run ng test --sourcemap only for debugging purposes, since this is not the build command, for which you might want to use --sourcemap in prod too sometimes.

[escalonn]

This issue is not limited to using --sourcemap, but is present on any ng test generating istanbul lcovonly report. if you ng new myapp && cd myapp && ng test --code-coverage, the lcov report will not match the TS line numbers.

[arobinson]

I'm having the same issue. I tried adding more reporter options:

      coverageIstanbulReporter: {
        dir: path.join(logDir, 'coverage'),
        reports: ['html', 'lcovonly', 'lcov', 'text-lcov', 'text', 'text-summary'],
        fixWebpackSourcePaths: true
      },

What is interesting is that the line coverage numbers printed to the console are correct, but those produced by lcovonly are wrong. Turning off source maps has no affect, so it seems to be running coverage on the JS files instead

A portion from lcov.info:

FN:6,(anonymous_0)
FN:15,(anonymous_1)
FN:25,(anonymous_2)
FN:28,(anonymous_3)
FN:34,(anonymous_4)
FN:37,(anonymous_5)
FN:40,(anonymous_6)
FN:43,(anonymous_7)
FN:46,(anonymous_8)
FNF:9
FNH:1

The same file output to the console:

FN:4,(anonymous_0)
FN:27,(anonymous_1)
FN:63,(anonymous_2)
FN:55,(anonymous_3)
FN:67,(anonymous_4)
FN:71,(anonymous_5)
FN:75,(anonymous_6)
FN:79,(anonymous_7)
FN:83,(anonymous_8)
FNF:9
FNH:1

Angular CLI: 7.3.9
Node: 10.16.0
OS: darwin x64
Angular: 7.2.15
... animations, common, compiler, compiler-cli, core, forms
... language-service, platform-browser, platform-browser-dynamic
... router

Package                            Version
------------------------------------------------------------
@angular-devkit/architect          0.13.9
@angular-devkit/build-angular      0.13.9
@angular-devkit/build-ng-packagr   0.13.9
@angular-devkit/build-optimizer    0.13.9
@angular-devkit/build-webpack      0.13.9
@angular-devkit/core               7.3.9
@angular-devkit/schematics         7.3.9
@angular/cli                       7.3.9
@ngtools/json-schema               1.1.0
@ngtools/webpack                   7.3.9
@schematics/angular                7.3.9
@schematics/update                 0.13.9
ng-packagr                         4.7.1
rxjs                               6.4.0
typescript                         3.2.4
webpack                            4.29.0

Work-around:
The problem only occurred when including both 'coverage' and 'coverage-istanbul'. When only using the 'coverage-istanbul' reporter without 'coverage' the issue goes away

[angular-automatic-lock-bot]

This issue has been automatically locked due to inactivity.
Please file a new issue if you are encountering a similar or related problem.

Read more about our automatic conversation locking policy.

_{This action has been performed automatically by a bot.}