ngcc: do not parse tsconfig.json more than once #36882

mpk · 2020-05-01T10:16:33Z

🐞 bug report

Affected Package

The issue is caused by package @angular/compiler-cli

Is this a regression?

Yes, the previous version in which this bug was not present was: 8.2

Description

(this issue was separated from issue #36874 (comment))

After upgrading Angular 8 to Angular 9 (and Ivy), I have a problem with Angular CLI being stuck at "0% compiling" for a few minutes.

In my app, I have 324 npm modules that should be compiled by NGCC. First ng serve after npm install takes some time before all of them are compiled and cached, but that's fine. However, subsequent serves are also very slow (~ 9 minutes).

I have done some digging and found out that for each npm module (before mainNgcc bails out early due to cache), the following line is run (mainNgcc -> getSharedSetup -> readConfiguration):
https://github.com/angular/angular/blob/master/packages/compiler-cli/src/perform_compile.ts#L197

This line parses and resolves application's tsconfig.json file (by walking through the entire project directory, which in my case is ~ 3700 files) and it takes about 1.1s to do so. However, since it is run for each npm module (in my case 324 modules), this results in 1.1s * 324 modules = 6 minutes delay before the actual application start compiling.

I think the tsconfig.json should be parsed and resolved only once to avoid this delay.

🔬 Minimal Reproduction

Create Angular 9 application with many npm modules and large project structure.
Run npm start.

🌍 Your Environment

Angular Version:


Angular CLI: 9.1.4
Node: 12.14.0
OS: win32 x64

Angular: 9.1.4
... animations, cli, common, compiler, compiler-cli, core, forms
... language-service, localize, platform-browser
... platform-browser-dynamic, router
Ivy Workspace: Yes

Package                           Version
-----------------------------------------------------------
@angular-devkit/architect         0.801.1
@angular-devkit/build-angular     0.901.4
@angular-devkit/build-optimizer   0.901.4
@angular-devkit/build-webpack     0.901.4
@angular-devkit/core              8.1.1
@angular-devkit/schematics        8.1.1
@angular/cdk                      9.2.1
@angular/material                 9.2.1
@ngtools/webpack                  9.1.4
@schematics/angular               8.1.1
@schematics/update                0.901.4
rxjs                              6.5.5
typescript                        3.8.3
webpack                           4.42.0

The text was updated successfully, but these errors were encountered:

petebacondarwin · 2020-05-04T11:06:55Z

Thanks for creating this issue @mpk - I hope to look into it later this week.

mohyeid · 2020-05-05T21:00:03Z

Is this a duplicate of #36272?

petebacondarwin · 2020-05-24T19:52:44Z

Is this a duplicate of #36272?

Not really. There are a number of reasons why the build can be slow at 0%. This is one particular reason.

@mpk - I would like to understand why parsing the config file content (i.e. calling ts.parseJsonConfigFileContent()) requires the entire file-system to be processed. Would you be willing to share a reproduction with me so that I could debug it?

mpk · 2020-05-26T08:04:53Z

@petebacondarwin I have created a project that demonstrates the problem: https://github.com/mpk/ng-many-components

The project has around 1000 components in many directories - this is important, because TS seems to spend a lot of time visiting those directories. When I generated components to a single directory, it was a lot faster.
To simulate many npm modules, I have imported ~180 Angular locale files to main.ts
On my machine, parseJsonConfigFileContent took 250-300ms to process every compiled module in this project

petebacondarwin · 2020-06-03T15:36:44Z

I cloned this repo...
On my machine the readConfiguration() function is called 175 times at an average of ~50ms per call. So this does add an extra 8 secs to the build.

This commit will store a cached copy of the parsed tsconfig that can be reused if the tsconfig path is the same. This will improve the ngcc "noop" case, where there is no processing to do, when the entry-points have already been processed. Previously we were parsing this config every time we checked for entry-points to process, which can take up to seconds in some cases. Resolves angular#36882

petebacondarwin · 2020-06-03T17:10:37Z

Hopefully #37417 will fix this for you. Once the CI has run, you should be able to try using the output from it, although you might need to update your project to v10 first.

This commit will store a cached copy of the parsed tsconfig that can be reused if the tsconfig path is the same. This will improve the ngcc "noop" case, where there is no processing to do, when the entry-points have already been processed. Previously we were parsing this config every time we checked for entry-points to process, which can take up to seconds in some cases. Resolves #36882 PR Close #37417

steverob2k · 2020-06-07T10:49:35Z

Please can this be added to a v9 release.

petebacondarwin · 2020-06-07T11:16:17Z

Looking into it... there should still be a 9.1.x release on Wednesday

This commit will store a cached copy of the parsed tsconfig that can be reused if the tsconfig path is the same. This will improve the ngcc "noop" case, where there is no processing to do, when the entry-points have already been processed. Previously we were parsing this config every time we checked for entry-points to process, which can take up to seconds in some cases. Cherry-picked from angular#37417 (6e7bd93). Resolves angular#36882

petebacondarwin · 2020-06-08T09:56:02Z

9.1.x is not the LTS branch - so I have created a separate PR for it: #37484

steverob2k · 2020-06-08T17:36:36Z

@petebacondarwin Thank you so much for this. Really appreciate your time and effort looking at this issue

mpk · 2020-06-12T10:50:37Z

@petebacondarwin Thank you for the fix, I have verified the 6 minute delay is gone and compilation times have returned to Angular 8 level.

petebacondarwin · 2020-06-12T10:53:52Z

🎉 I am really pleased. Thanks for the update.

steverob2k · 2020-06-12T20:23:55Z

Can also confirm that this has fixed the issue for me. Thanks once again.

This commit will store a cached copy of the parsed tsconfig that can be reused if the tsconfig path is the same. This will improve the ngcc "noop" case, where there is no processing to do, when the entry-points have already been processed. Previously we were parsing this config every time we checked for entry-points to process, which can take up to seconds in some cases. Resolves angular#36882 PR Close angular#37417

angular-automatic-lock-bot · 2020-07-13T16:13:51Z

This issue has been automatically locked due to inactivity.
Please file a new issue if you are encountering a similar or related problem.

Read more about our automatic conversation locking policy.

_{This action has been performed automatically by a bot.}

This commit will store a cached copy of the parsed tsconfig that can be reused if the tsconfig path is the same. This will improve the ngcc "noop" case, where there is no processing to do, when the entry-points have already been processed. Previously we were parsing this config every time we checked for entry-points to process, which can take up to seconds in some cases. Resolves angular#36882 PR Close angular#37417

mpk mentioned this issue May 1, 2020

CLI is stuck at "0% compiling" for some time #36874

Closed

2 tasks

petebacondarwin added comp: ngcc area: performance type: bug/fix labels May 1, 2020

ngbot bot added this to the needsTriage milestone May 1, 2020

petebacondarwin self-assigned this May 4, 2020

pkozlowski-opensource removed the type: needs investigation label May 19, 2020

petebacondarwin changed the title ~~Angular CLI stuck at 0% compiling with large project~~ ngcc: do not parse tsconfig.json more than once May 24, 2020

petebacondarwin added needs reproduction This issue needs a reproduction in order for the team to investigate further triage #1 labels May 24, 2020

petebacondarwin added state: confirmed and removed needs reproduction This issue needs a reproduction in order for the team to investigate further labels Jun 3, 2020

petebacondarwin mentioned this issue Jun 3, 2020

perf(ngcc): cache parsed tsconfig between runs #37417

Closed

petebacondarwin added the state: has PR label Jun 3, 2020

atscott closed this as completed in 6e7bd93 Jun 4, 2020

angular-automatic-lock-bot bot locked and limited conversation to collaborators Jul 13, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ngcc: do not parse tsconfig.json more than once #36882

ngcc: do not parse tsconfig.json more than once #36882

mpk commented May 1, 2020

petebacondarwin commented May 4, 2020

mohyeid commented May 5, 2020

petebacondarwin commented May 24, 2020

mpk commented May 26, 2020 •

edited

petebacondarwin commented Jun 3, 2020

petebacondarwin commented Jun 3, 2020

steverob2k commented Jun 7, 2020

petebacondarwin commented Jun 7, 2020

petebacondarwin commented Jun 8, 2020

steverob2k commented Jun 8, 2020

mpk commented Jun 12, 2020

petebacondarwin commented Jun 12, 2020

steverob2k commented Jun 12, 2020

angular-automatic-lock-bot bot commented Jul 13, 2020

ngcc: do not parse tsconfig.json more than once #36882

ngcc: do not parse tsconfig.json more than once #36882

Comments

mpk commented May 1, 2020

🐞 bug report

Affected Package

Is this a regression?

Description

🔬 Minimal Reproduction

🌍 Your Environment

petebacondarwin commented May 4, 2020

mohyeid commented May 5, 2020

petebacondarwin commented May 24, 2020

mpk commented May 26, 2020 • edited

petebacondarwin commented Jun 3, 2020

petebacondarwin commented Jun 3, 2020

steverob2k commented Jun 7, 2020

petebacondarwin commented Jun 7, 2020

petebacondarwin commented Jun 8, 2020

steverob2k commented Jun 8, 2020

mpk commented Jun 12, 2020

petebacondarwin commented Jun 12, 2020

steverob2k commented Jun 12, 2020

angular-automatic-lock-bot bot commented Jul 13, 2020

mpk commented May 26, 2020 •

edited