Allow specifying custom NGCC lockfile location (cannot build in parallel with Ivy)

[Rush]

🚀 feature request

Relevant Package

This feature request is for @angular/compiler-cli

Description

I am running multiple webpack build pipelines simultaneously in CI. It does work without Ivy, but with Ivy (Angular 9 rc 11) it fails with:

(If you are sure no ngcc process is running then you should delete the lockfile at /drone/src/github.com/Rush/angular-app/node_modules/@angular/compiler-cli/ngcc/__ngcc_lock_file__.)

Describe the solution you'd like

This line should allow overwriting the lock file location:

angular/packages/compiler-cli/ngcc/src/execution/lock_file.ts

Line 20 in d15cf60

this.fs.resolve(require.resolve('@angular/compiler-cli/ngcc'), '../__ngcc_lock_file__');

Perhaps like so:

 lockFilePath = process.env.ANGULAR_NGCC_LOCK_FILE_PATH || 
      this.fs.resolve(require.resolve('@angular/compiler-cli/ngcc'), '../__ngcc_lock_file__');

I think this is safe cause all simultaneous compiles are happening in Webpack so they're not conflicting with each other. @petebacondarwin @AndrewKushnir

Not having this would cause my builds running 10x more slowly due to a lot of parallelism.

Describe alternatives you've considered

Disabling the lock file with a config option.

[Toxicable]

If you run ngcc yourself beforehand then the angular cli wont run it as part of it's build step.
That should solve you issue of building multiple apps in parallel

[petebacondarwin]

@Rush - thanks for posting this issue.

From your description there are two scenarios:

1. You are triggering webpack multiple times simultaneously (perhaps on different parts of the project). While webpack might be compiling mutually exclusive source code, ngcc is processing shared packages (that are in node_modules). It is quite possible that two projects will depend upon the same packages, which means that two instances of ngcc could try to process the same package simultaneously. This can lead to corrupted packages whose symptoms are very hard to debug.

2. You are triggering a single build instance but it is spawning its own processes to build multiple parts of the system. While the build system may be able to ensure that it does not try to run ngcc on the same package simultaneously, it still does not prevent the multiple ngcc processes from conflicting. For example, if packages A and B both depend upon package C, then running ngcc on A and B simultaneously may cause ngcc to try to process C twice at the same time, once again leading to corruption.

As @Toxicable points out the correct approach, when you expect to be running parallel builds, is to pre-process your packages via the ngcc command line. E.g. ngcc --properties es2015 browser module main --create-ivy-entry-points. This will actually run in multiple processes that it manages itself to be safe of corruption, and so is no slower than doing it via the Angular CLI integration.

We should update the error message that is presented when the lockfile is found to explain this approach better.

[Rush]

Thank you for the advice. I think it makes sense for NGCC to block simultaneous access to node_modules. Unfortunately just having ngcc did not help completely and one of the pipelines was still failing. Some of my builds are using different targets so maybe that was the problem ....

ngcc --properties es2015 browser module main --create-ivy-entry-points also did not help. Pipelines were failing almost as much as when ngcc was used in the "postinstall" hook.

What has helped though was adding ngcc --create-ivy-entry-points

I can't say I understand what ngcc --create-ivy-entry-points is doing ... I'll be observing it for stability. Thank you!

[gkalpak]

@Rush, glad you got it working! What options you need to pass to ngcc depends on your build configs.

The set of options that emulates the default CLI behavior is ngcc --create-ivy-entry-points --first-only --properties es2015 browser module main. But if you are targeting both es2015 and es5 you might need to run two ngcc command (e.g. one with es2015 and one without).

Running ngcc --help prints some info on the various available options. In a nutshell:

• The --create-ivy-entry-points option tells ngcc to create an new sub-directory for each format of each package and put the processed files there, instead of overwriting the package files in place. (It shouldn't really matter in your case.)
• The --first-only option tells ngcc to only process the first available format (from those specified using the --properties option). This is what @angular/cli uses under the hood, but if you have different projects targeting different formats, then you might need to either omit --first-only (to process all formats for each package) or run ngcc multiple times (to process different formats).
• The --properties option allows you to specify which formats (i.e. which properties in package.json pointing to entry-point files for different formats) you are interested in. It defaults to looking at all supported format properties, but @angular/cli uses es2015 browser module main (when targeting es2015) or browser module main (when targeting es5). (If you are building for the ngUniversal/platform-server, then you will probably need a different set of props (main module iirc).)

I hope this sheds some light. Let us know if you are still having issues (ideally providing a minimal reproduction 🙏).

[jmbarbier]

Having the same issue upgrading from rc.8 to rc.10. ngcc preprocessing was already set in my rc.8 builds and parallel build worked well, but no build can be achieved with rc.10

{"scripts": {
"build:staging": "npm-run-all --parallel build:staging:*",
"postinstall": "ngcc --create-ivy-entry-points"
}}

tried the following combinations

ngcc --create-ivy-entry-points
ngcc --properties es2015 browser module main --create-ivy-entry-points
ngcc --properties es2015 browser module main --create-ivy-entry-points --first-only
ngcc --properties es2015 browser module main --create-ivy-entry-points --first-only && ngcc --properties browser module main --create-ivy-entry-points --first-only
ngcc --properties es2015 browser module main --create-ivy-entry-points && ngcc --properties browser module main --create-ivy-entry-points

ngcc postinstall runs well

ngcc --create-ivy-entry-points
 Compiling @angular/cdk/keycodes : fesm2015 as esm2015
 Compiling @angular/animations : fesm2015 as esm2015
 Compiling @angular/compiler/testing : fesm2015 as esm2015
 Compiling @angular/animations : fesm5 as esm5
 Compiling @angular/core : fesm5 as esm5
 Compiling @angular/animations : esm2015 as esm2015
 Compiling @angular/animations : esm5 as esm5
 Compiling @angular/core : esm5 as esm5
 Compiling @angular/core : main as umd

but on every build i have

npm-run-all --parallel build:staging:*
 > project@2.6.1 build:staging:admin /builds/group/project
 > node --max_old_space_size=3192 node_modules/.bin/ng build admin --configuration=staging
 > project@2.6.1 build:staging:member-front /builds/group/project
 > node --max_old_space_size=3192 node_modules/.bin/ng build member-front --configuration=staging 
 > project@2.6.1 build:staging:member-admin /builds/group/project
 > node --max_old_space_size=3192 node_modules/.bin/ng build member-admin --configuration=staging 
 > project@2.6.1 build:staging:resto-front /builds/group/project
 > node --max_old_space_size=3192 node_modules/.bin/ng build resto-front --configuration=staging 
 > project@2.6.1 build:staging:client-front /builds/group/project
 > node --max_old_space_size=3192 node_modules/.bin/ng build client-front --configuration=staging 
 > project@2.6.1 build:staging:group /builds/group/project
 > node --max_old_space_size=3192 node_modules/.bin/ng build group --configuration=staging 
 ERROR in ngcc is already running at process with id 315.
 (If you are sure no ngcc process is running then you should delete the lockfile at /builds/group/project/node_modules/@angular/compiler-cli/ngcc/__ngcc_lock_file__.)

as if despite preprocessing with ngcc the builds would still launch ngcc...

[gkalpak]

Any chance you could create a reproduction (e.g. a repo we can checkout and run), @jmbarbier?
It is nearly impossible to tell what the problem might be without one 😞

[jmbarbier]

@gkalpak i think i have one 😄 --> https://github.com/solidev/repro-angular-issue-35000

just clone, run npm install then npm run build. You may or may not encounter an error, as it is some race condition, it is somewhat random.

But i have added a simple console.log line in node_modules/@angular/compiler-cli/ngcc/src/execution/lock_file.js to LockFile.prototype.lock function to enhance the locking behavior.

It shows "Locked" message when a lock guarded function is used; we can of course see this lock during "postinstall" ngcc step, but also during parallel builds. Something somewhere during build is requesting a lock, so with parallel builds this can lead to failure(s).

[petebacondarwin]

@jmbarbier - great! thanks for the repro. I had a play with it and I see what the problem is now...

[petebacondarwin]

I added some more logging to your project and I see that ngcc is being triggered for a number of paths. Most of the time we are exiting early from ngcc without locking, as expected. But the following paths have not been marked as processed by the original run:

repro-angular-issue-35000/node_modules/tslib
repro-angular-issue-35000/node_modules/rxjs

and we are locking for those.

[petebacondarwin]

Since these are not Angular packages we need to do one of two things:

a) mark these as processed in the original run
b) exit early if we identify that this is not a package that needs processing

[petebacondarwin]

Working on a fix (b)

[angular-automatic-lock-bot]

This issue has been automatically locked due to inactivity.
Please file a new issue if you are encountering a similar or related problem.

Read more about our automatic conversation locking policy.

_{This action has been performed automatically by a bot.}