-
Notifications
You must be signed in to change notification settings - Fork 24.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat(ngcc): pause async processing if another process has the lockfile #35131
feat(ngcc): pause async processing if another process has the lockfile #35131
Conversation
724be43
to
cfd0535
Compare
4e8cf5b
to
9210143
Compare
9210143
to
ae578df
Compare
fb27dc1
to
15e611e
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM, except I think we should reset the waiting period when the pid
in the lockfile changes.
Really excited to incorporate this into the CLI workflow \o/
packages/compiler-cli/ngcc/src/execution/single_process_executor.ts
Outdated
Show resolved
Hide resolved
This is needed by ngcc when reading volatile files that may be changed by an external process (e.g. the lockfile).
…ckfile ngcc uses a lockfile to prevent two ngcc instances from executing at the same time. Previously, if a lockfile was found the current process would error and exit. Now, when in async mode, the current process is able to wait for the previous process to release the lockfile before continuing itself.
15e611e
to
5b6b43c
Compare
packages/compiler-cli/src/ngtsc/file_system/test/cached_file_system_spec.ts
Outdated
Show resolved
Hide resolved
]]); | ||
// We need to write to the rawFs to ensure that we don't update the cache at this point | ||
rawFs.writeFile(lockFile.lockFilePath, '444'); | ||
await new Promise(resolve => setTimeout(resolve, 250)); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This looks like a potential source of flakes 😁
Could we use fakeAsync() and tick() to make this test deterministic?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah, I was a little bit worried about this. We cannot use the Angular fakeAsync
stuff, since that is driven via Zone.js and we definitely not going to bring that in here just for testing...
I tried to make the timeouts as reliable as possible so that it would be extremely unlikely for it to flake:
- having proven that the class has already checked the lockfile (and read its value) we are immediately writing the new one synchronously so there is no chance for the
setTimeout
insideLockFile
to be triggered. - we then set a timeout for 250ms before the next check but the class has been configured to check the contents of the lockfile every 100ms. So there should be at least 2 opportunities for a
setTimeout
to be triggered before this testing one is triggered. - The class is setup not to fail until 10 attempts have occurred, which is a full second, so there is very little chance that the 250ms timeout is not triggered before we run out of attempts.
Do you have any better ideas?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Fair enough. I am fine leaving this as is and address it in the future if we start seeing flakes.
For future reference, one way to make this deterministic would be using Jasmine's Clock. There is one twist, because we need to also flush microtasks after each retryDelay
ms, so it would be something like:
const clock = jasmine.clock();
const moveToNextIteration = () => {
clock().tick(100); // Or whatever `retryDelay` we have specified.
return Promise.resolve(); // To wait for microtask queue to be flushed.
};
clock.withMock(async () => {
const rawFs = getFileSystem();
const fs = new CachedFileSystem(rawFs);
const lockFile = new LockFileUnderTest(fs);
fs.writeFile(lockFile.lockFilePath, '188');
const promise = lockFile.lock(async () => lockFile.log.push('fn()'));
expect(lockFile.log).toEqual(['create()']);
expect(lockFile.getLogger().logs.info).toEqual([...]);
rawFs.writeFile(lockFile.lockFilePath, '444');
await moveToNextIteration();
await moveToNextIteration();
expect(lockFile.getLogger().logs.info).toEqual([...]]);
...
});
@petebacondarwin Are you still waiting on a review from @alxhub or is this ready to presubmit/merge? |
Just a presubmit please @kara |
…ckfile (#35131) ngcc uses a lockfile to prevent two ngcc instances from executing at the same time. Previously, if a lockfile was found the current process would error and exit. Now, when in async mode, the current process is able to wait for the previous process to release the lockfile before continuing itself. PR Close #35131
…ckfile (#35131) ngcc uses a lockfile to prevent two ngcc instances from executing at the same time. Previously, if a lockfile was found the current process would error and exit. Now, when in async mode, the current process is able to wait for the previous process to release the lockfile before continuing itself. PR Close #35131
This issue has been automatically locked due to inactivity. Read more about our automatic conversation locking policy. This action has been performed automatically by a bot. |
PR Checklist
Please check if your PR fulfills the following requirements:
PR Type
What kind of change does this PR introduce?
What is the current behavior?
Issue Number: N/A
What is the new behavior?
Does this PR introduce a breaking change?
Other information