PackFileCacheStrategy is slow when saving cache for a large project #12418
Comments
|
Firstly - do not ignore the issue template and provide more information about your configuration and project |
|
Also, you have bad plugin: double check your version |
|
@alexander-akait Thanks for replying I have updated the description. I have commented out most of the plugins trying to scope the issue the only one left is |
|
Looks like Webpack is spending most of the time resolving the build dependency. I removed the plugin usages in webpack.config.js when testing but I didn't remove the required statements. After adding some log in |
|
Should help #12419 |
|
@randing89 Anyway can your provide logs? |
|
@alexander-akait I am running webpack with 2.2 GHz 6-Core Intel Core i7 MBP. The CPU is very busy even after the Update: |
Difficult to say. I guess would be that's a measurement problem. The high number of parallel operations may makes measuring a single operation difficult. Could you try to set the concurrency to 1 when measuring it. It's the last argument of I also made some fixes the build dependencies tracking, so could you please try again with the latest release. Setting this configuration might also give more information |
|
@sokra Tried again with 5.14.0 with following changes diff --git a/node_modules/webpack/lib/FileSystemInfo.js b/node_modules/webpack/lib/FileSystemInfo.js
index 590929c..cf2e01b 100644
--- a/node_modules/webpack/lib/FileSystemInfo.js
+++ b/node_modules/webpack/lib/FileSystemInfo.js
@@ -11,6 +11,7 @@ const AsyncQueue = require("./util/AsyncQueue");
const createHash = require("./util/createHash");
const { join, dirname, relative } = require("./util/fs");
const makeSerializable = require("./util/makeSerializable");
+const { performance } = require('perf_hooks');
/** @typedef {import("./WebpackError")} WebpackError */
/** @typedef {import("./logging/Logger").Logger} Logger */
@@ -1117,7 +1118,8 @@ class FileSystemInfo {
const key = `f\n${context}\n${path}`;
if (resolveResults.has(key)) {
return callback();
- }
+ }
+ const start = performance.now();
resolve(
context,
path,
@@ -1127,6 +1129,7 @@ class FileSystemInfo {
missingDependencies: resolveMissing
},
(err, result) => {
+ console.log(`resolve ${path} in ${(performance.now() - start) >> 0}`)
if (expected) {
if (result === expected) {
resolveResults.set(key, result);
@@ -1373,7 +1376,7 @@ class FileSystemInfo {
}
}
},
- 50
+ 1
);
queue.drain = () => {
callback(null, {
After changing 50 to 1 the build takes even more time: |
|
That doesn't make a lot of sense to me. Why should it take multiple seconds to resolve a file?? The same resolving process happens during the build a lot, and doesn't cause issues... Could you try on a different computer? Could you try to disable antivirus software? |
|
@sokra I dug into the code a little bit more. It turns out the After another few hours of digging around, I found removing I guess the Here is the log after the fix. |
|
Hi @sokra ! I stumbled over a similar issue with my update from
For some other projects we also hit times of 296 seconds (
@randing89 What exactly did you change here? |
Bug report
What is the current behavior?
PackFileCacheStrategy took 7 minutes from finishing building to saving the cache file to the disk.
Is there any way to improve performance?
Also, this problem is causing a slow rebuild. If a change is made before the cache is saved Webpack will do a full rebuild instead of incremental.
If the current behavior is a bug, please provide the steps to reproduce.
Delete the cache file and start Webpack build.
What is the expected behavior?
Webpack spend reasonable time to generate the cache.
Other relevant information:
webpack version: 5.13.0
Node.js version: v14.15.1
Operating System: 10.15.7
The text was updated successfully, but these errors were encountered: