[Build Performance] chunk graph very much slow

[chenyiqiao]

What is the current behavior?

I am currently maintaining a SPA using webpack 4.28.2, with about 300 chunks(a lot of async chunks built with dynamic import). Now initial build takes ~80s. Incremental build takes ~30s, which is not acceptable. In both builds, chunk graph step takes ~20s, which is way too long. If i can figure out why chunk graph takes such long time, problem is basically solved for me.

Things I've tried:

• dllplugin : not helpful
• happypack: initial build time from ~100s to ~80s(mostly in building step), but no use for incremental build.

Initial Build

~80s might be normal for a large SPA, But Several items unnormal:

• chunk graph : 18395ms
• after chunk id optimization : 3840ms
• chunk assets processing: 7770ms

44211ms building  
78ms finish module graph
12ms sealing
0ms basic dependencies optimization
0ms dependencies optimization
0ms advanced dependencies optimization
1ms after dependencies optimization
18395ms chunk graph           // too slow
1ms after chunk graph
0ms optimizing
2ms basic module optimization
0ms module optimization
0ms advanced module optimization
0ms after module optimization
676ms basic chunk optimization
1ms chunk optimization
102ms advanced chunk optimization
1ms after chunk optimization
0ms module and chunk tree optimization
0ms after module and chunk tree optimization
0ms basic chunk modules optimization
0ms chunk modules optimization
1ms advanced chunk modules optimization
0ms after chunk modules optimization
0ms module reviving
0ms module order optimization
0ms advanced module order optimization
67ms before module ids
3ms module ids
54ms module id optimization
0ms chunk reviving
16ms chunk order optimization
2ms before chunk ids
0ms chunk id optimization
3840ms after chunk id optimization
1ms record modules
231ms record chunks
157ms hashing
197ms content hashing
0ms after hashing
0ms record hash
3ms module assets processing
7770ms chunk assets processing 
23ms additional chunk assets processing
8ms recording
0ms additional asset processing
1ms chunk asset optimization
0ms after chunk asset optimization
0ms asset optimization
0ms after asset optimization
1ms after seal
839ms emitting
3ms after emitting
｢wdm｣: Hash: 44a77a94ba
Version: webpack 4.28.2
Time: 77017ms

Incremental Build

I just removed a word in a single file, expecting a 3~5s rebuild time, but it takes ~26s.
Several items unnormal:

• chunk graph : 18363ms
• after chunk id optimization: 3788ms

｢wdm｣: Compiling...
1300ms building
170ms finish module graph
31ms sealing
0ms basic dependencies optimization
3ms dependencies optimization
1ms advanced dependencies optimization
0ms after dependencies optimization
18363ms chunk graph                                //  way too slow
4ms after chunk graph
0ms optimizing
0ms basic module optimization
0ms module optimization
0ms advanced module optimization
1ms after module optimization
593ms basic chunk optimization
1ms chunk optimization
182ms advanced chunk optimization
0ms after chunk optimization
1ms module and chunk tree optimization
0ms after module and chunk tree optimization
0ms basic chunk modules optimization
0ms chunk modules optimization
0ms advanced chunk modules optimization
1ms after chunk modules optimization
18ms module reviving
0ms module order optimization
1ms advanced module order optimization
92ms before module ids
3ms module ids
69ms module id optimization
157ms chunk reviving
14ms chunk order optimization
0ms before chunk ids
1ms chunk id optimization
3788ms after chunk id optimization                      // slow
3ms record modules
227ms record chunks
210ms hashing
151ms content hashing
1ms after hashing
0ms record hash
5ms module assets processing
59ms chunk assets processing
47ms additional chunk assets processing
5ms recording
0ms additional asset processing
0ms chunk asset optimization
1ms after chunk asset optimization
0ms asset optimization
0ms after asset optimization
1ms after seal
628ms emitting
0ms after emitting
｢wdm｣: Hash: 646c7756b9
Version: webpack 4.28.2
Time: 26143ms

My webpack config

// webpack.conf.js
module.exports = function(env){
  return {
    mode: env.production ? 'production' : 'development',

    entry: {
      "main": [resolvePath('src/main.js')]
    },
    output: {
      hashDigestLength: 10,
      path: resolvePath('dist'),
      filename: env.production ? 'js/[name].[chunkhash].js' : 'js/[name].js',
      chunkFilename: env.production ? 'js/chunk-[name].[chunkhash].js' : 'js/chunk-[name].js'
    },
    resolve: {
      modules: [resolvePath('node_modules')],
      extensions: ['.js', '.vue', '.json'],
      alias: {
        'vue$': 'vue/dist/vue.esm.js',
        'src': resolvePath('src'),
        'assets': resolvePath('src/assets'),
        'components': resolvePath('src/components'),
        'views': resolvePath('src/views'),
        'utils': resolvePath('src/utils'),
        'vuex': resolvePath('src/vuex'),
        'api': resolvePath('src/api'),
        'node_modules': resolvePath('node_modules')
      }
    },
    module: {
      rules: moduleRules(env)
    },
    optimization: {
      splitChunks: {
        cacheGroups: {
          commons: {
            test: /[\\/]node_modules[\\/]/,
            name: 'vendors',
            chunks: 'all',
            minChunks: 4
          }
        }
      },
      runtimeChunk: {
        name: 'runtime'
      },
      minimizer: [
        new UglifyJsPlugin({
          cache: true,
          parallel: true,
          sourceMap: false,
          uglifyOptions: {
            compress: {
              drop_console: true
            }
          }
        }),
        new OptimizeCSSAssetsPlugin({})
      ]
    },
    devServer: devServer(env),
    plugins: plugins(env),
    devtool: env.production ? false : 'eval-source-map'
  }
}

// module-rule.js
module.exports = function(env) {
  return [
    {
      test: /\.vue$/,
      use: [ 'vue-loader' ]
    },
    {
      test: /\.js$/,
      exclude: [
        resolvePath('static'),
        resolvePath('node_modules')
      ],
      use: [ 'happypack/loader?id=babel' ]
    },
    // normal loader for css
    moduleRulesScss(env),
    moduleRulesCss(env),
    {
      test: /\.(png|jpe?g|gif)(\?\S*)?$/,
      use: [
        useFileLoader('images/')
      ]
    },
    {
      test: /\.(eot|svg|ttf|woff|woff2)(\?\S*)?$/,
      use: [
        useFileLoader('fonts/')
      ]
    }
  ]
}

// plugins
module.exports = function(env) {
  let plugins = [
    new CopyWebpackPlugin([{
      from: resolvePath('static'),
      to: 'static',
      ignore: ['.*']
    }]),

    new webpack.ProvidePlugin({
      '_': 'lodash'
    }),
    new VueLoaderPlugin(),
    new HappyPack({
      id: 'babel',
      loaders: [{
        loader: 'babel-loader?cacheDirectory=true'
      }]
    }),
    new webpack.NamedChunksPlugin()
  ]

  if (env.production) {
    plugins = plugins.concat([
      new HtmlWebpackPlugin({
        filename: resolvePath('dist/index.html'),
        template: 'index.html'
      }),
      new MiniCssExtractPlugin({
        filename: env.production ? 'css/app.[contenthash:10].css' : 'css/app.bundle.css'
      }),
      new webpack.HashedModuleIdsPlugin()
    ])
  } else {
    plugins = plugins.concat([
      new webpack.NamedModulesPlugin(),
      new HtmlWebpackPlugin({
        template: path.join(__dirname, '../index.html'),
        dllScript: '<script src="/dll/dll.js"></script>'
      }),
      new webpack.DllReferencePlugin({
        context: __dirname,
        manifest: require('../dll/manifest.json')
      })
    ])
  }
  return plugins
}

let me know if more info needed.

[chenyiqiao]

According to my test: the more dynamic import routes, the slower webpack build is. I think most of the huge spa with a lot of dynamic import routes would end up with long incremental build.

I updated webpack from v2.7.0 to v4.28.2 recently. When i was using v2.7.0, same problem exists and it turns out I'm not the only one: Slow webpack build time (advanced module optimization). This post solved my problem, but as webpack updates, this plugin doesn't work anymore.

Our solution is to (only in development mode) add a new custom plugin which removes the parents of every module as we don't need this optimization when running the app in our local machines.

According to what this post says, this plugin cheat webpack to think there are no parents to optimize so recompilation time is quite low on development mode(mostly in advanced module optimization step). It seems hacky so I wonder if there is anything i can do to fix it? @sokra

[chenyiqiao]

Problem solved.

For anyone developing a huge app with a lot of async routes, you might need babel-plugin-dynamic-import-node. You can transform import to require in development mode with it. In this way chunk graph is pretty fast and my incremental build time from ~30s to ~3s.

[alexander-akait]

@chenyiqiao great what you can solve problem yourself! so we can close issue?

[edmorley]

The solution above seems like a workaround rather than a fix?
It seems like this should remain open until a performance improvement lands in webpack? :-)

[chenyiqiao]

@edmorley yes you're right, actually it was brought up long ago, but ignored somehow.
#4636

[chenhonghui]

transform import to require in development mode

@chenyiqiao how???

[chenyiqiao]

.babelrc

"env": {
    "development": {
      "plugins": [
        "dynamic-import-node"
      ]
    }
 }

@chenhonghui

[wangxdmm]

@chenyiqiao thanks ...