Infinite rebuild loop in vagrant environment in watch mode #12340

thecreazy · 2021-01-05T16:22:14Z

Bug report

What is the current behavior?
Using Webpack 5.11.1 (update from 4) inside a vagrant machine on macOS i have a loop of rebuild in watch mode,
for example:

starting the client:watch with npx webpack --devtool eval-source-map --config webpack.client.js --watch
first build works
start a lot of rebuild because inside the compiler.removedFiles variables found the /src/common/js/modules path that is not really deleted from fs

If the current behavior is a bug, please provide the steps to reproduce.

this is our configuration:

{
  mode: 'development',
  target: 'web',
  watchOptions: { poll: 500, ignored: [ 'public', 'build' ] },
  watch: true,
  devtool: false,
  entry: {
    'bundle.0.0.1': [
      '/var/www/hs-goldenrod/wls/PROJECTNAME/src/js/client.js'
    ]
  },
   resolve: {
    extensions: [ '.js', '.jsx', '.scss', '.css' ],
    alias: {
      'module-analytics': '/var/www/PROJECTNAME/src/packages/module-analytics',
      ...
    },
    fallback: {
      fs: false,
      net: false,
      tls: false,
      console: false,
      child_process: false,
      path: '/node_modules/path-browserify/index.js',
      crypto: '/node_modules/crypto-browserify/index.js',
      stream: '/node_modules/stream-browserify/index.js',
      https: '/node_modules/https-browserify/index.js',
      http: '/node_modules/stream-http/index.js',
      zlib: '/node_modules/browserify-zlib/lib/index.js'
    }
  },
   output: {
    path: 'public/js',
    publicPath: '/js/',
    filename: '[name].min.js',
    chunkFilename: '[name].0.0.1.min.js'
  },
  node: {},
  optimization: { splitChunks: { cacheGroups: [Object] }, minimize: true }

the deleted and changed file list is printed by an our Webpack plugin

class WatchRunPlugin {
  // Define `apply` as its prototype method which is supplied with compiler as its argument
  apply(compiler) {
    // Specify the event hook to attach to
    compiler.hooks.watchRun.tap("WatchRunPlugin", (comp) => {
      if (comp.modifiedFiles) {
        const changedFiles = Array.from(
          comp.modifiedFiles,
          (file) => `\n  ${file}`
        ).join("");
        console.log("\n===============================");
        console.log("FILES CHANGED:", changedFiles);
        console.log("===============================");
      }
      if (comp.removedFiles) {
        const removedFiles = Array.from(
          comp.removedFiles,
          (file) => `\n  ${file}`
        ).join("");
        console.log("\n===============================");
        console.log("FILES REMOVED:", removedFiles);
        console.log("===============================");
      }
    });
  }
}

module.exports = () => {
  return new WatchRunPlugin();
};

and this is a example prompt

[webpack-cli] watching files for updates...
3% setup watch run webpack-cli[webpack-cli] Compilation starting...
3% setup watch run WatchRunPlugin
===============================
FILES CHANGED:
  /var/www/hs-goldenrod/wls/PROJECTNAME/src/common/js/server.js
===============================

===============================
FILES REMOVED:
===============================
99% done plugins webpack-cli[webpack-cli] Compilation finished
asset server.bundle.js 24.9 MiB [emitted] (name: server)
asset PropertyOverlay.bundle.js 100 KiB [emitted] (name: PropertyOverlay)
asset DetailsOverlay.bundle.js 96.9 KiB [emitted] (name: DetailsOverlay)
asset AmenitiesFilter.bundle.js 46.1 KiB [emitted] (name: AmenitiesFilter)
asset ReviewsOverlay.bundle.js 32.5 KiB [emitted] (name: ReviewsOverlay)
orphan modules 3.07 MiB [orphan] 542 modules
runtime modules 4.07 KiB 12 modules
javascript modules 7.36 MiB
  modules by path ./node_modules/ 3.69 MiB 623 modules
  modules by path ./src/ 3.66 MiB
    modules by path ./src/packages/ 2.89 MiB 171 modules
    modules by path ./src/common/js/ 789 KiB
      cacheable modules 788 KiB 2 modules
      2 modules
json modules 495 KiB
  modules by path ./node_modules/har-schema/lib/*.json 6.93 KiB 18 modules
  modules by path ./node_modules/iconv-lite/encodings/tables/*.json 86.7 KiB 8 modules
  modules by path ./node_modules/encoding/node_modules/iconv-lite/encodings/tables/*.json 86.7 KiB 8 modules
  modules by path ./node_modules/ajv/lib/refs/*.json 5.58 KiB 3 modules


webpack 5.11.1 compiled with 1 warning in 21320 ms
[webpack-cli] watching files for updates...
3% setup watch run webpack-cli[webpack-cli] Compilation starting...
3% setup watch run WatchRunPlugin
===============================
FILES CHANGED:
===============================

===============================
FILES REMOVED:
  /var/www/hs-goldenrod/wls/PROJECTNAME/src/common/js/modules
===============================
99% done plugins webpack-cli[webpack-cli] Compilation finished
asset server.bundle.js 24.9 MiB [emitted] (name: server)
asset PropertyOverlay.bundle.js 100 KiB [emitted] (name: PropertyOverlay)
asset DetailsOverlay.bundle.js 96.9 KiB [emitted] (name: DetailsOverlay)
asset AmenitiesFilter.bundle.js 46.1 KiB [emitted] (name: AmenitiesFilter)
asset ReviewsOverlay.bundle.js 32.5 KiB [emitted] (name: ReviewsOverlay)
orphan modules 3.07 MiB [orphan] 542 modules
runtime modules 4.07 KiB 12 modules
javascript modules 7.36 MiB
  modules by path ./node_modules/ 3.69 MiB 623 modules
  modules by path ./src/ 3.66 MiB
    modules by path ./src/packages/ 2.89 MiB 171 modules
    modules by path ./src/common/js/ 789 KiB
      cacheable modules 788 KiB 2 modules
      2 modules
json modules 495 KiB
  modules by path ./node_modules/har-schema/lib/*.json 6.93 KiB 18 modules
  modules by path ./node_modules/iconv-lite/encodings/tables/*.json 86.7 KiB 8 modules
  modules by path ./node_modules/encoding/node_modules/iconv-lite/encodings/tables/*.json 86.7 KiB 8 modules
  modules by path ./node_modules/ajv/lib/refs/*.json 5.58 KiB 3 modules
webpack 5.11.1 compiled with 1 warning in 18108 ms

In the first case the rebuild is correct because the edit of the /var/www/hs-goldenrod/wls/PROJECTNAME/src/common/js/server.js file was made by me

but in the second case the rebuild is wrong because nobody delete the folder /var/www/hs-goldenrod/wls/PROJECTNAME/src/common/js/modules and in the fs the folder is present and not touched

In fact the wrong rebuild is not just one but go in a infinite loop that have always the /var/www/hs-goldenrod/wls/PROJECTNAME/src/common/js/modules path inside the deletedFiles

What is the expected behavior?

just rebuild if a file is changed and not if a folder that is not deletes is in deleted list

Other relevant information:
webpack version: 5.11.1
Node.js version: v12.13.1
Operating System: vagrant machine with Debian

The text was updated successfully, but these errors were encountered:

alexander-akait · 2021-01-05T16:29:40Z

Please provide all steps to reproduce, because I can't reproduce

thecreazy · 2021-01-05T16:34:10Z

Please provide all steps to reproduce, because I can't reproduce

Hi @alexander-akait, what tipe of information could be utils for you ? can I pass you all the plugins and loader that I use ?

provide all steps to reproduce

I don't know hot to provide all the steps, is just an issue related with the state of our environment and code :/

alexander-akait · 2021-01-05T16:35:29Z

but in the second case the rebuild is wrong because nobody delete the folder /var/www/hs-goldenrod/wls/PROJECTNAME/src/common/js/modules and in the fs the folder is present and not touched

webpack doesn't remove directories, so something external does this

Hi @alexander-akait, what tipe of information could be utils for you ? can I pass you all the plugins and loader that I use ?

Yes, as minimal - configuration, as maximum reproducible test repo

I don't know hot to provide all the steps, is just an issue related with the state of our environment and code :/

What is command you use to run webpack and setup vagrant environment

thecreazy · 2021-01-05T16:39:46Z

webpack doesn't remove directories, so something external does this

in fact I think that the issue is related to that, nobody delete the folder/file, in fact I can navigate inside the modules folder on the vagrant machine and also on my Mac, but the compiler.removedFiles return this path inside the WatchRunPlugin write by me (the implementation is on the question)

Yes, as minimal - configuration, as maximum reproducible test repo

thx, just some time and I prepare all

alexander-akait · 2021-01-06T13:03:39Z

@thecreazy feel free to ping me when it was ready

thecreazy · 2021-01-07T09:25:09Z

@alexander-akait sorry for the delay but I was looking for a way to give you a better solution

I create a bug example repo https://github.com/thecreazy/webpack-bug-example-repo

in this example, (look the readme), the rebuild is done just two time but the bug is the same, in the deletedFiles we found two files that are not deleted by nobody

in our real repo the files are a lot and the rebuild process is done more time but the issue is the same

let me know if I can help you more

thx

sokra · 2021-01-07T10:14:32Z

Note that all entries in deletedFiles should be absolute paths, so something is putting relative paths into fileDependencies (e. g. via this.addDependency in loaders). I tracked this down to sass-loader: result.stats.includedFiles includes a few relative paths next to a lot absolute paths. This is only happening for .json files and globs imported by sass. There are processed by node-sass-json-importer resp. node-sass-glob-importer. Not sure if these plugins need to do something to provide correct paths to result.stats.includedFiles or if sass-loader need to do something here. To me it looks like that these relative paths are kind of virtual sass modules created by these importer plugins and they should be ignored here. On the other hand the JSON file should be watch and contribute the cache invalidation, so it must be added via addDependency but is not listed as absolute path.

sokra · 2021-01-07T10:26:53Z

importers probably would need to ability to provide their own includedFiles as result.

thecreazy · 2021-01-07T10:59:02Z

so do you think is a bug relative to this plugins ? because with Webpack 4 all works good, maybe are not compatible with 5

sokra · 2021-01-07T11:34:49Z

There is a difference how webpack 4 and 5 handle watching of non-existing files. These invalid paths are treated as non-existing (since fs calls fail).

add automatic workaround for invalid dependencies #12340 #12283

alexander-akait · 2021-02-08T15:43:48Z

@thecreazy here bug in node-sass too, node-sass runs importer before built-in sass resolving and returns relative path for the prev argument for the next importer, fixed in sass (dart-sass), so my recommendation - migrate from node-sass on sass (dart-sass, also you can see node-sass is deprecated). Now we output good warnings, unfortunately nothing to fix on webpack side more, also I prepare patch for mode-sass feel free to feedback webpack-contrib/sass-loader#927

webpack-bot added webpack-5 question labels Jan 5, 2021

sokra added a commit that referenced this issue Jan 7, 2021

add warning when invalid dependencies are reported by loaders/plugins

09862aa

add automatic workaround for invalid dependencies #12340 #12283

This was referenced Jan 11, 2021

Not working with Webpack > 5.11 maoberlehner/node-sass-magic-importer#224

Open

Not working with Webpack > 5.11 pmowrer/node-sass-json-importer#96

Closed

alexander-akait mentioned this issue Feb 8, 2021

fix: compatibility with custom importers for node-sass webpack-contrib/sass-loader#927

Merged

6 tasks

alexander-akait closed this as completed Feb 8, 2021

ZiuChen mentioned this issue Feb 22, 2022

执行npm run dev后项目重复循环构建 trim21/webpack-userscript-template#229

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Infinite rebuild loop in vagrant environment in watch mode #12340

Infinite rebuild loop in vagrant environment in watch mode #12340

thecreazy commented Jan 5, 2021

alexander-akait commented Jan 5, 2021

thecreazy commented Jan 5, 2021

alexander-akait commented Jan 5, 2021

thecreazy commented Jan 5, 2021 •

edited

alexander-akait commented Jan 6, 2021

thecreazy commented Jan 7, 2021

sokra commented Jan 7, 2021

sokra commented Jan 7, 2021

thecreazy commented Jan 7, 2021

sokra commented Jan 7, 2021

alexander-akait commented Feb 8, 2021

Infinite rebuild loop in vagrant environment in watch mode #12340

Infinite rebuild loop in vagrant environment in watch mode #12340

Comments

thecreazy commented Jan 5, 2021

Bug report

alexander-akait commented Jan 5, 2021

thecreazy commented Jan 5, 2021

alexander-akait commented Jan 5, 2021

thecreazy commented Jan 5, 2021 • edited

alexander-akait commented Jan 6, 2021

thecreazy commented Jan 7, 2021

sokra commented Jan 7, 2021

sokra commented Jan 7, 2021

thecreazy commented Jan 7, 2021

sokra commented Jan 7, 2021

alexander-akait commented Feb 8, 2021

thecreazy commented Jan 5, 2021 •

edited