Ignore copy errors (don't crash the watch process) #548

felixhao28 · 2020-11-02T07:38:50Z

Operating System: Windows 10
Node Version: v15.0.1
NPM Version: 7.0.3
webpack Version: 5.3.2
copy-webpack-plugin Version: 6.2.1

I am running webpack --watch to build a NodeJS program with systray's binary. systray uses a native executable binary to create an interactive tray icon on the tray/menu depending on the OS. So the executable must be copied into the output folder to be called upon build.

My config:

const path = require('path');
const CopyPlugin = require('copy-webpack-plugin');
const os = require("os");
const bp = path.resolve(__dirname, "..", process.env.BUILDDIR || "build");
let traybinary;
if (os.platform() === "win32") {
    traybinary = "tray_windows_release.exe";
} else if (os.platform() === "darwin") {
    traybinary = "tray_darwin_release";
} else {
    traybinary = "tray_linux_release";
}

module.exports = {
  // ...
  plugins: [
        new CopyPlugin({
            patterns: [{
                from: path.resolve(__dirname, 'node_modules', 'systray', 'traybin', traybinary),
                to: path.resolve(bp, 'traybin', traybinary)
            }]
        })
  ]
}

Expected Behavior / Situation

There should not be any errors or crashes when using webpack --watch. Even if the target file is in use.

Actual Behavior / Situation

Windows locks all files currently in use to prevent modification and deletion. This creates a problem for copy-plugin when it tries to overwrite the locked file. And then this error crashes the webpack --watch process.

Modification Proposal

There are three ways:

Ignore files if they already exist and are identical.

Based on #497, I guess this should already be on by default but I am not sure why it does not work in my case.

Hold the copying/build until the files are unlocked.
Ignore errors like this. Don't let it crash the whole webpack process.

The text was updated successfully, but these errors were encountered:

felixhao28 · 2020-11-02T07:44:02Z

I am currently using an approach suggested by this. This is not only cumbersome for a simple copy action, but event-hooks-webpack-plugin does not support webpack 5 yet, which makes copy-webpack-plugin less performant

alexander-akait · 2020-11-02T10:38:57Z

And what is the problem? We continue working... Just throw error(s)

felixhao28 · 2020-11-02T10:59:52Z

The problem is "npm run watch" (which calls node ./node_modules/webpack/bin/webpack.js --mode development --watch) crashes.

Click to show full log:

The comments surrounded by exclamation marks are made by me.

D:\projects\tfnative2\jsserver>npm run watch

> jsserver@1.0.0 watch
> node ./node_modules/webpack/bin/webpack.js --mode development --watch

[webpack-cli] Compilation starting...
(node:18428) [DEP_WEBPACK_WATCH_WITHOUT_CALLBACK] DeprecationWarning: A 'callback' argument need to be provided to the 'webpack(options, callback)' function when the 'watch' option is set. There is no way to handle the 'watch' option without a callback.
(Use `node --trace-deprecation ...` to show where the warning was created)
[webpack-cli] Compilation finished
assets by path assets/flag-icon-css-master/flags/ 4.51 MiB
  assets by path assets/flag-icon-css-master/flags/4x3/*.svg 2.25 MiB 257 assets
  assets by path assets/flag-icon-css-master/flags/1x1/*.svg 2.26 MiB 257 assets
assets by path assets/bootstrap-select-1.13.9/ 116 KiB
  assets by path assets/bootstrap-select-1.13.9/js/i18n/*.js 55.9 KiB 38 assets
  2 assets
assets by status 11.4 MiB [emitted] 11 assets
assets by path assets/bootstrap-3.4.1-dist/ 391 KiB 9 assets
assets by path assets/icon/*.png 45.2 KiB
  asset assets/icon/bg_login.png 38 KiB [compared for emit] [copied]
  asset assets/icon/logo.png 7.21 KiB [compared for emit] [copied]
assets by path assets/crypto-js/*.js 33 KiB
  asset assets/crypto-js/core.js 23.6 KiB [compared for emit] [copied]
  asset assets/crypto-js/md5.js 9.46 KiB [compared for emit] [copied]
5 assets
runtime modules 791 bytes 4 modules
javascript modules 4.04 MiB
  modules by path ./node_modules/ 3.82 MiB
    cacheable modules 3.82 MiB 586 modules
    5 modules
  modules by path ./src/ 218 KiB
    modules by path ./src/*.ts 120 KiB 13 modules
    2 modules
json modules 715 KiB
  modules by path ./node_modules/har-schema/lib/*.json 6.93 KiB 18 modules
  modules by path ./node_modules/iconv-lite/encodings/tables/*.json 86.7 KiB 8 modules
  modules by path ./node_modules/ajv/lib/refs/*.json 5.58 KiB
    ./node_modules/ajv/lib/refs/json-schema-draft-06.json 2.46 KiB [built] [code generated]
    ./node_modules/ajv/lib/refs/json-schema-draft-07.json 2.72 KiB [built] [code generated]
    ./node_modules/ajv/lib/refs/data.json 409 bytes [built] [code generated]

WARNING in ./node_modules/express/lib/view.js 81:13-25
Critical dependency: the request of a dependency is an expression
 @ ./node_modules/express/lib/application.js 22:11-28
 @ ./node_modules/express/lib/express.js 18:12-36
 @ ./node_modules/express/index.js 11:0-41
 @ ./src/index.ts 30:34-52

WARNING in ./node_modules/keyv/src/index.js 18:14-40
Critical dependency: the request of a dependency is an expression
 @ ./node_modules/cacheable-request/src/index.js 11:13-28
 @ ./node_modules/got/index.js 10:25-53
 @ ./node_modules/download/index.js 10:12-26
 @ ./src/selfLearn.ts 11:35-54
 @ ./src/index.ts 34:20-42

webpack 5.3.2 compiled with 2 warnings in 7522 ms
[webpack-cli] watching files for updates...
# !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
# !!!!!!!!!!!! Webpack is in watch mode now !!!!!!!!!!!!!
# !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
# !!!!!!!!!!!! Now I make a change and save !!!!!!!!!!!!!
# !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
[webpack-cli] Compilation starting...
[webpack-cli] Compilation finished
assets by path assets/flag-icon-css-master/flags/ 4.51 MiB 514 assets
assets by path assets/bootstrap-select-1.13.9/ 116 KiB 40 assets
assets by path assets/bootstrap-3.4.1-dist/ 391 KiB 9 assets
assets by path assets/*.js 586 KiB 6 assets
assets by path assets/*.html 37.4 KiB 4 assets
assets by path assets/icon/*.png 45.2 KiB
  asset assets/icon/bg_login.png 38 KiB [emitted] [copied]
  asset assets/icon/logo.png 7.21 KiB [emitted] [copied]
assets by path assets/crypto-js/*.js 33 KiB
  asset assets/crypto-js/core.js 23.6 KiB [emitted] [copied]
  asset assets/crypto-js/md5.js 9.46 KiB [emitted] [copied]
assets by path assets/*.ico 4.78 KiB
  asset assets/favicon-test.ico 3.19 KiB [emitted] [copied]
  asset assets/favicon.ico 1.6 KiB [emitted] [copied]
4 assets
cached modules 4.68 MiB [cached] 862 modules
runtime modules 791 bytes 4 modules
cacheable modules 58 KiB
  ./src/index.ts 48.1 KiB [built] [code generated]
  ./src/SysTrayControl.ts 9.83 KiB [built] [code generated]

WARNING in ./node_modules/express/lib/view.js 81:13-25
Critical dependency: the request of a dependency is an expression
 @ ./node_modules/express/lib/application.js 22:11-28
 @ ./node_modules/express/lib/express.js 18:12-36
 @ ./node_modules/express/index.js 11:0-41
 @ ./src/index.ts 30:34-52

WARNING in ./node_modules/keyv/src/index.js 18:14-40
Critical dependency: the request of a dependency is an expression
 @ ./node_modules/cacheable-request/src/index.js 11:13-28
 @ ./node_modules/got/index.js 10:25-53
 @ ./node_modules/download/index.js 10:12-26
 @ ./src/selfLearn.ts 11:35-54
 @ ./src/index.ts 34:20-42

webpack 5.3.2 compiled with 2 warnings in 2709 ms
[webpack-cli] watching files for updates...
[webpack-cli] Compilation starting...
# !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
# !!!!!!!!!!!! Now I make another change and save !!!!!!!!!!!!!
# !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
[webpack-cli] Compilation finished
assets by path assets/flag-icon-css-master/flags/ 4.51 MiB 514 assets
assets by path assets/bootstrap-select-1.13.9/ 116 KiB 40 assets
assets by path assets/bootstrap-3.4.1-dist/ 391 KiB 9 assets
assets by path assets/*.js 586 KiB 6 assets
assets by path assets/*.html 37.4 KiB 4 assets
assets by path assets/icon/*.png 45.2 KiB
  asset assets/icon/bg_login.png 38 KiB [emitted] [copied]
  asset assets/icon/logo.png 7.21 KiB [emitted] [copied]
assets by path assets/crypto-js/*.js 33 KiB
  asset assets/crypto-js/core.js 23.6 KiB [emitted] [copied]
  asset assets/crypto-js/md5.js 9.46 KiB [emitted] [copied]
assets by path assets/*.ico 4.78 KiB
  asset assets/favicon-test.ico 3.19 KiB [emitted] [copied]
  asset assets/favicon.ico 1.6 KiB [emitted] [copied]
4 assets
cached modules 4.68 MiB [cached] 862 modules
runtime modules 791 bytes 4 modules
cacheable modules 58 KiB
  ./src/index.ts 48.1 KiB [built] [code generated]
  ./src/SysTrayControl.ts 9.83 KiB [built] [code generated]

WARNING in ./node_modules/express/lib/view.js 81:13-25
Critical dependency: the request of a dependency is an expression
 @ ./node_modules/express/lib/application.js 22:11-28
 @ ./node_modules/express/lib/express.js 18:12-36
 @ ./node_modules/express/index.js 11:0-41
 @ ./src/index.ts 30:34-52

WARNING in ./node_modules/keyv/src/index.js 18:14-40
Critical dependency: the request of a dependency is an expression
 @ ./node_modules/cacheable-request/src/index.js 11:13-28
 @ ./node_modules/got/index.js 10:25-53
 @ ./node_modules/download/index.js 10:12-26
 @ ./src/selfLearn.ts 11:35-54
 @ ./src/index.ts 34:20-42

webpack 5.3.2 compiled with 2 warnings in 1911 ms
[webpack-cli] watching files for updates...
# !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
# !!!!!! Now I start the program, thus locking the file  !!!!!!
# !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
# !!!!!!!!!!!! Now I make another change and save !!!!!!!!!!!!!
# !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
[webpack-cli] Compilation starting...
[webpack-cli] [Error: EBUSY: resource busy or locked, open 'D:\projects\tfnative2\build\traybin\tray_windows_release.exe'] {
  errno: -4082,
  code: 'EBUSY',
  syscall: 'open',
  path: 'D:\\projects\\tfnative2\\build\\traybin\\tray_windows_release.exe'
}
npm ERR! code 1
npm ERR! path D:\projects\tfnative2\jsserver
npm ERR! command failed
npm ERR! command C:\WINDOWS\system32\cmd.exe /d /s /c "node ./node_modules/webpack/bin/webpack.js --mode development --watch"

npm ERR! A complete log of this run can be found in:
npm ERR!     C:\Users\felix\AppData\Local\npm-cache\_logs\2020-11-02T10_52_58_366Z-debug.log

# !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
# !!!!!!!!!!!!!!!! The webpack process crashes !!!!!!!!!!!!!!!!
# !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

D:\projects\tfnative2\jsserver>echo %ErrorLevel% 
1

# !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
# !!!!!!!!!!!!!!!!!! And the exit code is 1  !!!!!!!!!!!!!!!!!!
# !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

D:\projects\tfnative2\jsserver>

alexander-akait · 2020-11-02T11:05:34Z

@felixhao28 weird, we should continue watching, maybe you can create small example?

felixhao28 · 2020-11-02T13:25:32Z

https://github.com/felixhao28/copy-webpack-plugin-bugrepro

[Command Prompt 1]
npm i
npm run watch

[Command Prompt 2]
node dist\bin\entry.js
# At this point you should see an icon in the tray.

Now open src\index.js and make some irrelevant changes, and save.

Watch Command Prompt 1 fail catastrophically.

alexander-akait · 2020-11-02T17:08:46Z

Thanks, I will look at this tomorrow

alexander-akait · 2020-11-03T16:02:19Z

Reproduced 👍 edge case, but I think ~~we can fix it~~, unfortunately this cannot be fixed, why:

the code that throws this error in graceful-fs and we can't add try/catch
changing behavior will be big breaking change
we should be predictable to avoid problems with stats (stats.assets)
this is very rare edge case and working around it will be big job with many edge cases
fs problems should be not ignored, because in most often means big problems

My recommendation - copy this file before starting webpack watching, just add command to package.json

Feel free to feedback

felixhao28 · 2020-11-04T04:32:34Z

Maybe add an option to ignore errors (default action: crash)?

fs problems should be not ignored

Your first impression was not this:

weird, we should continue watching

We continue working... Just throw error(s)

And this is how normal users feel about it.

alexander-akait · 2020-11-04T11:52:58Z

Maybe add an option to ignore errors (default action: crash)?

Impossible because it require monkey patching fs module, so all fs methods can be broken (it is unsafe)

Your first impression was not this:

Yes, at first I did not think that the error occurred deep in fs/graceful-fs module

alexander-akait closed this as completed Nov 3, 2020

felixhao28 mentioned this issue Nov 14, 2020

When watching, CopyPlugin emits all unchanged files and emits file contents from outdated snapshots #553

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Ignore copy errors (don't crash the watch process) #548

Ignore copy errors (don't crash the watch process) #548

felixhao28 commented Nov 2, 2020

felixhao28 commented Nov 2, 2020

alexander-akait commented Nov 2, 2020 •

edited

felixhao28 commented Nov 2, 2020

alexander-akait commented Nov 2, 2020

felixhao28 commented Nov 2, 2020

alexander-akait commented Nov 2, 2020

alexander-akait commented Nov 3, 2020 •

edited

felixhao28 commented Nov 4, 2020

alexander-akait commented Nov 4, 2020 •

edited

Ignore copy errors (don't crash the watch process) #548

Ignore copy errors (don't crash the watch process) #548

Comments

felixhao28 commented Nov 2, 2020

Expected Behavior / Situation

Actual Behavior / Situation

Modification Proposal

felixhao28 commented Nov 2, 2020

alexander-akait commented Nov 2, 2020 • edited

felixhao28 commented Nov 2, 2020

alexander-akait commented Nov 2, 2020

felixhao28 commented Nov 2, 2020

alexander-akait commented Nov 2, 2020

alexander-akait commented Nov 3, 2020 • edited

felixhao28 commented Nov 4, 2020

alexander-akait commented Nov 4, 2020 • edited

alexander-akait commented Nov 2, 2020 •

edited

alexander-akait commented Nov 3, 2020 •

edited

alexander-akait commented Nov 4, 2020 •

edited