vsce package cannot be built with vscode and eslint

[kraabrsg]

Describe the bug

since Version 7.7.0 (i think) several modules cannot be built in a vscode extension (eslint) and have to be defined external (keytar way longer )

> xxxxxxx@0.0.1 esbuild-base
> esbuild  ./client/src/extension.ts --outdir=./client/out --bundle --external:vscode --external:keytar --format=cjs --platform=node && esbuild  ./server/src/server.ts --outdir=./server/out --bundle --external:vscode --external:keytar --format=cjs --platform=node --minify

X [ERROR] No loader is configured for ".node" files: node_modules/@zowe/cli/node_modules/cpu-features/build/Release/cpufeatures.node

    node_modules/@zowe/cli/node_modules/cpu-features/lib/index.js:3:24:
      3 │ const binding = require('../build/Release/cpufeatures.node');
        ╵                         ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

X [ERROR] No loader is configured for ".node" files: node_modules/@zowe/cli/node_modules/ssh2/lib/protocol/crypto/build/Release/sshcrypto.node

    node_modules/@zowe/cli/node_modules/ssh2/lib/protocol/crypto.js:30:20:
      30 │   binding = require('./crypto/build/Release/sshcrypto.node');
         ╵                     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

2 errors
node:child_process:910
    throw err;
    ^

Error: Command failed: C:\Entwicklung\tools\vscodeDevWorkspace\zowe\node_modules\esbuild-windows-64\esbuild.exe ./client/src/extension.ts --outdir=./client/out --bundle --external:vscode --external:keytar --format=cjs --platform=node
    at checkExecSyncError (node:child_process:871:11)
    at Object.execFileSync (node:child_process:907:15)
    at Object.<anonymous> (C:\Entwicklung\tools\vscodeDevWorkspace\zowe\node_modules\esbuild\bin\esbuild:211:28)
    at Module._compile (node:internal/modules/cjs/loader:1159:14)
    at Module._extensions..js (node:internal/modules/cjs/loader:1213:10)
    at Module.load (node:internal/modules/cjs/loader:1037:32)
    at Module._load (node:internal/modules/cjs/loader:878:12)
    at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:81:12)
    at node:internal/main/run_main_module:23:47 {
  status: 1,
  signal: null,
  output: [ null, null, null ],
  pid: 24432,
  stdout: null,
  stderr: null
}

build script (via "vsce package" runs "esbuild-base" and then minify) :

"vscode:prepublish": "npm run esbuild-base -- --minify",
....
"esbuild-base": "esbuild  ./client/src/extension.ts --outdir=./client/out --bundle --external:vscode --external:keytar --format=cjs --platform=node && esbuild  ./server/src/server.ts --outdir=./server/out --bundle --external:vscode --external:keytar --format=cjs --platform=node"

dependencies (we tried: 7.7.0, 7,8,0 and 7.9.0 ) :

		"@zowe/cli": "^7.7.0",
		"@zowe/imperative": "^5.7.2",
		"@zowe/zos-files-for-zowe-sdk": "^7.7.0",
		"@zowe/zos-ftp-for-zowe-cli": "2.1.0",
		"@zowe/zos-jobs-for-zowe-sdk": "^7.7.0",
		"@zowe/zos-tso-for-zowe-sdk": "^7.7.0",
		"@zowe/zosmf-for-zowe-sdk": "^7.7.0",
		"active-win": "^7.7.2",
		"await-notify": "^1.0.1",
		"axios": "0.27.2",
		"keytar": "7.9.0",

To fix this errors we declared these libs as external and included the node packages, which raises the size of vsix file from 6 mb to 48 mb (this could be reduce, by manually tryouts what is needed).

Maybe related to:
#1295

https://github.com/zowe/zowe-cli/blob/master/packages/cli/scripts/preinstall.js

Expected and actual results

how to avoid bundling these libraries (keytar, cpufeatures, sshcrpypto) as external with eslint ?

Describe your environment

WIN 10:
node: 18.12.1
vscode extension

Additional context

Thank you!

[t1m0thyj]

@kraabrsg In addition to keytar, could you try marking the cpu-features and sshcrypto.node libraries as external?

I believe the solution here is:

1. Mark all native dependencies (keytar, cpu-features, sshcrypto.node) as external to prevent your bundler (esbuild or webpack) from trying (and failing) to package them. I'm not exactly sure of the syntax on the ESBuild command to accomplish this, but the idea is similar to what Zowe Explorer does on its Webpack build here.
2. For a VS Code extension, do not bundle these dependencies yourself. They will increase the size of the VSIX as you mentioned, and are unnecessary to bundle because they are not required:
   • keytar - Instead of bundling it yourself, import the version that ships with VS Code as documented here.
   • cpu-features and sshcrypto.node - Optional dependencies that add support for additional SSH ciphers to the Zowe USS SDK for executing SSH commands. Your extension should work fine without having them bundled, unless you specifically import them.
3. Consider removing the @zowe/cli package as a dependency of your extension, which would also remove the cpu-features and sshcrypto.node libraries. For applications like your VSCE that use Zowe CLI as a 3rd party dependency, it is recommended to only use Imperative and the SDK packages, which should contain all the functionality you need without the extra CLI "bloat".

[schoko11]

@t1m0thyj thanks for getting back on this!

we would do 3.) first but can't for now because of #1573

as for 2.) we were not aware of that vscode ships with, we will try that thank you very much for the tip!

As for the optional dependencies, we will have to take a closer look

[kraabrsg]

Hi @t1m0thyj after importing keytar from vscode and remove the zowe sdk cli package (we just need the "real cli" for now) and bundling the two remaining as external, esbuild produces a 1,7mb file -> Awesome small for our quite big extension -> Thank you!