Add --ignoreDependencies to package command

[onilton]

For some extension projects, people may use a bundle file instead of the regular node_modules.

Even if we do that, we may get some errors, when running

npm list --production --parseable --depth=99999 --link=true

Since vsce always tries to list/find dependencies as can be seen in issue #246 .

The workaround suggest is to use --yarn, since it will check for a yarn.lock file, and if it don't find, nothing happens and vsce doesn't run npm list command.

Adding node_modules to ignore file won't help because this happens after we already tried to run npm list

I propose here to add an option to package called --ignoreDependencies, so we don't run the getNpmDependencies and instead just return [cwd] in getDependencies

[joaomoreno]

Can you show me what a bundle file means?

[onilton]

Like in here:

https://code.visualstudio.com/api/working-with-extensions/bundling-extension#configure-webpack

 output: {
    // the bundle is stored in the 'dist' folder (check package.json), 📖 -> https://webpack.js.org/configuration/output/
    path: path.resolve(__dirname, 'dist'),
    filename: 'extension.js',
    libraryTarget: 'commonjs2',
    devtoolModuleFilenameTemplate: '../[resource-path]'
  },

The only thing that needs to be packaged is extension.js (bundle with all the required libs).

But vsce still tries to look at node_modules even if it is not necessary (and sometimes this can cause issues).

[joaomoreno]

As mentioned here, once you start bundling, you should simply ignore node_modules using the .vscodeignore file.

[onilton]

@joaomoreno That does not solve the issue. I already have node_modules in my .vscodeignore.

Even if node_modules is included in .vscodeignore , npm list is still run. That's the problem.

Check the following files:

vscode-vsce/src/package.ts

Line 820 in 4960c7c

return getDependencies(cwd, useYarn, dependencyEntryPoints).then(deps => {

vscode-vsce/src/npm.ts

Line 182 in 3d7afdd

export function getDependencies(cwd: string, useYarn = false, packagedDependencies?: string[]): Promise<string[]> {

vscode-vsce/src/npm.ts

Line 57 in 3d7afdd

.then(() => exec('npm list --production --parseable --depth=99999 --loglevel=error', { cwd, maxBuffer: 5000 * 1024 }))

[joaomoreno]

Ah the problem is that it still runs, and it throws an error? What error does it throw, btw?

[onilton]

It doesn't necessarily always show an error. But in some scenarios it can show an error like the one described in the issue #246 . (the workaround proposed is to use --yarn, that doesn't use npm list.

My error looks like the one in the issue, but it is not the same one (although it is also caused by npm list). Unfortunately I can't post my exact error right now (I am not working in the project that have the issue).

My packaging is working perfectly, but npm list show an error for some dependency that is not really important and that forbids packaging.

Actually, I've never even run npm list before I found this issue.

[joaomoreno]

So... npm install works without an issue, yet npm list fails? It would be great to see the error. Also, which npm version are you on?

[onilton]

Npm version, 6.13.4, node version v12.16.1

[onilton]

npm install works without an issue.

bundle.

npm run package (node ./node_modules/vsce/out/vsce package")

Command failed: npm list --production --parseable --depth=99999 --loglevel=error
npm ERR! peer dep missing: @material-ui/core@^3.2.0, required by mui-datatables@2.14.0
npm ERR! peer dep missing: @material-ui/icons@^3.0.1, required by mui-datatables@2.14.0

[pmahend1]

Same issue

 ERROR  Command failed: npm list --production --parseable --depth=99999 --loglevel=error
npm ERR! extraneous: p-map@3.0.0 C:\Source\GitHub\resxpress\node_modules\make-fetch-happen\node_modules\p-map
npm ERR! missing: p-map@^4.0.0, required by cacache@15.0.5

npm run webpack succeeds but this fails

[kube]

I'm having the exact same issue when using a Github Action to deploy a VSCode extension:

It's in a monorepo using Lerna, and my VSCode extension package depends on another library in the repository:

- packages/
  - library/
  - vscode-extension/ # (depends on library)

There are 3 main jobs:

• install (using lerna link, so library is loaded from folder directly)
• build (using webpack, and get a bundle, so no further need for node_modules after that)
• publish (using vsce, which fails because node_modules are not present)

In the publish job, I should not need to load the node_modules cache, which even if fast, is not useful in the process.

---

So IMO this --ignoreDependencies flag would be useful.

[kube]

I think a simple solution here would be to say that if node_modules is present in the .vscodeignore file, there should be no check done about the dependencies.

[YXL76]

I think the flag is useful because I use yarn 2 which has no list command

[joechung-msft]

I was able to work around the issue by downgrading to npm@6 and recreating package-lock.json not to use lockfile version 2. Seems like a problem with the new lockfile version introduced with npm@7.

[onilton]

@joaomoreno do you still think we need more info? I see the label is still there.

[DumpySquare]

what information do we need for this? I've run into the same problem. I have an extension that includes other projects that are referenced directly during development, like below.

npm install works fine, but the npm list --production --parseable --depth=99999 --loglevel=error causes problems because it is finding all the dependencies from the other projects as "extraneous"

	"dependencies": {
		"@f5devcentral/f5-fast-core": "^0.7.0",
		"@types/js-yaml": "^3.12.6",
		"axios": "^0.21.1",
		"f5-conx-core": "file:../f5-conx-core",
		"f5-corkscrew": "file:../f5-corkscrew",
		"js-yaml": "^3.14.1",
		"keytar": "^6.0.1"
	}

ted@r2d2:~/projects/vscode-f5$ vsce package ?
Executing prepublish script 'npm run vscode:prepublish'...

> vscode-f5@3.0.0-beta-1 vscode:prepublish /home/ted/projects/vscode-f5
> npm run compile


> vscode-f5@3.0.0-beta-1 compile /home/ted/projects/vscode-f5
> tsc -p ./

 ERROR  Command failed: npm list --production --parseable --depth=99999 --loglevel=error
npm ERR! extraneous: @types/assert@1.5.2 /home/ted/projects/vscode-f5/node_modules/f5-corkscrew/node_modules/@types/assert
npm ERR! extraneous: @types/mocha@8.0.3 /home/ted/projects/vscode-f5/node_modules/f5-corkscrew/node_modules/@types/mocha
npm ERR! extraneous: @typescript-eslint/eslint-plugin@4.4.1 /home/ted/projects/vscode-f5/node_modules/f5-corkscrew/node_modules/@typescript-eslint/eslint-plugin
npm ERR! extraneous: @typescript-eslint/parser@4.4.1 /home/ted/projects/vscode-f5/node_modules/f5-corkscrew/node_modules/@typescript-eslint/parser
npm ERR! extraneous: assert@2.0.0 /home/ted/projects/vscode-f5/node_modules/f5-corkscrew/node_modules/assert
npm ERR! extraneous: eslint@7.11.0 /home/ted/projects/vscode-f5/node_modules/f5-corkscrew/node_modules/eslint
npm ERR! extraneous: mocha@8.2.1 /home/ted/projects/vscode-f5/node_modules/f5-corkscrew/node_modules/mocha
npm ERR! extraneous: nyc@15.1.0 /home/ted/projects/vscode-f5/node_modules/f5-corkscrew/node_modules/nyc
npm ERR! extraneous: ts-node@9.1.1 /home/ted/projects/vscode-f5/node_modules/f5-corkscrew/node_modules/ts-node
npm ERR! extraneous: typescript@4.1.3 /home/ted/projects/vscode-f5/node_modules/f5-corkscrew/node_modules/typescript

*****tried using yarn, but failed*****
ted@r2d2:~/projects/vscode-f5$ vsce package --yarn
Executing prepublish script 'yarn run vscode:prepublish'...
/bin/sh: 1: yarn: not found
 ERROR  yarn failed with exit code 127

*****removed ``"vscode:prepublish.old": "npm run compile",`` from package.json scripts, now yarn works*****
ted@r2d2:~/projects/vscode-f5$ vsce package --yarn
 DONE  Packaged: /home/ted/projects/vscode-f5/vscode-f5-3.0.0-beta-1.vsix (78 files, 5.39MB)
 INFO  
The latest version of vsce is 1.86.0 and you have 1.85.0.
Update it now: npm install -g vsce
ted@r2d2:~/projects/vscode-f5$ 

[Livven]

@joaomoreno npm list seems to perform much more validation than npm install, not all of which is actually relevant, such as peer dependencies missing or being included in different versions (that still work) and so on. Especially if the JS is bundled these validations are completely unnecessary anyway.

For now using --yarn is effective as a workaround, but a more "semantic" flag to disable the npm list call would be appreciated.

[joaomoreno]

Will add a --no-dependencies flag.

[github-actions]

🎉 This issue has been resolved in version 1.102.0 🎉

The release is available on:

• npm package (@latest dist-tag)
• GitHub release

Your semantic-release bot 📦🚀

[pkit]

@joaomoreno please update the docs about bundling here
As these must be run with --no-dependencies for bundling to work.
Otherwise either node_modules will still be included and it will still complain about bundling size:

This extension consists of 1614 files, out of which 1029 are JavaScript files. 
For performance reasons, you should bundle your extension: https://aka.ms/vscode-bundle-extension . 
You should also exclude unnecessary files by adding them to your .vscodeignore: https://aka.ms/vscode-vscodeignore

Or it will just die with the error from #439 (comment)

[ssbarnea]

I think that when this bug was fixed the ability to load the dependencies from package.json file did not work.

Based on REAME.md file, we were supposed to be able to configure inside package.json:

{
  ...
  "vsce": {
    "dependencies": false,
    "useYarn": false
  }
}

While useYarn works as expected and allow you to avoid having to add --no-yarn, the dependencies option has not effect at all, preventing you from fixing the package command from inside config.

That is indeed a big deal if you do not call vsce package yourself and not having ability to add the required extra arguments.

I tried to look at the d797b8d to spot the bug but I was not able to find exact what was missed.

@joaomoreno If you could double check this and maybe give a hint on why setting this option in package.json does not work it would be awesome. I was hit by this while trying to find a workaround for redhat-developer/vscode-extension-tester#451

[joaomoreno]

@ssbarnea Do you mind creating an issue explaining what exactly the bug is?

[ssbarnea]

Raise at #742 and PR will appear soon.