package-lock file isn't updated properly on lockfileVersion 3

[maybephilipp]

Current Behavior

When package-lock.json file has lockfileVersion===3 then lerna doesn't update packages' version - it causes "No matched version found" error in the end for lerna version command.

Expected Behavior

Correct behaviour: support lockfile v3 as it's used by default in the new npm projects. (change packages.*.version, packages.*.dependencies/devDevendencies.*.version)
It isn't regression, v3 was never supported.

Steps to Reproduce

Can you reproduce this on https://github.com/lerna/repro? - NO
I've created my own: https://github.com/philippmalkov/lerna-lockfile-v3-bug-repro

1. Go to project dir
2. npm i
3. Try lerna version 1.0.1 --no-push
4. You see and error:

lerna ERR! npm ERR! 404  Not Found - GET https://registry.npmjs.org/@maybephilipp%2fpkg3 - Not found
lerna ERR! npm ERR! 404 
lerna ERR! npm ERR! 404  '@maybephilipp/pkg3@^1.0.1' is not in this registry.

It isn't missing package error, it actually comes from that pkg3@^1.0.1 isn't found locally because npm builds its tree from lockfile where root packages show that they're v1.0.0.

If you remove v3 lockfile and replace it with v2 then the same lerna version command will work fine.

Failure Logs / Configuration

• Configuration is in repro repository.

lerna-debug.log

0 silly argv {
0 silly argv   _: [ 'version' ],
0 silly argv   push: false,
0 silly argv   lernaVersion: '6.1.0',
0 silly argv   '$0': '/Users/ileep/WebstormProjects/testing-monorepo-lockfile-v3/node_modules/.bin/lerna',
0 silly argv   bump: '1.0.1'
0 silly argv }
1 notice cli v6.1.0
2 verbose rootPath /Users/ileep/WebstormProjects/testing-monorepo-lockfile-v3
3 info current version 1.0.0
4 silly isAnythingCommitted
5 verbose isAnythingCommitted 1
6 silly getCurrentBranch
7 verbose currentBranch master
8 silly hasTags
9 verbose hasTags false
10 info Assuming all packages changed
11 verbose updated @maybephilipp/pkg1
12 verbose updated @maybephilipp/pkg2
13 verbose updated @maybephilipp/pkg3
14 verbose git-describe undefined => "df4a897"
15 silly git-describe parsed => {"refCount":"3","sha":"df4a897","isDirty":false}
16 info execute Skipping git push
17 info execute Skipping releases
18 silly lifecycle No script for "preversion" in "@maybephilipp/root-pkg", continuing
19 silly lifecycle No script for "preversion" in "@maybephilipp/pkg3", continuing
20 verbose version @maybephilipp/pkg3 has no lockfile. Skipping lockfile update.
21 silly lifecycle No script for "version" in "@maybephilipp/pkg3", continuing
22 silly lifecycle No script for "preversion" in "@maybephilipp/pkg1", continuing
23 silly lifecycle No script for "preversion" in "@maybephilipp/pkg2", continuing
24 verbose version @maybephilipp/pkg1 has no lockfile. Skipping lockfile update.
25 verbose version @maybephilipp/pkg2 has no lockfile. Skipping lockfile update.
26 silly lifecycle No script for "version" in "@maybephilipp/pkg1", continuing
27 silly lifecycle No script for "version" in "@maybephilipp/pkg2", continuing
28 verbose version Updating root package-lock.json
29 error Error: Command failed with exit code 1: npm install --package-lock-only --ignore-scripts
29 error npm ERR! code E404
29 error npm ERR! 404 Not Found - GET https://registry.npmjs.org/@maybephilipp%2fpkg3 - Not found
29 error npm ERR! 404
29 error npm ERR! 404  '@maybephilipp/pkg3@^1.0.1' is not in this registry.
29 error npm ERR! 404
29 error npm ERR! 404 Note that you can also install from a
29 error npm ERR! 404 tarball, folder, http url, or git url.
29 error
29 error npm ERR! A complete log of this run can be found in:
29 error npm ERR!     /Users/ileep/.npm/_logs/2022-12-05T07_25_53_783Z-debug-0.log
29 error     at makeError (/Users/ileep/WebstormProjects/testing-monorepo-lockfile-v3/node_modules/execa/lib/error.js:60:11)
29 error     at handlePromise (/Users/ileep/WebstormProjects/testing-monorepo-lockfile-v3/node_modules/execa/index.js:118:26)
29 error     at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
29 error     at async Promise.all (index 0)

Environment

System:
OS: macOS 13.0
CPU: (10) arm64 Apple M1 Pro
Binaries:
Node: 19.0.1 - ~/.nvm/versions/node/v19.0.1/bin/node
Yarn: 3.2.4 - ~/.nvm/versions/node/v19.0.1/bin/yarn
npm: 9.1.1 - ~/.nvm/versions/node/v19.0.1/bin/npm
Utilities:
Git: 2.37.1 - /usr/bin/git
npmPackages:
lerna: ^6.1.0 => 6.1.0

Relevant issues

Issues for lockfile v2:
#3090
#2832
#3091
#2891

[maybephilipp]

Update on this: lockfile v2 is buggy too. After one version command lerna changes lockfile and version is changed successfully.

BUT after that all consequent version updates fail (the same error: module not found). After investigation I got that Lerna adds .name property to each package inside lockfile. Removing this names from lockfile fixes consequent version updates -_-

[maybephilipp]

Yet another update: after Lerna adds .name it seems like it breaks Lerna updating algorithm itself, because consequent version updates write 'Updating root package-lock.json', but package-lock.json is left unchanged.

Most likely this breaks Lerna's algorithm to detect a package in lockfile.

[rxmarbles]

FYI I discussed this w/ some folks from npm during open office hours and they advised to open an issue as it's two pronged. First here is the link to the open issue in npm npm/cli#5967. and 2nd the feedback is that most likely lerna would need to bump the version in the lockfile as well on top of NPM being defensive.

[azu]

It looks like lerna version + npm 9.4.2+ works again.
e.g. https://github.com/textlint/textlint/actions/runs/4274657356/jobs/7441431353

npm v9.4.2 revert this behavior.
npm/cli#5967 (comment)

📝 Note
Node v18.14.0 (LTS) upgrated to npm 9.x.
Node v18.14.2 (LTS) updated npm 9.5.0.

Node.js	npm	lerna version
v18.14.0	v9.3.1	❌ NG
v18.14.1	v9.3.1	❌ NG
v18.14.2	v9.5.0	✅ OK

[wraithgar]

The underlying issue still stands.

[JamesHenry]

Hi @maybephilipp

Thanks a lot for providing the example repository!

I'm pleased to say that when regenerating your lockfile on node 18.5.0 and npm 9.5.0 and using the latest lerna 6.6.0 everything seems to work for me:

[maybephilipp]

@azu @JamesHenry Thank you for your updates! I confirm that npm 9.4.2+ works like a charm. All updates I made past days (4-5 releases) worked well, all deps and lock-file deps where updated correctly. So yes, this issue is solved. Thanks :)