chore: migrate from lerna to lerna-lite

[SimenB]

Summary

This allows us to use workspace protocol.

---

I tried to update to lerna@5 (without the second commit), but it doesn't like workspace in any workspace, not just the packages to be published (probably due to the new useWorkspaces: true flag which then picks out a bunch more workspaces).

Test plan

I've played a bit around locally, but I'll publish a new alpha after landing

[SimenB]

@ghiscoding I'm trying to publish, but I get You cannot publish over the previously published versions: 29.0.0-alpha.1 - seems the version in package.json isn't updated before the publish command is run?

This is a typical diff - changing the versions of (dev) deps, but not the version field of the package itself.

Prompt before publish, fwiw:

[SimenB]

(I'm happy to discuss this over in your repo, btw)

[ghiscoding]

@SimenB
nice to see you're giving this a try, it's working fine in my libs (including Lerna-Lite itself) but I've never used alpha tags and I always use it in fixed version mode. I would suggest to try with the --git-dry-run flag that I added so that you can inspect what the output will be, you can see an example here where it helped in fixing a bug. The workspace: version replacement happens in this updateLocalDependency and is a temp version change, you won't see it change but you will see its output with the git-dry-run mode. Have you check on unpkg what was really published? I wonder if some of the regex that were put in place for the workspace: are working as expected or not, again the dry-run would really help here. Also note that the dry-run will update the changelog (if enabled) but won't commit them, so make sure to rollback once you're done.

So can you post the --git-dry-run result? What's the command you called by the way? You should get something similar to what you see below. I improved the log for tarball publish and I also added the tarball path of the temp folder so you can also inspect it the tgz file that could be sent to the npm publish

lerna-lite notice === Tarball Details ===
lerna-lite notice name:          @lerna-lite/cli
lerna-lite notice version:       1.4.2
lerna-lite notice filename:      lerna-lite-cli-1.4.2.tgz
lerna-lite notice package size:  14.9 kB
lerna-lite notice unpacked size: 68.2 kB
lerna-lite notice shasum:        10fa5b50f555adc46f2c651e579cda8d04037370
lerna-lite notice integrity:     sha512-0hsvQ+NI0URkX[...]8hS6KLOWh5flA==
lerna-lite notice total files:   39
lerna-lite notice --- dry-run details ---
lerna-lite notice temp location: C:\Temp\16f72185-bd88-499d-89d2-1b5756b00df1\lerna-lite-cli-1.4.2.tgz
lerna-lite notice package name: @lerna-lite/cli
lerna-lite notice dependencies:
lerna-lite notice @lerna-lite/core    | ^1.4.2 
lerna-lite notice @lerna-lite/info    | 1.4.2
lerna-lite notice @lerna-lite/init    | ^1.4.2
lerna-lite notice @lerna-lite/publish | 1.4.2
lerna-lite notice @lerna-lite/version | ^1.4.2
lerna-lite notice dedent              | ^0.7.0
lerna-lite notice dotenv              | ^16.0.1
lerna-lite notice import-local        | ^3.1.0
lerna-lite notice npmlog              | ^6.0.2
lerna-lite notice yargs               | ^17.5.1
lerna-lite notice
lerna-lite info dry-run> git checkout -- package.json packages\cli\package.json packages\core\package.json packages\exec\package.json packages\info\package.json packages\init\package.json packages\list\package.json packages\optional-cmd-common\package.json packages\publish\package.json packages\run\package.json packages\version\package.json
Successfully published:
 - @lerna-lite/cli@1.4.2

BTW, the bump is coming from Lerna's default code, I didn't change that behavior but I'm not sure why it doesn't work in your use case. In this old Lerna issue the maintainer asked if he had lifecycle script that could interfere? though it was fixed few years back. What you could try maybe is to rollback 1 of the workspace: to a previous alpha tag and see if it gets bumped properly, then we could see if it's caused by the workspace: or not (I would image that the result is the same since again I didn't touch the bump code).

There's also this commit from evocateur, I know it's for canary but perhaps you tried from multiple branches and have name clashing?

Before publishing to npm, it creates the new version tag by taking the current version, bumping it to the next minor version, adding the provided meta suffix (defaults to alpha) and appending the current git sha (ex: 1.0.0 becomes 1.1.0-alpha.0+81e3b443).

If you have publish canary releases from multiple active development branches in CI,
it is recommended to customize the --preid and --dist-tag <tag> on a per-branch basis to avoid clashing versions.

---

I cloned the Jest repo and ran these 2 commands shown below (it might be helpful to see which command you ran) and it seems fine in git-dry-run mode (I chose custom version since I'm not sure what you took from the prompt)

npx lerna version --git-dry-run

and the publish

npx lerna publish from-package --git-dry-run

[ghiscoding]

done some more testing with the Jest repo and I tried couple of commands, still be nice to see which command you ran.

I see that this command, identifies all packages to be bumped in the prompt but answering Yes really only update 5x package "version" property, it does however update all package dependencies and that is where the mismatch happens.

npx lerna publish --git-dry-run --preid alpha --dist-tag next prerelease

however if we use the 2 separate command (version then publish from-package) then the from-package will only detect the ones that had their "version" updated and only update those. I'm not sure where the mismatch comes from but what I see is that the prompt doesn't match what really gets bump for each package. I'm assuming this is a bug Lerna always had but I can't confirm and there's a lot that I have to learn to understand what is really happening. I wouldn't be surprised that this is a problem only for prereleases since I didn't hear anything from other users

Are you really expecting all packages to be updated as the prompt is showing? or only a few to be updated?

[SimenB]

done some more testing with the Jest repo and I tried couple of commands, still be nice to see which command you ran.

Hi! Thanks for looking into this 😀 Command I'm running is yarn lerna publish prerelease --dist-tag next --pre-dist-tag next.

I see that this command, identifies all packages to be bumped in the prompt but answering Yes really only update 5x package "version" property, it does however update all package dependencies and that is where the mismatch happens.

Yeah, that seems consistent with the diff in the repo after publishing fails.

Are you really expecting all packages to be updated as the prompt is showing? or only a few to be updated?

No, I pasted the diff thing after running with --force-publish, just in case that helped. This is without that flag

Which seems somewhat reasonable looking at the diff in 20d678b (#13097) where the lockfile for most packages are changed. But it's 43 out of 51, so not all.

[SimenB]

Hmm, it seems some packages were published before it failed.

[ghiscoding]

Which seems somewhat reasonable looking at the diff in 20d678b (#13097) where the lockfile for most packages are changed. But it's 43 out of 51, so not all.

Yeah I counted them and there were 43 indeed, so I'm not sure why the prompt says 43 (which seems fair) and then it only applies the change to "version" on just 5 of them. It's this mismatch that I need to find where that comes from and again I wouldn't be surprised that it's only happening on prerelease. I downgraded the version of Lerna-Lite down to 1.2.0 which is when I introduced the workspace: and it does the mismatch as well, so I know it's not something recent that affected this.

Also just to be clear, I assume you used this command in the past for prerelease? I assume you did since there was an alpha.0 and by the way where are you Lerna scripts? I couldn't find them (checked in package.json and GH workflow)

Hmm, it seems some packages were published before it failed.

That was on the first try before you contacted me, right? Perhaps you can check their "version" property and see if they got bumped, it's a bit weird that you got more than 5 but it might be because you used the force publish flag. I checked 1 of them that didn't appear in the list of 5 I saw, the one I checked was babel-plugin-jest-hoist and it did get bumped to 29.0.0-alpha.2

So in summary, I'll look into this with the clone of Jest that I currently have, it does replicate the issue consistently (that is the mismatch between prompt and what it applies). However, I'm not sure how long that will take me to find it because there's a lot of code that I don't understand well in Lerna, so we'll see. I would suggest you keep using the --git-dry-run, that only exist in Lerna-Lite, to see what really gets changed before running the real thing.

[ghiscoding]

@SimenB after troubleshooting with the Jest project, I finally found the problem to be caused by this Lerna PR 2874 - drastically reduce time taken to check for cycles that I replicated in Lerna-Lite (PR 7). I will apply a fix and release a minor version shortly. 😉

[ghiscoding]

@SimenB
It seems all good now after updating to the new Lerna-Lite release v1.10.0, thanks again for using the lib, this is by far the biggest lib to ever switched so far 🍾 I also use Jest on a lot of my libs too, so that seems fair 😉

Cheers and happy coding

oh and BTW, I saw you added the option syncWorkspaceLock but I'm not sure if that will work with your version of yarn, which seems to be yarn 1.x classic isn't it? The yarn command that I associated with that option is the following: yarn install --mode update-lockfile. If that works with other versions than Yarn Berry (3.x), I'd like to know so that I could possible update the --sync-workspace-lock docs if need be, I'm asking because I switched to pnpm so I wasn't 100% sure on the yarn side when I added that sync lock option in Lerna-Lite, it would be great to confirm and thanks for the great feedback :)

https://github.com/facebook/jest/blob/6d19139964fa3a422e4b7d31d85d0443adcefc22/lerna.json#L7-L9

Scratch that I just found that you are using indeed Yarn 3.x. so it should be all good then :)

https://github.com/facebook/jest/blob/6d19139964fa3a422e4b7d31d85d0443adcefc22/package.json#L180-L181

[SimenB]

thanks, was able to publish now!

[SimenB]

Regarding syncWorkspaceLock, looking at 0998187 it seems lockfile is unchanged (probably due to using workspace), so might not need it at all

[ghiscoding]

thanks, was able to publish now!

glad to see that the fork I worked on doesn't delay Jest releases 🙈

Regarding syncWorkspaceLock, looking at 0998187 it seems lockfile is unchanged (probably due to using workspace), so might not need it at all

indeed it seems to have no effect as well in Lerna-Lite itself where I enabled it too. I think that it would probably only update the lock file if you have a mix of deps using workspace: and some not, then in that case it would have to update the lock file. So anyway, there's no harm in keeping it enabled (it's actually a good thing, since it keeps your lock file always up to date)

Cheers

[SimenB]

Thanks for a minimal lerna just for publishing (and no forced nx integration 😅)! 😀

[ghiscoding]

indeed forcing Nx seems like a bad, but expected, marketing decision, it will remain totally optional in Lerna-Lite so no worries 😉

[MirKml]

Nice to see lerna-lite adoption from jest! Such a crucial project for whole javascript world. Keep pushing lerna-lite, @ghiscoding 🚀

[ghiscoding]

@SimenB
Your last release should be a "pre-release" instead of a "Latest" release, right? npm seems ok though, I assume you are not creating the GitHub Release through Lerna-Lite, is that correct? I'm actually also curious what you're using to create your release, I think I remember see you writing on the Yarn discussion board that you use Lerna only for the new version tagging and publishing, is that correct? If so that makes it even more sense to switch to Lerna-Lite, since you barely use anything from Lerna :)

[SimenB]

Ah, good catch 👍 made into prerelease now

[github-actions]

This pull request has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.
Please note this issue tracker is not a help forum. We recommend using StackOverflow or our discord channel for questions.