Handling Workspace Protocol in Peer Dependencies with Lerna

[ma-multipla]

I'm having trouble comprehending the purpose of a specific code fragment. In the context of utilizing the workspace protocol for peer dependencies (e.g. "workspace:^"). Yarn Berry's default behavior is to substitute them with the locally resolved version upon generating the tarball through yarn pack. However, with the current Lerna version, the workspace protocol remains unaltered when applied to peer dependencies, resulting in incompatibility with package managers that don't support the protocol, such as npm.

Using the --allow-peer-dependencies-update flag only bumps the version in specific cases and in my case it only removed the "workspace:" portion, leaving the "^". I've read the documentation about the flag, but I have difficulties seeing the benefits of the current approach.
My questions is as follows:

Why isn't the --allow-peer-dependencies-update flag enabled by default for the workspace protocol and why is it only bumping the workspace protocol when the version is specified like this "workspace:^1.2.0". This essentially makes it not possible to only use the npm range operator without specifying the version, e.g. "workspace:^". Why would you want this?
The documentation is also a bit misleading:

{
  "name": "B",
  "dependencies": {
    "A": "workspace:^1.2.0"   // will update to "workspace:^1.3.0",
    "B": "^0.4.0":            // will update to "^0.5.0"
   },
  "peerDependencies": {
    "A": "workspace:^1.2.0"   // will update to "workspace:^1.3.0"
    "B": ">=0.2.0":           // will not be updateed because range with operator (>=) are skipped
  }
}

https://github.com/lerna-lite/lerna-lite/blob/main/packages/version/README.md#with-flag-enabled

It will update to "^1.3.0" not "workspace:^1.3.0". My expectation would be that either "workspace:^" as well as "workspace:^1.2.0" should lead to a version bump to "^1.3.0" considering the example above.

Here's the code snippet in question:

// Perform certain actions when there are peer dependencies
if (this.peerDependencies?.[depName]) {
  // If the user allows peer bump and it's a regular semver version, add it to the array of dependencies to potentially bump
  // However, don't do this when the semver has a range with an operator, e.g., bump ("^2.0.0") but not (">=2.0.0" or "workspace:<2.0.0" or "workspace:*")
  // prettier-ignore
  if (allowPeerDependenciesUpdate && /^(workspace:)?[~^]?[\d\.]+([\-]+[\w\.\-\+]+)*$/i.test(this.peerDependencies[depName] || '')) {
    updatingDependencies.push(this.peerDependencies);
  }
  // When peer bump is disabled, some peerDependencies might not be reviewed and could still have the `workspace:` prefix, so ensure to remove any of these prefixes
  else if (updatedByCommand === 'publish' && this.peerDependencies[depName].startsWith('workspace:')) {
    this.peerDependencies[depName] = this.peerDependencies[depName].replace('workspace:', '');
  }
}

Source: Lerna Lite Core Package.ts

[ghiscoding]

First of all, this flag never existed in the original Lerna and then to avoid unexpected behavior to unfamiliar users, this feature was added under a flag in the PR #363, there are also examples and unit tests associated with it under that PR. Note that I do not use this flag, even though I created the PR #363. At the time I asked the user who requested this flag to provide more info but barely got any feedback and as far as I know this user still doesn't use the flag, so it's possible that it's incorrect in some use case because of that reason.

I'm having trouble comprehending the purpose of a specific code fragment. In the context of utilizing the workspace protocol for peer dependencies (e.g. "workspace:^"). Yarn Berry's default behavior is to substitute them with the locally resolved version upon generating the tarball through yarn pack. However, with the current Lerna version, the workspace protocol remains unaltered when applied to peer dependencies, resulting in incompatibility with package managers that don't support the protocol, such as npm.

Unless there's a bug, what you said here is incorrect, Lerna-Lite does the substitution but only at the end just before publishing, if you're only executing version, it won't substitute but will instead bump any workspace: in every package.json it finds

Using the --allow-peer-dependencies-update flag only bumps the version in specific cases and in my case it only removed the "workspace:" portion, leaving the "^". I've read the documentation about the flag, but I have difficulties seeing the benefits of the current approach.

Reading your comments, I am assuming that you might not have tried it, perhaps you should give it a try with the --dry-run flag to see what will be produced.

This essentially makes it not possible to only use the npm range operator without specifying the version, e.g. "workspace:^". Why would you want this?
The documentation is also a bit misleading:

Again I assume you did not give it a try, and if you find the docs too confusing, you can always create a PR to make it clearer. Maybe the confusing part is the comment and should be changed to something like this

// will update the version to "workspace:^1.3.0" or "^1.3.0" at publish time

Also note that you were also reading the documentation related to version (only) which does not (and should not) detail anything about publish since that is a separate process and docs. For more details about the publish and workspace: substitution, you can find more info under the publish readme at publish#workspace-protocol

Lastly, I would really suggest you to try with the --dry-run flag (for both version and publish), when run on the publish command, it will also show you a quick summary of the dependencies and the tarball temp folder location (note that Dev deps and others aren't shown in the summary, however you can inspect the tarball directly in the temp folder location that is provided).

Please Note that this entire project is an ongoing effort executed in my spare time (like many other Open Source projects) and I only use it on other Open Source projects of mine, I am not paid for any of these projects including this one. On the contrary, the Lerna project is now run by Nrwl paid employees, so please keep that in mind (in other words, I'm doing my best here).

[ma-multipla]

Thanks for the fast feedback!
I am aware of that. There seem to be some issues or I am using it wrongly. I made a simple repo to use as playground and reproduce it. I also stumbled on another small issue:
--allow-peer-dependencies-update can only be used via lerna.json but not as cli flag, as it's not yet listed here I would assume at first glance. I can make a PR for this.

I did try it, but might have used it wrong.

with or without allowPeerDependenciesUpdate

{
  "devDependencies": {
    "bar": "workspace:^"
  },
  "peerDependencies": {
    "bar": "workspace:^"
  }
}

yarn lerna version --amend --no-private will lead to:

{
  "devDependencies": {
    "bar": "^1.0.1"
  },
  "peerDependencies": {
    "bar": "^"
  }
}

I am not sure there is any case where you would want that when using the workspace protocol. When trying to publish it with
yarn lerna publish --dry-run from-git -y I am getting the following error without allowPeerDependenciesUpdate in lerna.json:

lerna-lite ERR! Error: Invalid tag name "^" of package "foo@^": Tags may not have any characters that encodeURIComponent encodes.
lerna-lite ERR!     at invalidTagName (/Projects/lerna-lite-workspace-protocol/node_modules/npm-package-arg/lib/npa.js:118:15)
lerna-lite ERR!     at fromRegistry (/Projects/lerna-lite-workspace-protocol/node_modules/npm-package-arg/lib/npa.js:426:13)
lerna-lite ERR!     at Function.resolve (/Projects/lerna-lite-workspace-protocol/node_modules/npm-package-arg/lib/npa.js:87:12)
lerna-lite ERR!     at file:///Projects/lerna-lite-workspace-protocol/node_modules/@lerna-lite/core/dist/package-graph/package-graph.js:45:38
lerna-lite ERR!     at Array.forEach (<anonymous>)
lerna-lite ERR!     at file:///Projects/lerna-lite-workspace-protocol/node_modules/@lerna-lite/core/dist/package-graph/package-graph.js:30:44
lerna-lite ERR!     at Map.forEach (<anonymous>)
lerna-lite ERR!     at new PackageGraph (file:///Projects/lerna-lite-workspace-protocol/node_modules/@lerna-lite/core/dist/package-graph/package-graph.js:26:14)
lerna-lite ERR!     at new QueryGraph (file:///Projects/lerna-lite-workspace-protocol/node_modules/@lerna-lite/core/dist/utils/query-graph.js:17:22)
node:internal/process/promises:279
            triggerUncaughtException(err, true /* fromPromise */);

The same publish command works with allowPeerDependenciesUpdate, but you still end up with "bar": "^", I doubt that that should ever be the case. When using allowPeerDependenciesUpdate it works when specifying the version, e.g:

{
  "devDependencies": {
    "bar": "workspace:^"
  },
  "peerDependencies": {
    "bar": "workspace:^1.0.0"
  }
}

yarn lerna version --amend --no-private will lead to:

{
  "devDependencies": {
    "bar": "^1.0.1"
  },
  "peerDependencies": {
    "bar": "^1.0.1"
  }
}

What about making allowPeerDependenciesUpdate the default for the workspace protocol and resolving the restriction of having to specify the version. I understand from the docs that this is not recommended:

Please use with caution when enabling this option, it is not recommended for most users since the npm standard is to never mutate (bump) any peerDependencies when publishing new version in an automated fashion, at least not without a user intervention, as explained by core Lerna maintainer:

https://github.com/lerna-lite/lerna-lite/blob/main/packages/version/README.md#--allow-peer-dependencies-update
But I think it would still be better to have a consistent version bumping behavior between "devDependencies", "dependencies" and "peerDependencies". For me this behavior was confusing.

Please let me know when we are using this wrong. If this is not the intended I would like to help. Anyway thank you for all your efforts you put in this project! We really appreciate your work.

[ghiscoding]

What do you think about using the range operator as indicator for the intended peer dependency version bump, e.g:

Lets assume that the "bar" package has version "1.3.8".

that seems worst and entirely unexpected for the users to end up seeing 1.0.0 as a result. in most cases the users who requested this feature were expecting to see the peerDeps bumped as well (even if workspace: wasn't supported at the time of the peerDeps request, ie this Lerna issue). I also don't want to overcomplicate the logic (it's already complicate) in the package#updateLocalDependency(), the only support I want to provide is the same bump output as any other deps but under the allow flag. So the only thing I would look at is to look at possibly supporting, only if it's doable, the use case and expectation that you first brought up with workspace:^... the other thing is that if you want 1.0.0 in your peerDeps then you might as well just fix it to that version and be done with it "bar": "1.0.0" or "bar": ">=1.0.0" (both are currently supported since the allow flag will not touch any fixed or range versions).

With all this being said, that shows that the idea of making this allow flag as optional (opt-in) was a good idea because not every users would agree with the way that it bumps these peerDeps the way it currently does it

[Marvin1003]

Okay, I think that this behavior would be quite handy and has a useful amount of automation, but I see your point that it's confusing as default and complicates the code. How do you want to handle "workspace:[operator]" for peer dependencies as the current behavior leads to invalid package.jsons? Do you want to throw an error to indicate that it shouldn't be used without a version or use the same logic as the --allow-peer-dependencies-update flag and basically yarn pack? Either is an improvement.

[ghiscoding]

it's too early to say without looking at the code but the goal is to have the same logic for all types of deps (deps, devDeps, peerDeps, ...) produce the same output, that is what I was expected it to do anyway and if the input cannot be parsed or is restricted then its output should be the same as the output (and it's already doing that for fixed and range versions, they are left untouched expect when publishing then any workspace: are being removed)... so anyway, I need to look at the code and debug in order to see what will be returned, there's too much assumptions at the moment

[ghiscoding]

I have converted the Discussion to an issue #590 and opened a PR #591, in it I identified and fixed 2 problems which are mentioned in the PR. I also updated the docs, it would be good if you could do a quick review of the PR #591

Note, the missing flag in the CLI was not addressed in this PR since you said you wanted to contribute a PR

[ma-multipla]

Didn't realize I was commenting on my private account before. Thanks for looking into this. I will take a look at the PR later today and make a PR for the flag.

[ma-multipla]

Some unexpected stuff came up that prevented me from creating the PR and reviewing yours. I will do that next week.