-
Notifications
You must be signed in to change notification settings - Fork 2.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
cargo package --workspace is not very useful #10948
Comments
This sounds very useful! |
I took a look. First time contributor to cargo so just trying to find the relevant parts of the codebase. It looks to me like
So I believe that here we is where we would need to implement @conradludgate 's proposed solution, and to do so in the order the crates depend on another. This is after a quick look at the code for the very first time ever, so I may be off base. I'm going to keep looking and try to confirm what I think. Also I'm on Zulip if people want to chat/mentor there EDIT: After this Zulip discussion, due to the limited bandwidth of cargo team members new features should be accepted first before opening a PR. I am going to work on some issues raised in that thread that already have momentum to try my first contribution |
One background knowledge: One thing a bit weird is that packaging a crate with all verifications but not publishing the path dependencies. If cargo permit that, it make me feel like we try to convince cargo that our build is verified but only locally. As aforementioned a At this time being, a good tool for releasing crates is cargo-release, created by sunny87 and epage. I haven't tried it but I believe it works. Perhaps @epage can give you more experiences on this topic :) |
BTW, this is somehow related to #9260 (comment). There are more discussions in the wild regarding a more sequential and smooth packaging/publishing process, though I cannot find them now. |
Cargo release is a good tool for workspaces. Unfortunately, we can't use it as-is since we don't have the ability to publish using cargo. However, its something we could fork and fix to work with our own registry |
@jneem and I are looking into this. Some of the aspects we are considering are:
If anybody has opinions on this, feel free to chime in. |
@torhovland taking on a task like this, you might want to coordinate more with the Cargo team. For example, we have Office Hours. One way this could possibly be split up is
For verification, we should be verifying the generated I wonder if we can inject patches rather than changing anything about the code. This can all be done in-memory, rather than writing it out to disk.
I have no preference between whether we package+verify one at a time or package all then verify all. We should likely do separate compilation per verify though. #5931 would speed up the compile times for this.
There are times to depend on old versions of packages.
This isn't relevant to this issue but the follow up one. Let's keep the conversations in each Issue focused and move this over to there. |
Thanks for your input, it's been noted.
Sure, we will show up there. The rough idea we have so far is:
|
Running into a chicken-and-egg problem with the lock files: When packaging, Cargo strips away all path dependencies and generates lock files by expecting to find dependent packages online. So that elements like this can be put into the lock file: [[package]]
name = "my-dep"
version = "0.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "780f1cebed1629e4753a1a38a3c72d30b97ec044f0aef68cb26650a3c5cf363c" But when generating a lock file using just a local path dependency, we do not get any So it seems that packaging and publishing are strongly coupled, and there's no way around packaging+publishing one crate at a time. Unless we introduced something like Any ideas? |
A possible solution is to let |
Assuming we could use the We could of course try to modify the lock file after serialisation, but that doesn't seem very nice. The alternative is to modify the resolver code so it can treat a crate as if it was pulled from a registry. |
We have two problem steps in
|
Problem
Let's say you have a workspace
If you have already published
a
to crates.io, thencargo package --workspace
will complete successfully.Now, you update
a
to0.1.1
, and updateb
to use that new minimum version. If you trycargo package --workspace
again, it will no longer work. You will get an error that0.1.1
was not found.This happens because
cargo package
makes a dummy package for your verification and it strips out all workspace and path information. This means it tries to retrieve thea 0.1.1
from the registry, which it fails to do.Proposed Solution
If you have a workspace where some package
b
depends on another packagea
, wherea
is both specified by a version AND a path, AND that path is within the workspace members, then the following will happen:will make the new dummy project to verify the crates, but it will leave in the path information for
a
withinb
.Notes
In my experience, I've never had a workspace where all crates are independent. There's always at least one that depends on another crate. When managing private registries, it's not uncommon to
cargo package
and upload the.crate
file manually.Currently, it's necessary to package each workspace member individually and upload them in the order they depend.
Ideally, we can run a single
after bumping all versions, then get all of the
.crate
files and upload them in a batch.The text was updated successfully, but these errors were encountered: