Provide a way to run cargo subcommands on all crates in the workspace

[euclio]

This is mentioned as a "future extension" in RFC 1525.

[alexcrichton]

Thanks for the issue @euclio! This was indeed intended to have an issue. And to expand a bit the thought was to add --all flags to many subcommands so executions like:

cargo test --all

would test all the crates in a workspace, or

cargo publish --all

would publish all crates in a workspace that haven't already been published.

[DrRibosome]

just wondering if anyone is still working on this

[alexcrichton]

@jdc2172 I don't believe the currently is, but if you'd like to take it feel free!

[retep998]

cargo doc --all would be exceptionally useful.

[euclio]

I'd be interested in tackling this, but I think we need to hammer out which commands should get this flag.

Also, what sort of implementation approach should be taken?

[steveklabnik]

cargo build and cargo test are the biggies for me.

[DrRibosome]

Cargo test definitely needs this.

Not sure cargo build is as high priority given that it already builds path
deps in the crate

On Fri, Sep 30, 2016, 6:32 PM Steve Klabnik notifications@github.com
wrote:

cargo build and cargo test are the biggies for me.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#2878 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABHmfeh16DgMThxKntuIleFSaX_eVtI-ks5qvY34gaJpZM4JMpzT
.

[steveklabnik]

Oh yeah. That's what I get for hasty commenting! On my project that uses workspaces, those are the only two sub commands I use, so I didn't think it through 😅😞

On Sep 30, 2016, 18:57 -0400, Jack Crawford notifications@github.com, wrote:

Cargo test definitely needs this.

Not sure cargo build is as high priority given that it already builds path
deps in the crate

On Fri, Sep 30, 2016, 6:32 PM Steve Klabnik notifications@github.com
wrote:

cargo build and cargo test are the biggies for me.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#2878 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABHmfeh16DgMThxKntuIleFSaX_eVtI-ks5qvY34gaJpZM4JMpzT
.

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub (#2878 (comment)), or mute the thread (https://github.com/notifications/unsubscribe-auth/AABsijei0_YDV9nu6hCIS8o69VRvBBEFks5qvZPkgaJpZM4JMpzT).

[alexcrichton]

@euclio to me yeah test and doc are the biggies to cover here. I think publish would also be great but that has other tricky semantics we'd want to cover later.

[euclio]

@alexcrichton Trying to get my feet wet with test. What's the best plan of attack here? Modifying ops::compile to take a flag to compile all members of the workspace instead of the current?

[alexcrichton]

@euclio yeah there's a few points where the -p flag is passed down, but that could just be an enum of either a -p flag or a --all flag. When that needs to be resolved we can then just iterate over all packages in a workspace.

[euclio]

@alexcrichton How should I handle the ambiguity when the member has the same name as a dependency?

[alexcrichton]

Oh that's ok, -p is a "package id specification" which will uniquely identify a crate in the dependency graph

[cbiffle]

Is there a reason not to make --all the default in "virtual" workspaces?

It seems like a more reasonable default than the current behavior, which is:

$ cargo build
error: manifest path `/path/embrs/Cargo.toml` is a virtual manifest, but this command requires running against an actual package in this workspace

[alexcrichton]

@cbiffle perhaps yeah, I figured it'd be good to ask for an explicit --all but there's not really any other interpretation of a cargo build against a virtual manifest, so just defaulting seems reasonable.

[Arnavion]

I've had a PS function cargoall in my local profile to do cargoall { cargo test }. Perhaps an approach to add a cargo all subcommand (eg cargo all test or cargo all cargo test) will be preferable to adding --all to every existing subcommand?

function cargoall ([ScriptBlock] $ScriptBlock) {
    (cargo metadata --no-deps | ConvertFrom-Json).packages |
        %{ ([System.IO.FileInfo] $_.manifest_path).Directory } |
        %{
            $_.FullName
            Push-Location $_
            &$ScriptBlock
            Pop-Location
        }
}

(This runs cargo metadata --no-deps, parses the JSON output, then executes the parameter as a command inside the directory of each package.)

[rocallahan]

cargo build --all would be very helpful to us. We have a collection of crates, several of which are --bin, and it would be nice to have a single cargo command to build them all. That would let us get a parallel build.

[sdroege]

cargo build --all (#3511) and cargo doc --all (#3515) also exist now

[Dushistov]

@sdroege
It would be nice to exclude some crates from build --all, for example
I have bunch of crates and one of them specific for android, it depends on android NDK,
and it is possible to build it only with cargo build --target=arm-linux-androideabi,
so cargo build --all failed, it would be nice to have something like cargo build --all --exclude android

[gifnksm]

It would be very nice to have cargo bench --all.

[sdroege]

It would be very nice to have cargo bench --all.

Indeed, I'll take a look at that tomorrow if nobody else is faster. What other commands would be useful to have with an --all variant?

[alexcrichton]

Thanks @sdroege!

[kvark]

@Dushistov

It would be nice to exclude some crates from build --all, for example
I have bunch of crates and one of them specific for android, it depends on android NDK,

This is very applicable to gfx-rs as well, we have dependencies on Windows, OSX, and other stuff, that can't be ON at the same time.

[torkleyy]

I created a PR for the --exclude flag ( #4031 ), would somebody like to review?

[azriel91]

Hi, does this issue cover supporting --all to any arbitrary custom subcommand to run that subcommand across all workspace crates? For example, cargo fmt works on the current crate, but I'd like to have a way to run it against all workspace members.

The way the current code looks (to my untrained eyes), each subcommand determines whether or not it supports running against the workspace via the --all flag. I'm wondering if it makes sense to have a command that switches the order like so:

cargo workspace <subcommand> [opts] [args]
# this would run the following against each crate in the workspace:
cargo subcommand [opts] [args]

That way any custom subcommands will be able to be run against all crates in the workspace without needing to implement it.

[alexcrichton]

@azriel91 I'm not currently thinking that we'd have such a subcommand or interpretation of this, it'd be up to subcommands to interpret and act on --all if they'd like.

[gnzlbg]

cargo fmt --all and cargo clippy --all would be really useful.

[carols10cents]

A bunch of commands have --all flags now, and the remaining ones should get their own issue, like publish does. Any 3rd party subcommands would need to implement this on their own, so issues should be filed on those commands' repos.

So I'm closing this!

[gnzlbg]

@carols10cents

and the remaining ones should get their own issue

Did you fill these issues?

[azriel91]

@gnzlbg that's up to the subcommands (and we probably don't know all of them), at least these subcommands support it:

• build
• test
• clippy (see Add --all flag for a workspace rust-clippy#1707)
• rustfmt

Another cargo command which I found useful is cargo make, which supports running arbitrary commands in each workspace member. The default configuration is pretty good, and you can add your own if you wish.

[gnzlbg]

That's good news, 9 days ago clippy and fmt still did not support --all. I think the only other one missing from the ones mentioned here is cargo bench --all.

[carols10cents]

@gnzlbg

$ cargo --version
cargo 0.21.0 (5b4b8b2ae 2017-08-12)
$ cargo bench --help
Execute all benchmarks of a local package

Usage:
    cargo bench [options] [--] [<args>...]

...
    --all                        Benchmark all packages in the workspace
...

It looks like cargo bench supports --all to me.

[azriel91]

This ugly command tells you what subcommands you have installed have the --all option:

cargo --list | grep -vF 'Installed' | xargs -I {} bash -c 'echo -n {}: ; cargo {} --help 2>&1 | grep -m 1 "^.\+--all[^-a-z]" || echo' | grep '.\+--all'

Sample output:

bench:    --all                        Benchmark all packages in the workspace
build:    --all                        Build all packages in the workspace
check:    --all                        Check all packages in the workspace
doc:    --all                        Document all packages in the workspace
fmt:        --all           format all packages (only usable in workspaces)
test:    --all                        Test all packages in the workspace