xtask: add coverage command #2067
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
mejrs
approved these changes
Dec 26, 2021
| @@ -119,7 +119,8 @@ harness = false | |||
| members = [ | |||
| "pyo3-macros", | |||
| "pyo3-macros-backend", | |||
| "examples" | |||
| "examples", | |||
| "xtask" | |||
There was a problem hiding this comment.
We don't necessarily need to name this "xtask". I don't have a motivation for changing it. But we could.
There was a problem hiding this comment.
Yeah, I don't personally love the name xtask but I do like that it's a documented "approach" (https://github.com/matklad/cargo-xtask), so without good motivation I don't feel a need to change it.
For example I'd personally enjoy the alias being called cargo run-script. There's nothing stopping us adding a couple extra aliases that all do the same later, if we wanted ;)
xtask/Cargo.toml
Outdated
|
|
||
| [dependencies] | ||
| anyhow = "1.0.51" | ||
| clap = { version = "3.0.0-rc.7", features = ["derive"] } |
There was a problem hiding this comment.
Does this add clap to pyo3's dependency tree? If so can we avoid that?
There was a problem hiding this comment.
I don't believe it does, no. This is an excerpt of Cargo.lock from a test project built against this branch:
[[package]]
name = "pyo3"
version = "0.15.1"
dependencies = [
"cfg-if",
"indoc",
"libc",
"num-bigint",
"parking_lot",
"paste",
"pyo3-build-config",
"pyo3-macros",
"unindent",
]
So clap hasn't come through as a dependency here.
|
Great! I've tidied this up a bit and added If CI passes I'll merge this now; I'll follow up on the coverage command improvements later once the |
davidhewitt
force-pushed
the
xtask-coverage
branch
from
f87b9c8
to
f8a83be
Compare
davidhewitt
force-pushed
the
xtask-coverage
branch
from
f8a83be
to
f344c23
Compare
bors bot
added a commit
to taiki-e/cargo-llvm-cov
that referenced
this pull request
Jan 1, 2022
115: cli: add show-env subcommand r=taiki-e a=davidhewitt Implements the `show-env` suggestion from #93 (comment) An example output: ``` $ cargo llvm-cov show-env RUSTFLAGS=" -Z instrument-coverage --remap-path-prefix /home/david/dev/pyo3/= --cfg coverage --cfg trybuild_no_target" LLVM_PROFILE_FILE="/home/david/dev/pyo3/target/llvm-cov-target/pyo3-%m.profraw" CARGO_INCREMENTAL="0" CARGO_TARGET_DIR="/home/david/dev/pyo3/target/llvm-cov-target" ``` After exporting these vars, I had a go at building pyo3's `cdylib` examples and then running them with `pytest`. Pleased to see that I do get profile output in the expected location: ``` $ ls target/llvm-cov-target/ CACHEDIR.TAG pyo3-6622988374613674511_0.profraw debug pyo3-8390305564713967804_0.profraw pyo3-10144924340354712083_0.profraw pyo3-8655209440434687052_0.profraw pyo3-12448306348928428713_0.profraw pyo3-9115749969700858718_0.profraw pyo3-13212718739740305352_0.profraw pyo3-9296542518232051450_0.profraw pyo3-17208771140764127027_0.profraw pyo3.profdata pyo3-40338836728015828_0.profraw release pyo3-5761173678925874245_0.profraw wheels pyo3-6135639819960792837_0.profraw ``` There's still some pain points: 1. I need to unset `CARGO_TARGET_DIR` before attempting to run `cargo llvm-cov` command, or I get crashes because it tries to append `llvm-cov-target` for a second time: ``` $ cargo llvm-cov --no-run --summary-only error: no input files specified. See llvm-profdata merge -help error: failed to merge profile data: process didn't exit successfully: `/home/david/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/bin/llvm-profdata merge -sparse -o /home/david/dev/pyo3/target/llvm-cov-target/llvm-cov-target/pyo3.profdata` (exit status: 1) ``` 2. After unsetting `CARGO_TARGET_DIR`, I still get a failure to generate the report. I suspect this is because `cargo-llvm-cov` is not finding and passing the `cdylib` outputs to `llvm-cov report`? ``` $ cargo llvm-cov --no-run --summary-only --verbose Running `/home/david/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/bin/llvm-profdata merge -sparse /home/david/dev/pyo3/target/llvm-cov-target/pyo3-10144924340354712083_0.profraw /home/david/dev/pyo3/target/llvm-cov-target/pyo3-12448306348928428713_0.profraw /home/david/dev/pyo3/target/llvm-cov-target/pyo3-13212718739740305352_0.profraw /home/david/dev/pyo3/target/llvm-cov-target/pyo3-17208771140764127027_0.profraw /home/david/dev/pyo3/target/llvm-cov-target/pyo3-40338836728015828_0.profraw /home/david/dev/pyo3/target/llvm-cov-target/pyo3-5761173678925874245_0.profraw /home/david/dev/pyo3/target/llvm-cov-target/pyo3-6135639819960792837_0.profraw /home/david/dev/pyo3/target/llvm-cov-target/pyo3-6622988374613674511_0.profraw /home/david/dev/pyo3/target/llvm-cov-target/pyo3-8390305564713967804_0.profraw /home/david/dev/pyo3/target/llvm-cov-target/pyo3-8655209440434687052_0.profraw /home/david/dev/pyo3/target/llvm-cov-target/pyo3-9115749969700858718_0.profraw /home/david/dev/pyo3/target/llvm-cov-target/pyo3-9296542518232051450_0.profraw -o /home/david/dev/pyo3/target/llvm-cov-target/pyo3.profdata` Running `/home/david/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/bin/llvm-cov report -instr-profile=/home/david/dev/pyo3/target/llvm-cov-target/pyo3.profdata -ignore-filename-regex '(^|/)(rustc/[0-9a-f]+|(\.)?cargo/(registry|git)|(\.)?rustup/toolchains|tests|examples|benches|target/llvm-cov-target)/|^/home/david/|^pyo3-macros/|^pyo3-macros-backend/|^pyo3-build-config/|^examples/|^xtask/'` No filenames specified! error: failed to generate report: process didn't exit successfully: `/home/david/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/bin/llvm-cov report -instr-profile=/home/david/dev/pyo3/target/llvm-cov-target/pyo3.profdata -ignore-filename-regex '(^|/)(rustc/[0-9a-f]+|(\.)?cargo/(registry|git)|(\.)?rustup/toolchains|tests|examples|benches|target/llvm-cov-target)/|^/home/david/|^pyo3-macros/|^pyo3-macros-backend/|^pyo3-build-config/|^examples/|^xtask/'` (exit status: 1) ``` If you're interested, you can see the companion PyO3 at PyO3/pyo3#2067 Co-authored-by: David Hewitt <1939362+davidhewitt@users.noreply.github.com> Co-authored-by: Taiki Endo <te316e89@gmail.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This is a bit of an experiment to add a
cargo xtask coveragealias.Ultimately this will re-implement the coverage recipe in ci. I also was playing around with adding coverage from out examples and python tests. It doesn't seem to be working quite yet; needs upstream support in
cargo-llvm-covwhich I'm still trying to figure out.