Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

More detail/documentation for "functions" coverage #971

Open
detly opened this issue Mar 1, 2023 · 1 comment
Open

More detail/documentation for "functions" coverage #971

detly opened this issue Mar 1, 2023 · 1 comment

Comments

@detly
Copy link
Contributor

detly commented Mar 1, 2023

This is a bit of a wishy washy issue, but it comes down to: I saw a big red number on my otherwise-green coverage report, but I still don't really understand what it means or how to get more detail on it (or how to fix it). As I go into more detail I've flagged the more specific questions and issues I have.

There's an example project here, and here's the coverage report. It's from a real Rust project I'm working on, it's deliberately not minimal (I'll get to that), and the command to generate the coverage report is:

env \
LLVM_PROFILE_FILE="test-binary-%p-%m.profraw" \
RUSTFLAGS="-C instrument-coverage" \
RUST_BACKTRACE=full \
cargo test

grcov . \
-t html \
--binary-path ./target/debug -s . \
--branch --llvm \
--ignore-not-existing --keep-only 'src/**' --ignore 'src/bin/test_*' \
--excl-br-start "mod tests \{" --excl-start "mod tests \{" \
--excl-br-line "#\[derive\(" --excl-line "#\[derive\(" \
-o coverage

When I generate the report, it shows the line and branch coverage as 100%. But between those two is something labelled "functions" which sits at 52.27% (in RED). Now, with no other information, I'd guess that that means only about 50% of the functions are being tested. But several things make me think this guess is wrong:

  • How could line coverage be 100% if function coverage is only 50%?
  • Drilling down into the code view shows every line to be green. (Question: I assume this is because that view only shows line coverage? Or not? Is there a way to switch between the three metrics?)
  • I did search for questions/issues around this, and found For Rust projects, many lines are marked as partial hits that ideally would not be. #476 and $454. But both those issues say to try the "new" (in 2020) source-based coverage. I'm pretty sure I'm using that now. (Right?)
  • Those issues say that this might be related to macros. But (issue:) the report does not show me what macros, either by highlighting them in the place they're used or by listing them separately.
  • This is why I haven't made a minimal example: if I reduce this project down to a single macro, it's easy to see the problem with grcov's analysis but it's also hard to see the problem with its reporting. There it is! I might say. That's the macro causing all the trouble! But try doing that with this project and you'll quickly run out of patience.

Issue: Documentation would remove the guesswork, but I cannot find details on this in Grcov's docs.

My overall issue is simply this: if low function coverage warrants a big, red number on the top of the report, it should also provide a way to account for that number, near to where it's shown. A link to documentation would be better than nothing, but ideally a way to visually display why the number is what it is.

Environment details

Gitlab CI: rust:1-slim docker image

$ cargo --version
cargo 1.67.1 (8ecd4f20a 2023-01-10)
$ rustc --version
rustc 1.67.1 (d5a82bbd2 2023-02-07)
$ grcov --version
grcov 0.8.13

Dev machine: Ubuntu 22.10

~ ⚬ cargo --version
cargo 1.67.1 (8ecd4f20a 2023-01-10)
~ ⚬ rustc --version
rustc 1.67.1 (d5a82bbd2 2023-02-07)
~ ⚬ grcov --version
grcov 0.8.13
@detly
Copy link
Contributor Author

detly commented Mar 1, 2023

Of course, of course ten minutes after posting this I find #793.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant