Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

In clover.xml, count all method definitions once #115

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

mike-burns
Copy link
Contributor

@mike-burns mike-burns commented Mar 26, 2024

The clover.xml specification does not answer what a count means for a line with type="method".

In the case of PHP's CodeCoverage library, it is set to the maximum count for any statement within the method.

Therefore, even if a file is parsed and run (so the method is defined), if the method is never called then it will have a count of 0. This differs from how other PHP tools display "method" lines, which instead treat them as run.

So, treat all methods as if they have a count of 1.

☑️ Checklist

  • Add specs

[The `clover.xml` specification] does not answer what a `count` means for
a line with `type="method"`.

In the case of PHP's CodeCoverage library, it is set to [the maximum
count] for any statement within the method.

Therefore, even if a file is parsed and run (so the method is defined),
if the method is never called then it will have a count of `0`. This
differs from how other PHP tools display `"method"` lines, which instead
treat them as run.

So, treat all methods as if they have a count of `1`.

[The `clover.xml` specification]: https://bitbucket.org/atlassian/clover/src/master/etc/schema/clover.xsd
[the maximum count]: https://github.com/sebastianbergmann/php-code-coverage/blob/main/src/Report/Clover.php#L89-L104
Copy link

Pull Request Test Coverage Report for Build 8443662829

Details

  • 4 of 4 (100.0%) changed or added relevant lines in 1 file are covered.
  • No unchanged relevant lines lost coverage.
  • Overall coverage increased (+0.01%) to 93.731%

Totals Coverage Status
Change from base Build 6721884478: 0.01%
Covered Lines: 942
Relevant Lines: 1005

💛 - Coveralls

Copy link
Member

@afinetooth afinetooth left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mike-burns please see my comment on Line 60 in clover_parser.cr. I'm mostly seeking clarity---not sure if I've assessed the code properly.

else
hits = line_node.attributes["count"].content.to_u64
coverage[line_number] = hits
end
end
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mike-burns question:
Will this solution place the correct number of hits for the method on the line declaring the method (in the Coveralls source file report)? See the example I refer to below, pls.

I see that this code will solve the problem of lines declaring functions displaying as (relevant but) uncovered. It appears this will always at least make sure they are shown as covered (green) in Coveralls reports.

However, given your discovery that the number of hits for a method is taken to be equivalent to the highest number of hits for any of its (statement) lines, won't this miss the opportunity to show the correct number of hits for the method, and always just show (1)?

I can understand using this code as a failsafe to make sure lines declaring a method appear as covered, but it seems to me that the correct number of hits should be what we get from a correct XML report, which appears to be that highest number for the method's lines.

Check out the example pertaining to the AbstractArrayDeclarationSniff.php file in my comment on the OP's public issue:

Screenshot 2024-04-02 at 4 27 34 PM

To be perfectly honest, I kind of forget the point of my comment there, but now I'm stuck on the fact that recent source file reports (like this one, as opposed to this older version from Aug 2023) seem to show the correct hit count for the method. in the case of Line 178, 52.

Isn't that what a correct clover.xml report should show as the hit count for the method? And isn't that what we want to show in our reports?

Forgive me for being rusty on this one, I may have missed something.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

won't this miss the opportunity to show the correct number of hits for the method, and always just show (1)?

This really comes down to: what number do we want to show for line 178 in that screenshot? What is "correct", in this case?

  • 52, the highest number of any line within the method (so 0 for an untested method).
  • max(52, 1), the highest number of any line within the method (or 1 for an untested method, meaning that line 178 is OK).
  • 1, simply indicating that the method was defined.

As a base case, what numbers do we want for this method definition:

public function someNoOpMethod()
{
}

Do we want to say that the public function someNoOpMethod() line was never run? No amount of test coverage can change that -- calling the method executes zero statements, so the method definition line is still considered 0.

I'm happy to go with your decision -- you might have more context than I do around what these numbers are used for.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants