Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Issue #11665: Create a robust method of surviving mutation suppression #11731

Merged
merged 1 commit into from Jun 25, 2022

Conversation

Vyom-Yadav
Copy link
Member

@Vyom-Yadav Vyom-Yadav commented Jun 12, 2022

Resolves #11665

Only the last commit is under review.

Proofs ->

  • CI task failing from 1 extra mutation in file
  • CI task failing from more than 1 extra mutation in file
  • CI task failing from 1 missing mutation in file
  • CI task failing from more than 1 missing mutation in file
  • Proof of parsing comments in suppression file, multiple line and single. Comments are present in multiple suppressions files, with no problem whatsoever.
  • Proof of same mutations from pitest.sh via cli (can just use grep/ sed/ diff). We may have been obfuscating multiple violations previously, we should keep this in mind when considering results.

Here is the difference between old pitest suppressions and new pitest suppressions.
The topmost mutation } is due to hcoles/pitest#688 (See this) and the 6 added mutations were multiple mutations on the same line. Report is good according to me. A mix of sed, awk, groovy was used, report is accurate.

@Vyom-Yadav Vyom-Yadav force-pushed the robustPitestReport branch 3 times, most recently from 822364a to 94c63c1 Compare June 12, 2022 13:20
@nrmancuso
Copy link
Member

nrmancuso commented Jun 12, 2022

@Vyom-Yadav let's add codenarc to CI execution now that we are adding groovy to main repo. Take a look at contribution to see how it is done there: https://github.com/checkstyle/contribution/blob/master/checkstyle-tester/codenarc.sh

Issue created at #11734

@Vyom-Yadav Vyom-Yadav force-pushed the robustPitestReport branch 13 times, most recently from 0fae0d0 to 1b22465 Compare June 15, 2022 10:04
@Vyom-Yadav Vyom-Yadav force-pushed the robustPitestReport branch 3 times, most recently from 8751d77 to 82b51fc Compare June 15, 2022 10:34
Copy link
Member

@nrmancuso nrmancuso left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some early feedback:

.ci/pitest.groovy Outdated Show resolved Hide resolved
.ci/pitest.groovy Outdated Show resolved Hide resolved
.ci/pitest.groovy Outdated Show resolved Hide resolved
.github/workflows/pitest.yml Outdated Show resolved Hide resolved
.ci/pitest.groovy Outdated Show resolved Hide resolved
.ci/pitest.sh Outdated Show resolved Hide resolved
@Vyom-Yadav Vyom-Yadav force-pushed the robustPitestReport branch 2 times, most recently from 0e36130 to e7e3df3 Compare June 16, 2022 04:28
@Vyom-Yadav Vyom-Yadav requested a review from baratali June 23, 2022 09:19
Copy link
Member

@romani romani left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Items

.github/workflows/pitest.yml Outdated Show resolved Hide resolved
@Vyom-Yadav Vyom-Yadav force-pushed the robustPitestReport branch 11 times, most recently from c7de5b5 to 73f7fee Compare June 23, 2022 18:33
@Vyom-Yadav Vyom-Yadav requested a review from romani June 23, 2022 18:36
@Vyom-Yadav Vyom-Yadav force-pushed the robustPitestReport branch 6 times, most recently from 81a3fa3 to fba897f Compare June 24, 2022 15:38
@Vyom-Yadav Vyom-Yadav force-pushed the robustPitestReport branch 2 times, most recently from 8b79115 to 9ea8a0d Compare June 25, 2022 05:50
Copy link
Member

@romani romani left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok to merge

Copy link
Member

@nrmancuso nrmancuso left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can deal with team city failures later, let's get this merged and kill some mutants.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Pitest: Create robust method of surviving mutation suppression
6 participants