Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

dependency: bump pitest-maven from 1.8.1 to 1.9.0 #11761

Merged
merged 2 commits into from Jun 26, 2022

Conversation

dependabot[bot]
Copy link
Contributor

@dependabot dependabot bot commented on behalf of github Jun 23, 2022

Bumps pitest-maven from 1.8.1 to 1.9.0.

Release notes

Sourced from pitest-maven's releases.

1.9.0

  • #1035 - Improve coverage memory consumption
  • #1038 - Support Maven Site Plugin 3.12.0+ (thanks IOs)
  • #1040 - Faster JUnit 5 coverage calculation
  • #1040 - Remove dependencyDistance parameter

This release greatly improves the speed and efficiency of the coverage analysis performed before mutation analysis begins (particularly if you are using JUnit 5).

As a result of #1040 any external pitest test plugins (eg JUnit 5, TestNG, arcmutate accelerator) must be updated to a compatible version (1.0.0) when upgrading.

The dependencyDistance parameter is no longer supported.

Commits
  • 61eabfb Merge pull request #1041 from hcoles/cleanup/maven_versions
  • 161d2f7 cleanup versions a little
  • 2f11641 Merge pull request #1038 from l0s/1037_maven-site-4.x
  • 37152be update readme for 1.9.0
  • 98b9f64 Merge pull request #1040 from hcoles/feature/junit5_style_discovery
  • 4875ec7 support mixed discovery/execution of tests
  • c40f054 Downgrade maven-reporting to latest non-milestone
  • d3c0bcd remove dependency distance support
  • ae8c498 Merge pull request #1035 from hcoles/feature/coverage_memory_consumption
  • a6179d7 write all blocks in xml output
  • Additional commits viewable in compare view

Dependabot compatibility score

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

  • @dependabot rebase will rebase this PR
  • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
  • @dependabot merge will merge this PR after your CI passes on it
  • @dependabot squash and merge will squash and merge this PR after your CI passes on it
  • @dependabot cancel merge will cancel a previously requested merge and block automerging
  • @dependabot reopen will reopen this PR if it is closed
  • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
  • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
  • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
  • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)

@dependabot dependabot bot added the dependencies Pull requests that update a dependency file label Jun 23, 2022
@rnveach
Copy link
Member

rnveach commented Jun 23, 2022

Execution default-cli of goal org.pitest:pitest-maven:1.9.0:mutationCoverage failed: Coverage generation minion exited abnormally!

@rnveach
Copy link
Member

rnveach commented Jun 24, 2022

I confirmed the issue on my local, but it is not clear why it is happening. I will report the issue.

> mvn -version
Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
Maven home: C:\mde-tools\eclipse-2021-06-jee-kit\apache-maven-3.6.3\bin\..
Java version: 11.0.10, vendor: Oracle Corporation, runtime: C:\mde-tools\eclipse-2021-06-jee-kit\openjdk-11.0.10_9
Default locale: en_US, platform encoding: Cp1252
OS name: "windows 10", version: "10.0", arch: "amd64", family: "windows"

@rnveach
Copy link
Member

rnveach commented Jun 24, 2022

Things are passing better now, except:

Run ./.ci/pitest.sh pitest-xpath
Diff:
--- target/ignored.txt 2022-06-24 00:53:48.766280012 +0000
+++ target/actual.txt 2022-06-24 00:53:48.766280012 +0000
@@ -0,0 +1 @@
+XpathFileGeneratorAstFilter.java.html:<td class='covered'><pre><span class='survived'> if (event.getTokenType() != 0) {</span></pre></td></tr>

Edit:
Surviving mutation is:

  1. removed conditional - replaced equality check with false → SURVIVED

For me, testTabWidth and test report test failures with the mutation hardcoded but pitest reproduced the issue on my local. So I am not understanding the situation.
It seems it might be regression from the version bump, but it could be something funky with the byte code.

Before upgrade, report says it was killed by testTabWidth.

@hcoles
Copy link

hcoles commented Jun 24, 2022

I've taken a look at the surviving mutant. I managed to reproduce the issue once, but the mutant was killed on all subsequent runs.

Looking at the code, I'm reasonable confident that this is an existing test order dependency issue in the checkstyle suite, rather than an issue introduced in pitest 1.9.0.

The MESSAGE_QUERY_MAP map in XpathFileGeneratorAstFilter is static state, which will be modified by any test that executes accept.

Depending on which tests have been run in the JVM before, the state required to make XpathFileGeneratorAstFilterTest pass may already be in MESSAGE_QUERY_MAP before the mutant is introduced (possibly as a result of XpathFileGeneratorAstFilterTest itself having been previously run).

Although pitest aims to be fully deterministic, it cannot guarantee the same order of tests in every run. This is because the order is (partially) determined by test timings, which can be effected by external factors. Time outs due to (presumed) infinite loops also mean that pitest cannot gurantee that the same number of jvms launches occurs for each run.

Assuming this is indeed an order dependency due to static state, it can be mitigated by ensuring the finishLocalSetup method of XpathFileGeneratorAstFilter is run after a test modifies the static state, or before a test executes that relies upon it.

The simplest fix for this particular issue might be to add this to XpathFileGeneratorAstFilterTest

    @BeforeEach
    public void ensureCleanStartingState() {
        new XpathFileGeneratorAstFilter() {
            @Override
            public void finishLocalSetup() {
                super.finishLocalSetup();
            }
        }.finishLocalSetup();
    }

It's unfortunately a little messy as it needs to overcome the access restrictions on finishLocalSetup

dependabot bot and others added 2 commits June 24, 2022 08:14
Bumps [pitest-maven](https://github.com/hcoles/pitest) from 1.8.1 to
1.9.0.
- [Release notes](https://github.com/hcoles/pitest/releases)
- [Commits](hcoles/pitest@1.8.1...1.9.0)
Bumps [pitest-junit5-plugin](https://github.com/pitest/pitest-junit5-plugin)
from 0.16 to 1.0.0.
- [Release
notes](https://github.com/pitest/pitest-junit5-plugin/releases)
- [Commits](pitest/pitest-junit5-plugin@0.16...1.0.0)

---
updated-dependencies:
- dependency-name: org.pitest:pitest-maven
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
@rnveach rnveach force-pushed the dependabot/maven/org.pitest-pitest-maven-1.9.0 branch from d0b9f78 to 821a570 Compare June 24, 2022 12:25
@rnveach rnveach requested review from nrmancuso and romani June 24, 2022 12:58
@rnveach
Copy link
Member

rnveach commented Jun 24, 2022

I feel pitest is now resolved and can be reviewed.

Copy link
Member

@romani romani left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I hope one day we will migrate to BDD style completely.
So we will less likely run into such problems.

@nrmancuso nrmancuso merged commit df3b5ea into master Jun 26, 2022
@nrmancuso nrmancuso deleted the dependabot/maven/org.pitest-pitest-maven-1.9.0 branch June 26, 2022 05:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
dependencies Pull requests that update a dependency file miscellaneous
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants