Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

JUnit rule for flaky test retry #1680

Merged
merged 19 commits into from Aug 6, 2019
Merged

JUnit rule for flaky test retry #1680

merged 19 commits into from Aug 6, 2019

Conversation

rnorth
Copy link
Member

@rnorth rnorth commented Jul 29, 2019

Fixes #1629

@rnorth rnorth changed the title Experimental JUnit rule for flaky test retry WIP: Experimental JUnit rule for flaky test retry Jul 29, 2019
@rnorth rnorth changed the title WIP: Experimental JUnit rule for flaky test retry Experimental JUnit rule for flaky test retry Aug 2, 2019
@rnorth
Copy link
Member Author

rnorth commented Aug 2, 2019

Un WIP'd, please could you have a look too @kiview?

@@ -35,9 +33,28 @@ subprojects {
}
}

project.tasks.sourceJar.from(delombok)
// specific modules should be excluded from publication
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a way to disable publication of these modules. Not sure if it's the best.

I've tested this manually using ./gradlew -x test publishToMavenLocal and verified that these three modules are no longer created under ~/.m2/repository/org/testcontainers

We should obviously be cautious to check the same applies to the bintray publication as well.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AFAIK artifacts.removeAll() will do the trick (no publishing if there are no artifacts to publish)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I honestly can't figure out how to make this work - my Gradle fu is clearly not strong enough. Unless there's something wrong with my approach I'd prefer to keep with something that I can understand!

@rnorth rnorth changed the title Experimental JUnit rule for flaky test retry JUnit rule for flaky test retry Aug 2, 2019
* format (e.g. {@code 2020-12-03}). Now + 3 months is suggested. Once this date has passed, retries will no longer
* be applied.
*/
String reviewDate();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

WDYT about letting them fail explicitly instead of not retrying?
Which would mean, we either have to remove the annotation or prolong it?

Or does this create too much maintenance? I think the maintenance should be the same, just without explicitly failing, we will miss the flaky tests for some time.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suspect it could end up being a PITA for contributors, to be honest!

@rnorth rnorth requested a review from kiview August 6, 2019 07:01
@rnorth rnorth merged commit bdb82b7 into master Aug 6, 2019
@rnorth rnorth deleted the flaky-test-retry branch August 6, 2019 11:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Make it possible to automatically rerun selected flaky tests
3 participants