Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New behaviour for RetryingTest for Assumptions #293

Merged
merged 6 commits into from
Jul 9, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
12 changes: 12 additions & 0 deletions docs/retrying-test.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,18 @@ void failsAlways() {
The test `failsAlways` is executed three times (all of which fail).
The first two executions are marked as ignored/aborted, while the last as failed - each contains the underlying exception.

```java
@RetryingTest(3)
void aborted() {
// test code that is aborted e.g. because of an Assumption.
}
```

If a test is aborted (e.g.: because of a failed Assumption) `@RetryingTest` won't try again after that.
The test `aborted` is aborted before (or during) its first execution.
The first execution is marked as aborted/skipped, containing the underlying cause.
The test suite as a whole is also marked as aborted/skipped.

== Combining `@RetryingTest` with `@Test` et al

If `@RetryingTest` is combined with `@Test` or `TestTemplate`-based mechanisms (like `@RepeatedTest` or `@ParameterizedTest`), the test engine will execute it according to each annotation (i.e. more than once).
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -85,6 +85,10 @@ static FailedTestRetrier createFor(Method test) {
}

void failed(Throwable exception) {
if (exception instanceof TestAbortedException)
throw new TestAbortedException("Test execution was skipped, possibly because of a failed assumption.",
exception);

exceptionsSoFar++;

boolean allRetriesFailed = exceptionsSoFar == maxRetries;
Expand Down
14 changes: 14 additions & 0 deletions src/test/java/org/junitpioneer/jupiter/RetryingTestTests.java
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@
import java.lang.annotation.RetentionPolicy;
import java.lang.annotation.Target;

import org.junit.jupiter.api.Assumptions;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.TestTemplate;
import org.junit.jupiter.api.parallel.Execution;
Expand Down Expand Up @@ -74,6 +75,14 @@ void failsAlways_executedThreeTimes_fails() {
assertThat(results.numberOfFailedTests()).isEqualTo(1);
}

@Test
void skipByAssumption_executedOnce_skipped() {
ExecutionResults results = PioneerTestKit.executeTestMethod(RetryingTestTestCase.class, "skipByAssumption");

assertThat(results.numberOfDynamicRegisteredTests()).isEqualTo(1);
assertThat(results.numberOfAbortedTests()).isEqualTo(1);
}

// TEST CASES -------------------------------------------------------------------

static class RetryingTestTestCase {
Expand Down Expand Up @@ -107,6 +116,11 @@ void failsAlways() {
throw new IllegalArgumentException();
}

@RetryingTest(3)
void skipByAssumption() {
Assumptions.assumeFalse(true);
}

}

@Target({ METHOD, ANNOTATION_TYPE })
Expand Down