Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DisableIfArgument #368

Merged
merged 36 commits into from
May 29, 2021
Merged
Show file tree
Hide file tree
Changes from 23 commits
Commits
Show all changes
36 commits
Select commit Hold shift + click to select a range
d99e74b
Created PoC extension for disabling parameterized tests
Michael1993 Sep 26, 2020
62bf604
Merge branch 'master' into issue313/explore-invocation-interceptor
Michael1993 Oct 31, 2020
d223cfd
Replaced value() with contains() and matches()
Michael1993 Oct 31, 2020
9574ea8
Disable based on containing and RegEx
Michael1993 Oct 31, 2020
0168cc1
Updated tests
Michael1993 Oct 31, 2020
c4e3576
Spotless, no :(
Michael1993 Oct 31, 2020
f9f46c8
Add documentation based on DisableIfDisplayName
Michael1993 Nov 2, 2020
81fddd9
README.md updated to include contribution
Michael1993 Nov 2, 2020
98663ed
Simplify orElseThrow
Michael1993 Nov 2, 2020
2cfc8c1
Update tests to include a poem (Oscar Wilde: Requiescat)
Michael1993 Nov 2, 2020
b85e3e0
Update package-info.java with the new addition
Michael1993 Nov 2, 2020
5643ebf
Fix comma typo
Michael1993 Nov 2, 2020
10e4d8a
Add references to contribution
Michael1993 Nov 2, 2020
bbe9dd0
Update code and documentation according to feedback
Michael1993 Nov 10, 2020
26fdcb5
In progress DisableIfParameter
Michael1993 Nov 24, 2020
ff66f85
No parameters can not be disabled.
Michael1993 Dec 22, 2020
4b8d27b
Update DisableIfParameter based on feedback
Michael1993 Mar 23, 2021
c3459d9
Merge remote-tracking branch 'origin/main' into issue313/explore-invo…
Michael1993 Mar 23, 2021
a5bd7ca
Update tests and formatting
Michael1993 Mar 23, 2021
b0e08e3
Update tests
Michael1993 Mar 23, 2021
8bb3765
Invalid input returns exception
Michael1993 Mar 23, 2021
6903f5b
Invalid input returns exception, try #2
Michael1993 Mar 23, 2021
b6886d0
Add another test
Michael1993 Mar 23, 2021
87637cb
Update docs
Michael1993 Apr 6, 2021
6c32440
Merge branch 'main' into issue313/explore-invocation-interceptor
Michael1993 Apr 8, 2021
cf6d5dc
Trigger checks again
Michael1993 Apr 8, 2021
faf3d60
Add JavaDoc on annotations
Michael1993 Apr 17, 2021
6d99b67
More JavaDoc updates
Michael1993 Apr 17, 2021
612030f
Minor documentation polish
Michael1993 Apr 18, 2021
b905274
Merge branch 'main' into issue313/explore-invocation-interceptor
Michael1993 Apr 21, 2021
960051f
Merge branch 'main' into issue313/explore-invocation-interceptor
May 18, 2021
76aa6d8
Edit documentation
May 18, 2021
4fad9b5
Moving code around
May 18, 2021
058a1a2
Rename "Parameter" to "Argument"
May 18, 2021
7f8c331
Merge branch 'main' of https://github.com/junit-pioneer/junit-pioneer…
Michael1993 May 29, 2021
27bd4fb
Add since in javadoc
Michael1993 May 29, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -125,7 +125,7 @@ The least we can do is to thank them and list some of their accomplishments here
* [Ignat Simonenko](https://github.com/simonenkoi) fixed a noteworthy bug in the default locale extension (#146 / #161)
* [Mark Rösler](https://github.com/Hancho2009) contributed the [environment variable extension](https://junit-pioneer.org/docs/environment-variables/) (#167 / #174 and #241 / #242)
* [Matthias Bünger](https://github.com/Bukama) opened, vetted, and groomed countless issues and PRs and contributed multiple refactorings (e.g. #165 / #168) and fixes (e.g. #190 / #200) before getting promoted to maintainer
* [Mihály Verhás](https://github.com/Michael1993) contributed [the StdIO extension](https://junit-pioneer.org/docs/standard-input-output/) (#34 / #227), [the ReportEntryExtension](https://junit-pioneer.org/docs/report-entries/) (#134, #179 / #183, #216, #294), [the CartesianProductTestExtension](https://junit-pioneer.org/docs/cartesian-product/) (#321, #362 / #68, #354), added tests to other extensions (#164 / #272), the Pioneer assertions and contributed to multiple issues (e.g. #217 / #298) and PRs (e.g. #253, #307)
* [Mihály Verhás](https://github.com/Michael1993) contributed [the StdIO extension](https://junit-pioneer.org/docs/standard-input-output/) (#34 / #227), [the ReportEntryExtension](https://junit-pioneer.org/docs/report-entries/) (#134, #179 / #183, #216, #294), [the CartesianProductTestExtension](https://junit-pioneer.org/docs/cartesian-product/) (#321, #362 / #68, #354), [the DisableIfParameterExtension](https://junit-pioneer.org/docs/disable-parameterized-tests/) (#313, #368) added tests to other extensions (#164 / #272), the Pioneer assertions and contributed to multiple issues (e.g. #217 / #298) and PRs (e.g. #253, #307)
Michael1993 marked this conversation as resolved.
Show resolved Hide resolved
* [Nishant Vashisth](https://github.com/nishantvas) contributed an [extension to disable parameterized tests](https://junit-pioneer.org/docs/disable-if-display-name/) by display name (#163 / #175)
* [Simon Schrottner](https://github.com/aepfli) contributed to multiple issues and PRs and almost single-handedly revamped the build and QA process (e.g. #192 / #185) before getting promoted to maintainer
* [Sullis](https://github.com/sullis) improved GitHub Actions with Gradle Wrapper Validation check (#302)
Expand Down
82 changes: 3 additions & 79 deletions docs/disable-if-display-name.adoc
Original file line number Diff line number Diff line change
@@ -1,81 +1,5 @@
:page-title: Disable Based on DisplayName
:page-title: Disable Parameterized Test Based on DisplayName
:page-description: Extends JUnit Jupiter with `@DisableIfDisplayName`, which selectively disables parameterized tests

The `@DisableIfDisplayName` annotation can be used to selectively disable parameterized tests based on their display names, which are dynamically registered on runtime.
The annotation is only supported on test method level for parameterized tests.
Unlike the `@Disabled` API provided in JUnit Jupiter, which disables the test on first encounter of the annotation , `@DisableIfDisplayName` is validated before each parameterized test execution.
As a consequence, instead of disabling the entire set of parameterized tests, each test (name) can be evaluated and possibly disabled individually.

[source,java]
----
// disable invocations whose display name contains "disable"
@DisableIfDisplayName(contains = "disable")
@ParameterizedTest(name = "See if enabled with {0}")
@ValueSource(
// Disabled: 1,2,3,4,5
// Not disabled: 6
strings = {
"disable who", // 1
"you, disable you", // 2
"why am I disabled", // 3
"what has been disabled must stay disabled", // 4
"fine disable me all you want", // 5
"not those one, though!" // 6
}
)
void testExecutionDisabled(String reason) {
if (reason.contains("disable"))
fail("Test should've been disabled " + reason);
}
----

You can also specify more than one substring at a time:

[source,java]
----
@DisableIfDisplayName(contains = {"1", "2"})
@ParameterizedTest(name = "See if enabled with {0}")
@ValueSource(ints = { 1, 2, 3, 4, 5 })
void testDisplayNameString(int num) {
if (num == 1 || num == 2)
fail("Test should've been disabled for " + num);
}
----

If substrings are not powerful enough, you can also use regular expressions:

[source,java]
----
// disable invocations whose display name
// contains "disable " or "disabled "
@DisableIfDisplayName(matches = ".*disabled?\\s.*")
@ParameterizedTest(name = "See if enabled with {0}")
@ValueSource(
// Disabled: 1,2,4,5
// Not disabled: 3,6
strings = {
"disable who", // 1
"you, disable you", // 2
"why am I disabled", // 3
"what has been disabled must stay disabled", // 4
"fine disable me all you want", // 5
"not those one, though!" // 6
}
)
void single(String reason) {
// ...
}
----

You can even use both, in which case a test is disabled if contains a substring _or_ matches an expression:

[source,java]
----
@DisableIfDisplayName(contains = "000", matches = ".*10?")
@ParameterizedTest(name = "See if enabled with {0}")
@ValueSource(ints = { 1, 10, 100, 1_000, 10_000 })
void containsAndMatches_containsAndMatches(int number) {
if (number != 100)
fail("Test should've been disabled for " + number);
}
----
This document has been moved and updated.
See link:disable-parameterized-tests.adoc[the updated document].
250 changes: 250 additions & 0 deletions docs/disable-parameterized-tests.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,250 @@
:page-title: Disable Parameterized Test
:page-description: Extends JUnit Jupiter with multiple extensions, which selectively disables parameterized tests

JUnit Pioneer offers multiple extensions for selectively disabling parameterized tests.
These are as follows:

- DisableIfDisplayName
- DisableIfParameter

== DisableIfDisplayName


The `@DisableIfDisplayName` annotation can be used to selectively disable parameterized tests based on their display names, which are dynamically registered on runtime.
The annotation is only supported on test method level for parameterized tests.
Unlike the `@Disabled` API provided in JUnit Jupiter, which disables the test on first encounter of the annotation , `@DisableIfDisplayName` is validated before each parameterized test execution.
As a consequence, instead of disabling the entire set of parameterized tests, each test (name) can be evaluated and possibly disabled individually.

[source,java]
----
// disable invocations whose display name contains "disable"
@DisableIfDisplayName(contains = "disable")
@ParameterizedTest(name = "See if enabled with {0}")
@ValueSource(
// Disabled: 1,2,3,4,5
// Not disabled: 6
strings = {
"disable who", // 1
"you, disable you", // 2
"why am I disabled", // 3
"what has been disabled must stay disabled", // 4
"fine disable me all you want", // 5
"not those one, though!" // 6
}
)
void testExecutionDisabled(String reason) {
if (reason.contains("disable"))
fail("Test should've been disabled " + reason);
}
----

You can also specify more than one substring at a time:

[source,java]
----
@DisableIfDisplayName(contains = {"1", "2"})
@ParameterizedTest(name = "See if enabled with {0}")
@ValueSource(ints = { 1, 2, 3, 4, 5 })
void testDisplayNameString(int num) {
if (num == 1 || num == 2)
fail("Test should've been disabled for " + num);
}
----

If substrings are not powerful enough, you can also use regular expressions:

[source,java]
----
// disable invocations whose display name
// contains "disable " or "disabled "
@DisableIfDisplayName(matches = ".*disabled?\\s.*")
@ParameterizedTest(name = "See if enabled with {0}")
@ValueSource(
// Disabled: 1,2,4,5
// Not disabled: 3,6
strings = {
"disable who", // 1
"you, disable you", // 2
"why am I disabled", // 3
"what has been disabled must stay disabled", // 4
"fine disable me all you want", // 5
"not those one, though!" // 6
}
)
void single(String reason) {
// ...
}
----

Using both `matches` and `contains` in a single annotation is no longer permitted as of {VERSION_OR_DATE_PLACHOLDER}.
Michael1993 marked this conversation as resolved.
Show resolved Hide resolved

== DisableIfParameter

This extension can be used to selectively disable parameterized tests based on their parameter values (converted with `toString()`).
The extension comes with three annotations, covering different use-cases:

- `@DisableIfAnyParameter`, non-repeatable
- `@DisableIfAllParameters`, non-repeatable
- `@DisableIfParameter`, repeatable

The annotations are only supported on test method level for parameterized tests.
Unlike the `@Disabled` API provided in JUnit Jupiter, which disables the test on first encounter of the annotation, the extension evaluates each execution of a parameterized test.
As a consequence, instead of disabling the entire set of parameterized tests, each test is possibly disabled individually.
All three annotations require that you specify one of two parameters, `contains` *or* `matches`.
`@DisableIfAnyParameter` will disable test executions if *any* argument either contains or matches any of the given strings.
`@DisableIfAllParameters` will disable test executions if *all* arguments either contain or match any of the given strings.
`@DisableIfParameter` will disable test executions if a *specified* argument either contains or matches any of the given strings.

=== Using `contains`

[source,java]
----
@DisableIfAllParameters(contains = "the")
@ParameterizedTest
@CsvSource(value = {
"If the swift moment I entreat:;Tarry a while! You are so fair!",
"Then forge the shackles to my feet,;Then I will gladly perish there!",
"Then let them toll the passing-bell,;Then of your servitude be free,",
"The clock may stop, its hands fall still,;And time be over then for me!"
}, delimiter = ';')
void disableAllContains(String line, String line2) {
}
----

The test `disableAllContains` ordinarily would run four times, but the second execution gets disabled because both arguments contain "the" (the second argument as part of "there").
Using the same test with a different annotation would look like this:

[source,java]
----
@DisableIfAnyParameter(contains = "Then")
@ParameterizedTest
@CsvSource(value = {
"If the swift moment I entreat:;Tarry a while! You are so fair!",
"Then forge the shackles to my feet,;Then I will gladly perish there!",
"Then let them toll the passing-bell,;Then of your servitude be free,",
"The clock may stop, its hands fall still,;And time be over then for me!"
}, delimiter = ';')
void disableAnyContains(String line, String line2) {
}
----

The test `disableAnyContains` ordinarily would run four times, but the second and third executions get disabled because an argument contains "Then".
The last execution does not get disabled, because the extension is case-sensitive.
Michael1993 marked this conversation as resolved.
Show resolved Hide resolved

You can specify more than one substring at a time:

[source, java]
----
@DisableIfAnyParameter(contains = { "Then", "then" })
@ParameterizedTest
@CsvSource(value = {
"If the swift moment I entreat:;Tarry a while! You are so fair!",
"Then forge the shackles to my feet,;Then I will gladly perish there!",
"Then let them toll the passing-bell,;Then of your servitude be free,",
"The clock may stop, its hands fall still,;And time be over then for me!"
}, delimiter = ';')
void disableAnyContains(String line, String line2) {
}
----

The extension disables the second, third and fourth executions because an argument contains either "Then" or "then".

`@DisableIfParameter` requires you to target a specific parameter.
You can do this in three ways:

- By a `name` https://docs.oracle.com/javase/8/docs/api/java/lang/reflect/Parameter.html#isNamePresent--[if parameter naming information is present].
- By an explicit `index`, starting from 0.
- By an implicit index.

Using both `name` and `index` in a single `@DisableIfParameter` annotation is not permitted.

==== Targeting by `name`

If naming information is included during compilation, you can target parameters by their name.

[source, java]
----
@DisableIfParameter(name = "line2", contains = "swift")
@ParameterizedTest
@CsvSource({
"If the swift moment I entreat:;Tarry a while! You are so fair!",
"Then forge the shackles to my feet,;Then I will gladly perish there!"
})
void targetName(String line, String line2) {
}
----

The test gets executed two times because we explicitly targeted the second parameter, which never contains the word "swift".

==== Targeting by `index`

You can target your parameters with their index, starting from 0.
Michael1993 marked this conversation as resolved.
Show resolved Hide resolved

[source, java]
----
@DisableIfParameter(index = 1, contains = "swift")
@ParameterizedTest
@CsvSource({
"If the swift moment I entreat:;Tarry a while! You are so fair!",
"Then forge the shackles to my feet,;Then I will gladly perish there!"
})
void targetIndex(String line, String line2) {
}
----

Again, the test gets executed two times, because we targeted the second parameter.

==== Targeting by implicit index

You can opt to not specify `index` or `name` and use annotation order to specify what parameter to target.
In this case the first `@DisableIfParameter` targets the first parameter, the second annotation the second parameter, etc.
This is mainly for convenience when you have a test method with a single parameter.
Using this method to target parameters when your test has multiple parameters is discouraged.

[source, java]
----
@DisableIfParameter(contains = "gibberish")
@DisableIfParameter(contains = "gladly")
@ParameterizedTest
@CsvSource({
"If the swift moment I entreat:;Tarry a while! You are so fair!",
"Then forge the shackles to my feet,;Then I will gladly perish there!"
})
void targetByOrder(String line, String line2) {
}
----

The test gets executed once.
The second execution is disabled because the second argument contains "gladly".

=== Using `matches`

If substrings are not powerful enough, you can also use regular expressions, with the `matches` value.

[source,java]
----
// disable invocations whose parameter ends with 'knew' or 'grew'
@DisableIfParameter(matches = { ".*knew", ".*grew" })
@ParameterizedTest
@ValueSource(strings = {
"Lily-like, white as snow,",
"She hardly knew",
"She was a woman, so",
"Sweetly she grew"
})
void interceptMatches(String value) {
}
----

These test invocations get disabled:

* The second invocation, because it has a parameter that matches ".*knew" - ends with knew.
* The fourth invocation, because it has a parameter that matches ".*grew" - ends with grew.

Just like with `contains`, if any parameter value matches any expression from `matches`, the invocation gets disabled.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would add some kind of theader here, because the not is not a note for the particular example above

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't know what you mean. Could you clarify?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For me I see some kind of break between line 244 (Just like..) and 246 (NOTE:) where the sentences are not connected by each other but from the formatting it seems like they belong together.

NOTE: While the documentation uses `String` values for demonstration purposes, you can use it to disable tests with other parameter types.
However, the arguments will be converted to `String` with `Object#toString()` before evaluation.
Make sure that your parameter types have a meaningful `toString` method.

Using both `matches` and `contains` in a single annotation is not permitted.
4 changes: 2 additions & 2 deletions docs/docs-nav.yml
Original file line number Diff line number Diff line change
Expand Up @@ -10,8 +10,8 @@
url: /docs/environment-variables/
- title: "Default Locale and TimeZone"
url: /docs/default-locale-timezone/
- title: "Disable Based on DisplayName"
url: /docs/disable-if-display-name/
- title: "Disable Parameterized Tests"
url: /docs/disable-parameterized-tests/
- title: "Issue information"
url: /docs/issue/
- title: "Publishing Report Entries"
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
/*
* Copyright 2016-2021 the original author or authors.
*
* All rights reserved. This program and the accompanying materials are
* made available under the terms of the Eclipse Public License v2.0 which
* accompanies this distribution and is available at
*
* http://www.eclipse.org/legal/epl-v20.html
*/

package org.junitpioneer.jupiter.params;

import java.lang.annotation.ElementType;
import java.lang.annotation.Retention;
import java.lang.annotation.RetentionPolicy;
import java.lang.annotation.Target;

import org.junit.jupiter.api.extension.ExtendWith;

@Target(ElementType.METHOD)
@Retention(RetentionPolicy.RUNTIME)
@ExtendWith(DisableIfParameterExtension.class)
public @interface DisableIfAllParameters {

String[] contains() default {};

String[] matches() default {};

}