CartesianProductTestExtension from JUnit examples

[Michael1993]

Opened to let us move forward with #68

---

PR checklist

The following checklist shall help the PR's author, the reviewers and maintainers to ensure the quality of this project.
It is based on our contributors guidelines, especially the "writing code" section.
It shall help to check for completion of the listed points.
If a point does not apply to the given PR's changes, the corresponding entry can be simply marked as done.

Documentation (general)

• There is documentation (Javadoc and site documentation; added or updated)
• There is implementation information to describe why a non-obvious source code / solution got implemented
• Site documentation has its own .adoc file in the docs folder, e.g. docs/report-entries.adoc
• Only one sentence per line (especially in .adoc files)
• Javadoc uses formal style, while sites documentation may use informal style (see Documentation: Personal or formal style #265)

Documentation (new extension)

• The docs/docs-nav.yml navigation has an entry for the new extension
• The package-info.java contains information about the new extension

Code

• Code adheres to code style, naming conventions etc.
• Successful tests cover all changes
• There are checks which validate correct / false usage / configuration of a functionality and there are tests to verify those checks (see Require configuration failiure tests? #164)
• Tests use AssertJ or our own PioneerAssert (which are based on AssertJ)

Contributing

• A prepared commit message exists
• The list of contributions inside README.md mentions the new contribution (real name optional)

---

I hereby agree to the terms of the JUnit Pioneer Contributor License Agreement.

[Michael1993]

Things to consider:

• Explicit factory method option (i.e. @CartesianProductTest(factory = "getSources"))
• Define parameter source with a separate annotation
• Supporting @ArgumentsSource annotations (which are not repeatable, apparently) -> relevant JUnit 5 issue
• What should this throw, AssertionError or ExtensionConfigurationException?
• How do different argument sources interact? Should it be illegal to specify multiple argument sources?
• Should CartesianProductTest be moved to org.junitpioneer.jupiter.params instead of org.junitpioneer.jupiter?
• Internally, we use List but should we enforce unique elements? It'd make sense, since {1, 1} ⨯ {2, 3} is functionally equivalent to {1} ⨯ {2, 3}.

[Bukama]

My request changes are mostly about docs. The rest are more questions.

[beatngu13]

I just briefly skimmed over the PR, really like that feature. Personally, I already needed the Cartesian product several times in the past. Kudos @Michael1993!

I have only one question: When it comes to supplying test parameters, I just wonder if it would be helpful to stick to the style of @ParameterizedTest, where the parameter source is defined via a separate annotation?

[Bukama]

Some smaller things in the docs I obviusly missed on the first turn. Sorry for that. After that it's okay for me.

[Bukama]

Then you argue to throw Pioneer Annotations out, because we can use the JUnit things. The idea was Pioneer Annotation was to encapsulate the JUnit thing with our own, improved methods. But I'll leave it to you / the others which way we use.

Same for TestKit or PioneerAssertions 🤷‍♂️

[Michael1993]

When it comes to supplying test parameters, I just wonder if it would be helpful to stick to the style of @ParameterizedTest, where the parameter source is defined via a separate annotation?

@beatngu13 There is a bit of a clash.

When you create a @ParameterizedTest, you supply a single (!) @ArgumentsSource annotation where you define every test case. This is also the reason why annotations annotated with @ArgumentsSource (e.g.: @CsvSource or @ValueSource) are not @Repeatable.

This is different from @CartesianProductTest, where you want to define argument sets for each parameter individually and let the extension define the test cases. This is an entirely different approach from @ParameterizedTest.

We cannot support the annotations used by @ParameterizedTest because we would need them to be @Repeatable.
They never will be repeatable because their design philosophy is different from what we need.

We can create our own annotation(s), (e.g.: @CartesianValueSource, which works) that are repeatable - the question is, how confusing is the difference between these argument sources and their behaviour?

[beatngu13]

@Michael1993 ah, I see. Thanks a lot for that detailed explanation!

I think this is important:

When you create a @ParameterizedTest, you supply a single (!) @ArgumentsSource annotation where you define every test case. […] This is different from @CartesianProductTest, where you want to define argument sets for each parameter individually and let the extension define the test cases.

So, @CartesianProductTest is more like a form of dynamic tests? I just want(ed) to find out if there is an existing JUnit 5 concept / API style we can build upon.

However, the PR is already great, @Bukama likes it and since we are pioneers … 😉 Let's see what @nicolaiparlog thinks.

[nipafx]

A few details...

[nipafx]

Can we get a little more detail on how to use the extension and a link to the website with details?

[nipafx]

The advantage of our annotation methods is that they make it easy to find meta annotations that are _indirectly present: (if a supertype of the element is annotated), meta-present (if an annotation that is present on the element is itself annotated), or _enclosing-present (if an enclosing type [think opposite of @Nested] is annotated). As I see it, only meta-present is relevant here, but that should indeed be allowed. Not sure if AnnotationSupport.findAnnotation does that.

[Michael1993]

From the documentation of AnnotationSupport:

Find the first annotation of annotationType that is either directly present, meta-present, or indirectly present on the supplied element.
If the element is a class and the annotation is neither directly present nor meta-present on the class, this method will additionally search on interfaces implemented by the class before finding an annotation that is indirectly present on the class.

It should be enough to use here.

[sonarcloud]

Kudos, SonarCloud Quality Gate passed!

0 Bugs
0 Vulnerabilities (and 0 Security Hotspots to review)
0 Code Smells

100.0% Coverage
0.0% Duplication