How to run JUnit 5 @Suite tests via Maven Surefire #2753

praveenreddynarala · 2021-10-26T12:55:27Z

praveenreddynarala
Oct 26, 2021

Hi Team,

I am unable to run Junit 5 Test Suite (@suite) using the Maven command line. It always says, Total tests 0. I have tried various options/configurations from the maven-surefire plugin v3.0.0-M5.

Sample Test Suite:

package com.test.testsuites;

import com.test.qa.junit5.MCTest;
import junit5.tags.ArchiveSuite;
import org.junit.platform.suite.api.IncludeClassNamePatterns;
import org.junit.platform.suite.api.IncludeTags;
import org.junit.platform.suite.api.SelectPackages;
import org.junit.platform.suite.api.Suite;
import org.junit.platform.suite.api.SuiteDisplayName;

@Suite
@SuiteDisplayName("Junit first test suite")
@SelectPackages(
        {
                "com.test.archive",
                "com.test.legalhold"
        })
@IncludeClassNamePatterns(".*Tests")
@IncludeTags("Archive")
@ArchiveSuite
public class ArchiveTestSuite extends MCTest {

}

1 Option: I have used Junit 5 Paralallism configuration inside the Surefire plugin as below to run all the tests from inside the test suite in parallel. None of the tests was picked up. It says no tests to run.

<plugins>
            <plugin>
                <artifactId>maven-compiler-plugin</artifactId>
                <version>${maven.compiler.plugin}</version>
                <configuration>
                    <source>${java.version}</source>
                    <target>${java.version}</target>
                </configuration>
            </plugin>
            <plugin>
                <artifactId>maven-surefire-plugin</artifactId>
                <version>${surefire.plugin.version}</version>
                <configuration>
                    <trimStackTrace>false</trimStackTrace>
                    <includes>
                        <include>**/*TestSuite.java</include>
                        <include>**/*TestSuites.java</include>
                    </includes>
                    <properties>
                        <groups>${Tags}</groups>
                        <excludedGroups>Skip, WIP</excludedGroups>
                        <!-- Parallel execution using Junit 5 -->
                        <configurationParameters>
                            junit.jupiter.conditions.deactivate = *
                            junit.jupiter.extensions.autodetection.enabled = true
                            junit.jupiter.execution.parallel.enabled = true
                            junit.jupiter.execution.parallel.mode.default = concurrent
                            junit.jupiter.execution.parallel.config.strategy=fixed
                            junit.jupiter.execution.parallel.config.fixed.parallelism=6
                            junit.jupiter.testinstance.lifecycle.default = per_class
                        </configurationParameters>
                    </properties>
                    <systemPropertyVariables>

                        <!-- https://bugs.openjdk.java.net/browse/JDK-8212755 -->
                        <java.locale.providers>COMPAT</java.locale.providers>
                    </systemPropertyVariables>
                </configuration>
            </plugin>

2nd Option: Using Maven Surefire Forks to run the tests inside the test suite in parallel with the below configuration from the POM.xml file.

And used mvn -Dgroups=Test command to run the tests using Tags. No tests were detected.

Response from Maven

[INFO] --- maven-surefire-plugin:3.0.0-M5:test (default-test) @ api-archive ---
[INFO] 
[INFO] -------------------------------------------------------
[INFO]  T E S T S
[INFO] -------------------------------------------------------
[INFO] 
[INFO] Results:
[INFO] 
[INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0

POM.XML

<plugin>
                <artifactId>maven-surefire-plugin</artifactId>
                <version>${surefire.plugin.version}</version>
                <configuration>
                    <trimStackTrace>false</trimStackTrace>
                    <forkCount>4</forkCount>
                    <reuseForks>true</reuseForks>
                    <parallel>suiteAndClasses</parallel>
                    <includes>
                        <include>**/*TestSuite.java</include>
                        <include>**/*TestSuites.java</include>
                    </includes>
                    <properties>
                        <groups>${Tags}</groups>
                        <excludedGroups>Skip, WIP</excludedGroups>
                    </properties>
                    <systemPropertyVariables>

                        <!-- https://bugs.openjdk.java.net/browse/JDK-8212755 -->
                        <java.locale.providers>COMPAT</java.locale.providers>
                    </systemPropertyVariables>
                </configuration>
            </plugin>

3rd Option: I have used the same configuration as like above POM.xml, but I have used Test Cases Tag name instead of Test Suite Tag name. Now the test cases were run in sequential, however, the test suite still shows 0 Total Test Cases.

Thank you.
Praveen

Answered by sormuras

Oct 27, 2021

Taking https://github.com/junit-team/junit5-samples/blob/main/junit5-jupiter-starter-maven/pom.xml as a common base and applying the following patch:

Index: junit5-jupiter-starter-maven/pom.xml
===================================================================
diff --git a/junit5-jupiter-starter-maven/pom.xml b/junit5-jupiter-starter-maven/pom.xml
--- a/junit5-jupiter-starter-maven/pom.xml	(revision 0aa3e933a3e90d3d6175c8f1da0c63283348d263)
+++ b/junit5-jupiter-starter-maven/pom.xml	(date 1635318636736)
@@ -31,6 +31,11 @@
 			<artifactId>junit-jupiter</artifactId>
 			<scope>test</scope>
 		</dependency>
+		<dependency>
+			<groupId>org.junit.platform</groupId>
+			<artifactId>junit-plat…

View full answer

sormuras · 2021-10-26T14:25:15Z

sormuras
Oct 26, 2021
Collaborator

Converted into a discussion, as it is not an issue with a component of JUnit.

I'll try to find an answer later this week. Although I think that the current implementation of Surefire's JUnit Platform Provider is not capable of running suites out-of-the-box.

Note to myself: Does it work with https://github.com/sormuras/junit-platform-maven-plugin ?

1 reply

ChakotayChou Oct 27, 2021

I am learning JUnit 5 (5.8.1). When I learn and try @suite, the suite class cannot find any tests. I don't know if it is the same cause? If I go back to use RunWith(JUnitPlatform.class) and it will find all tests.
Also, I search the google, find no @suite workable example. All the existed examples I found are using "RunWith(JUnitPlatform.class)".
https://stackoverflow.com/questions/69732084/what-is-the-correct-way-to-create-test-suite-in-junit-5-5-8-1

Hope someone can help to provide a workable @suite example.

sormuras · 2021-10-27T07:30:41Z

sormuras
Oct 27, 2021
Collaborator

Taking https://github.com/junit-team/junit5-samples/blob/main/junit5-jupiter-starter-maven/pom.xml as a common base and applying the following patch:

Index: junit5-jupiter-starter-maven/pom.xml
===================================================================
diff --git a/junit5-jupiter-starter-maven/pom.xml b/junit5-jupiter-starter-maven/pom.xml
--- a/junit5-jupiter-starter-maven/pom.xml	(revision 0aa3e933a3e90d3d6175c8f1da0c63283348d263)
+++ b/junit5-jupiter-starter-maven/pom.xml	(date 1635318636736)
@@ -31,6 +31,11 @@
 			<artifactId>junit-jupiter</artifactId>
 			<scope>test</scope>
 		</dependency>
+		<dependency>
+			<groupId>org.junit.platform</groupId>
+			<artifactId>junit-platform-suite</artifactId>
+			<scope>test</scope>
+		</dependency>
 	</dependencies>
 
 	<build>
@@ -42,6 +47,11 @@
 			<plugin>
 				<artifactId>maven-surefire-plugin</artifactId>
 				<version>2.22.2</version>
+				<configuration>
+					<includes>
+						<include>**/MySuite.java</include>
+					</includes>
+				</configuration>
 			</plugin>
 		</plugins>
 	</build>
Index: junit5-jupiter-starter-maven/src/test/java/com/example/MySuite.java
===================================================================
diff --git a/junit5-jupiter-starter-maven/src/test/java/com/example/MySuite.java b/junit5-jupiter-starter-maven/src/test/java/com/example/MySuite.java
new file mode 100644
--- /dev/null	(date 1635318673680)
+++ b/junit5-jupiter-starter-maven/src/test/java/com/example/MySuite.java	(date 1635318673680)
@@ -0,0 +1,9 @@
+package com.example;
+
+import org.junit.platform.suite.api.SelectPackages;
+import org.junit.platform.suite.api.Suite;
+
+@Suite
+@SelectPackages("com.example.project")
+public class MySuite {
+}

Running mvn test yields:

...

[INFO] --- maven-surefire-plugin:2.22.2:test (default-test) @ junit5-jupiter-starter-maven ---
[INFO] 
[INFO] -------------------------------------------------------
[INFO]  T E S T S
[INFO] -------------------------------------------------------
[INFO] Running MySuite
[INFO] Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.078 s - in MySuite
[INFO] 
[INFO] Results:
[INFO]
[INFO] Tests run: 5, Failures: 0, Errors: 0, Skipped: 0
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------

Ergo, running @Suite annotated classes works with Surefire. Using Surefire's current milestone version 3.0.0-M5 should also work.

As Marc mentioned here #2749 (comment) Surefire doesn't support running tests in parallel on the JUnit Platform.

Answer to myself: sormuras/junit-platform-maven-plugin@75be966 ✅

9 replies

sormuras Oct 27, 2021
Collaborator

Yes, I want to filter subsets (Test classes) from a Test Suite using Test Case Tags.

No, not with a test runner like Surefire. Your suite is in control here, and only there you may statically configure/filter/etc. what you want.

And, would it b possible to run subsest of test classes in parallel?

I don't think so.

praveenreddynarala Oct 27, 2021
Author

Yes, I want to filter subsets (Test classes) from a Test Suite using Test Case Tags.

No, not with a test runner like Surefire. Your suite is in control here, and only there you may statically configure/filter/etc. what you want.

And, would it b possible to run subsets of test classes in parallel?

I don't think so.

However, it would be good to have these features from Junit 5 configurations at least when we set these things inside maven surefire. However, I have found one thing, when I run the test suite directly from the class and if I set Junit 5 parallelism properties from the properties file in src/test/resources. All subset (Test cases from the target package) of that test Suite is running in parallel.

floscher Nov 3, 2021

If the name of the @Suite class ends with Test, you can even leave out the includes in the surefire configuration. I only added the junit-platform-suite and renamed the class to SuiteTest and it was picked up.

marcphilipp Nov 3, 2021
Maintainer

You'll have to make sure your regular tests aren't executed twice then.

liujinhui1994 May 24, 2022

@praveenreddynarala @IncludeTags("functional") When I use this tag, Test doesn't work properly. Could not detect Test, how did you do it?

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

How to run JUnit 5 @Suite tests via Maven Surefire #2753

{{title}}

{{editor}}'s edit

{{editor}}'s edit

Replies: 2 comments 10 replies

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

Select a reply

How to run JUnit 5 @Suite tests via Maven Surefire #2753

praveenreddynarala Oct 26, 2021

Replies: 2 comments · 10 replies

sormuras Oct 26, 2021 Collaborator

ChakotayChou Oct 27, 2021

sormuras Oct 27, 2021 Collaborator

sormuras Oct 27, 2021 Collaborator

praveenreddynarala Oct 27, 2021 Author

floscher Nov 3, 2021

marcphilipp Nov 3, 2021 Maintainer

liujinhui1994 May 24, 2022

praveenreddynarala
Oct 26, 2021

Replies: 2 comments 10 replies

sormuras
Oct 26, 2021
Collaborator

sormuras
Oct 27, 2021
Collaborator

sormuras Oct 27, 2021
Collaborator

praveenreddynarala Oct 27, 2021
Author

marcphilipp Nov 3, 2021
Maintainer