Running tests where the test suite has a module descriptor

[io7m]

I need to run my tests in module-aware mode. This means that the junit artifacts are on the module-path, and the classpath is actually empty. I need to run tests this way because I don't support running any of my libraries in classpath mode (and so ServiceLoader services are only declared in module descriptors).

Using junit 5.10.2, running tests results in the following error:

Exception in thread "main" java.lang.IllegalAccessError: class org.junit.platform.launcher.TestIdentifier (in unnamed module @0x87f383f) cannot access class org.junit.platform.commons.util.Preconditions (in module org.junit.platform.commons) because module org.junit.platform.commons does not export org.junit.platform.commons.util to unnamed module @0x87f383f
	at org.junit.platform.launcher.TestIdentifier.from(TestIdentifier.java:68)
	at com.intellij.junit5.JUnit5IdeaTestRunner.<clinit>(JUnit5IdeaTestRunner.java:72)
	at java.base/java.lang.Class.forName0(Native Method)
	at java.base/java.lang.Class.forName(Class.java:421)
	at java.base/java.lang.Class.forName(Class.java:412)
	at com.intellij.rt.junit.JUnitStarter.getAgentClass(JUnitStarter.java:241)
	at com.intellij.rt.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:222)
	at com.intellij.rt.junit.JUnitStarter.main(JUnitStarter.java:55)

This can be worked around if the test suite is run with the following arguments:

--add-opens org.junit.platform.commons/org.junit.platform.commons.logging=ALL-UNNAMED --add-opens org.junit.platform.commons/org.junit.platform.commons.util=ALL-UNNAMED

But this is a pain, and means that the default IDE settings are never correct.

Steps to reproduce

Run tests in an IDE such as Intellij IDEA. A good example of a test suite I have with module descriptors is idstore:

https://www.github.com/io7m-com/idstore

Right click the com.io7m.idstore.tests module and "Run all tests in com.io7m.idstore.tests...". You'll immediately see the above error.

Context

• JUnit Jupiter 5.10.2, JDK 21
• Intellij IDEA, Maven

Deliverables

Tests should run in module-path mode without any special arguments.

[io7m]

To be clear: I've been using this workaround for about two years now. That's a long time to carry a workaround from project to project to project. 😅

[sormuras]

[...] because module org.junit.platform.commons does not export org.junit.platform.commons.util to unnamed module @0x87f383f

Something IS running on the classpath: unnamed module @0x87f383f ... your test code?

[sormuras]

By the way, when you publish your modules to Maven Central, my module census receives large increases:

4389861 6556 unique modules (2024.05.12)
9195f9a 6474 unique modules (2024.05.10)

[sormuras]

Looking at com.io7m.idstore.tests/src/main/java/module-info.java

open module com.io7m.idstore.tests

shows that you're defining this module as "open" to deep reflection for testing by every other named module: good!

JUnit's modules are read: good

  requires org.junit.jupiter.api;
  requires org.junit.jupiter.engine;
  requires org.junit.platform.commons;
  requires org.junit.platform.engine;

With org.junit.jupiter.engine and org.junit.platform.engine normally not showing up here; but that's fine.

Hm? idstore/com.io7m.idstore.tests/src/main/java/module-info.java is under src/main/java - shouldn't make a difference for IDEA, I guess.

What does the actual command-line generated by IDEA look like?

[sormuras]

Looking through the requires directives again, I see this list of "external" (non system, non com.io7m, non-junit) module names:

open module com.io7m.idstore.tests {
  // ...
  requires com.helger.css;
  // ...
  requires freemarker;
  requires io.helidon.webserver;
  requires io.opentelemetry.api;
  requires jakarta.mail;
  // ...
  requires net.bytebuddy.agent;
  requires net.bytebuddy;
  requires net.jqwik.api;
  requires org.mockito;
  requires org.postgresql.jdbc;
  requires org.slf4j;
  requires subethasmtp;
  // ...
}

Are all of them playing nice on the module path?

[sormuras]

This means that the junit artifacts are on the module-path, and the classpath is actually empty.

Those JUnit (and friends) artifacts land on the classpath:

• junit-platform-launcher-1.10.2.jar
• junit-platform-engine-1.10.2.jar
• opentest4j-1.3.0.jar
• junit-platform-commons-1.10.2.jar
• apiguardian-api-1.1.2.jar

And the classpath contains a lot more elements. For a full list, see below.

What does the actual command-line generated by IDEA look like?

After cloning and running "Maven compile" I get for "Run 'All tests'" an IDEA launch configuration named All in com.io7m.idstore.tests which uses those command-line arguments:

• https://gist.github.com/sormuras/f129a7a0c654bb49226d90a1ca945e83

If only there was an option reading Do not use --class-path option

https://youtrack.jetbrains.com/issue/IDEA-277330/Do-not-use-class-path-when-Run-All-Tests-of-a-Java-module

[sormuras]

Reading up on the work-arounds noted in IDEA-277330 I get the following results after adding module org.junit.platform.launcher to the list of required modules:

Here's the patch I used:

Index: com.io7m.idstore.tests/src/main/java/module-info.java
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
diff --git a/com.io7m.idstore.tests/src/main/java/module-info.java b/com.io7m.idstore.tests/src/main/java/module-info.java
--- a/com.io7m.idstore.tests/src/main/java/module-info.java	(revision d71d7958cccefe4fa63e9b4bd0cb59b356d59394)
+++ b/com.io7m.idstore.tests/src/main/java/module-info.java	(date 1715500432539)
@@ -80,6 +80,7 @@
   requires org.junit.jupiter.engine;
   requires org.junit.platform.commons;
   requires org.junit.platform.engine;
+  requires org.junit.platform.launcher;
   requires com.io7m.blackthorne.core;
 
   exports com.io7m.idstore.tests.database;
Index: pom.xml
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
diff --git a/pom.xml b/pom.xml
--- a/pom.xml	(revision d71d7958cccefe4fa63e9b4bd0cb59b356d59394)
+++ b/pom.xml	(date 1715500618817)
@@ -116,6 +116,7 @@
     <org.jline.version>3.25.1</org.jline.version>
     <org.jooq.version>3.19.8</org.jooq.version>
     <org.junit.version>5.10.2</org.junit.version>
+    <org.junit-platform.version>1.10.2</org.junit-platform.version>
     <org.mockito.version>5.12.0</org.mockito.version>
     <org.postgresql.version>42.7.3</org.postgresql.version>
     <org.slf4j.version>2.0.13</org.slf4j.version>
@@ -599,6 +600,11 @@
         <groupId>org.junit.jupiter</groupId>
         <artifactId>junit-jupiter-engine</artifactId>
         <version>${org.junit.version}</version>
+      </dependency>
+      <dependency>
+        <groupId>org.junit.platform</groupId>
+        <artifactId>junit-platform-launcher</artifactId>
+        <version>${org.junit-platform.version}</version>
       </dependency>
       <dependency>
         <groupId>net.jqwik</groupId>
Index: com.io7m.idstore.tests/pom.xml
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
diff --git a/com.io7m.idstore.tests/pom.xml b/com.io7m.idstore.tests/pom.xml
--- a/com.io7m.idstore.tests/pom.xml	(revision d71d7958cccefe4fa63e9b4bd0cb59b356d59394)
+++ b/com.io7m.idstore.tests/pom.xml	(date 1715500432539)
@@ -237,6 +237,10 @@
       <artifactId>junit-jupiter-engine</artifactId>
     </dependency>
     <dependency>
+      <groupId>org.junit.platform</groupId>
+      <artifactId>junit-platform-launcher</artifactId>
+    </dependency>
+    <dependency>
       <groupId>org.mockito</groupId>
       <artifactId>mockito-core</artifactId>
     </dependency>

[sormuras]

Note that with using JUnit's BOM as described here https://junit.org/junit5/docs/current/user-guide/#running-tests-build-maven you may keep defining the single <org.junit.version>5.10.2</org.junit.version> constant.

And should be able to remove the direct dependencies to Jupiter and Platform .engine modules.

[io7m]

Hello!

Hm? idstore/com.io7m.idstore.tests/src/main/java/module-info.java is under src/main/java - shouldn't make a difference for IDEA, I guess.

This is the one way I diverge from Maven's conventions. I usually have a single module containing all of the tests for all of the other modules, and so having a separate src/test directory within that one module causes other issues with the tools (they tend to get upset when there's an empty src/main). I started putting tests in src/main for the tests module, and that seemed to generally improve things.

And the classpath contains a lot more elements. For a full list, see below.

Sorry, this was a very poor choice of projects on my part. Usually the classpath is empty (aside from the JUnit artifacts you pointed out), but idstore was the most recent project I released and so of course I managed to pick the project with the very much non-empty classpath. 🤦‍♂️

After cloning and running "Maven compile" I get for "Run 'All tests'" an IDEA launch configuration named All in com.io7m.idstore.tests which uses those command-line arguments:

Thanks for doing all this investigation. I posted this ticket last thing at night and didn't expect to get a response for a few days... Woke up to this forensic examination. 😆

Note that with using JUnit's BOM as described here...

I'll give it a shot. It would be nice to eliminate the separate versioning.

[io7m]

Here's the patch I used:

THANK YOU!

That missing requires org.junit.platform.launcher; was definitely the culprit. I've tried this on multiple projects now, and the IDE always does the right thing and I no longer have to add the workaround everywhere.

🎉

[sbrannen]

Hi @io7m,

Thanks for letting us know that worked for you.