mockito-inline on OpenJDK 17 fails to start due to ByteBuddy agent failure #2436
Comments
|
Looks like this also fails on non-final classes when we use the inline mockmaker. Disabling inline mockmaker entirely resolves the issue. |
|
On further investigation, this appears to affect the version of OpenJDK 17 distributed on Dockerhub also. Will remove GraalVM out of the title since this is not GraalVM-specific. FROM openjdk:17-alpine
COPY . .
RUN java -version
RUN javac -version
RUN ./mvnw -B -q clean packageUpdated POM to use build GAV Attached tarball of code to reproduce for reference. If you need any other details, or if this is mirrored by another issue, please let me know! |
ascopes
changed the title
|
Any updates on this? Currently this is blocking us from using Mockito on JDK 17, as we need the inline mockmaker for other internals. |
|
The exception stacktrace mentions the following, which I think is the root cause: Can you try a different VM and see if that works? FWIW we are running our test suite on JDK 17 and it is passing: mockito/.github/workflows/ci.yml Line 29 in dacb492 |
|
When I ran this, I ran it on the standard OpenJDK provided on the Arch User Repository as well as with Graal. I will recheck this when I have some time, and get back to you! |
|
Closing this as "not reproducible" for now, as we are successfully building on JDK 17. Let us know if you can reproduce this issue specifically with JDK 17. |
This appears to be a persistent issue with GraalVM 17+35-jvmci-21.3-b02; I am just trying to check if there is a more up-to-date version on the AUR for this machine, as I used the version on SDKMAN on another Linux machine, and on a Mac OS machine and it was not reproducible which feels a bit strange. |
|
Can confirm that this seems to be an issue with that build of GraalVM. Updating fixes the issue. |
|
This happens on 17.0.1 (arm64) "Oracle Corporation" - "Java SE 17.0.1" /Library/Java/JavaVirtualMachines/jdk-17.0.1.jdk/Contents/Home but not x86_64 version. Confirmed locally. |
|
I have the same issue: Java : 17
JVM vendor name : Oracle Corporation
JVM vendor version : 17.0.7+7
JVM name : OpenJDK 64-Bit Server VM
JVM version : 17.0.7+7
JVM info : mixed mode
OS name : Linux
OS version : 6.1.33-1-lts |
|
I have same issue: It appears as if your JDK does not supply a working agent attachment mechanism. Please reopen it because it blocking update to Spring Boot 3. |
|
@plmakoc probably worth making a new issue and linking this one. I can replicate it with termux's openjdk build as well on Java 17. |
|
Ok, I will make, becuase I got this issue also using Amazon Correto 17 |
|
Why on earth is this closed!? A long thread of people confirming a serious issue with Mockito, and then it just get closed? |
|
@TimvdLippe is it worth reopening this? |
|
If anybody can submit us a PR with a regression test setup that allows us to debug what is going on here, we can triage this further. For now, we have seen this error pop up as a result of local setup problems (e.g. missing permissions), unrelated to Mockito. Still, if we can have a regression test to show this with Mockito on our CI, we can work on fixing it. |
|
Right now, running this on Termux within Android on OpenJDK 17 seems to replicate this for me (Android 13, ARM) Might be a red herring, but is probably worth looking into. Even if it is a problem with certain Java buillds, providing a more descriptive error message in these cases may be useful (given the complexity of how ByteBuddy and Mockito instrument things internally) |
|
I just figured this out on my machine, it was caused by the security software on my computer killing the Java process. It is flagged as an "OopAllocate" violation (I'm guessing its Out Of Process Memory Allocation or something similar) which triggers the software to terminate the process. You may want to check your Event Viewer Logs to see if there is anything created there when running your project. I was able to work around the issue by running a docker container with If it helps anyone here is the command I used to troubleshoot |
|
This option saved my life |
|
We have the same issue running tests on docker with maven:3-temurin-17 and maven:3-openjdk-17. Running with Edit We had a successful run on using podman on an Ubuntu 22.04 host with kernel 6.2.0 and a temurin-based image. However the same image does not work in our CI pipeline using the same kernel. |
|
With Rancher Desktop on macOS, I discovered I needed to use a |
|
I got the same issue but only as root. |
|
Can reproduce the issue using Mockito 5.10.0 and ByteBuddy 1.14.12 (or 1.14.11), I had to stick with 4.11.0 which works fine. This does not work running on a vmware fusion instance of ubuntu: Ubuntu 20.04.4 LTS This does not work either on openjdk 17. This does work however on host OS, macOS: Darwin Kernel Version 23.3.0: Wed Dec 20 21:30:44 PST 2023; root:xnu-10002.81.5~7/RELEASE_ARM64_T6000 arm64 On Ubuntu, I get: |
|
@sbailliez are you able to possibly provide something like a Vagrantfile that can be used to consistently reproduce this issue? I was only able to reproduce it originally on an older JVM and on Fedora but updating fixed it. The other place I have been able to reproduce it was via Termux but I have since contributed a change that detects the latter edge case. This looks like it is related to attaching from a separate process... could it be SELinux related or something perhaps? i.e. if you ran |
|
@ascopes I will try to find time to investigate this week and either debug or provide a way to reproduce easily |
|
Update :when debugging, I found the bytebuddy system property So you can run which gave me behind the scene the bytebuddy agent tries to run this process: The failure seems to match raphw/byte-buddy#612 I was able to make it work by configuring the surefire plugin to pass in my case: This might be helpful for people looking for a workaround, but trying to dig deeper. |
|
Also adding for context that SELinux is NOT enabled
|
|
I have managed to reproduce the issue and narrowed it down to a file attribute issue. In my current vmware/ubuntu/vagrant setup, I have nfs to share code between the host and guest. Any file that gets created on nfs ends up with a uid/gid that does not match the uid/gid of the current user. I originally used this piece of code to run the bytebuddy agent install: import net.bytebuddy.agent.ByteBuddyAgent;
import com.sun.security.auth.module.UnixSystem;
import java.lang.instrument.Instrumentation;
import java.nio.file.Files;
import java.nio.file.Path;
import java.io.File;
public class Bug {
public void check() throws Exception {
System.out.println("Current pid: " + ProcessHandle.current().pid());
System.out.println("Current command line: " + ProcessHandle.current().info().commandLine().orElse(null));
UnixSystem system = new com.sun.security.auth.module.UnixSystem();
System.out.format("User uid:gid -> %d:%d%n", system.getUid(), system.getGid());
// get the class file path, won't work as is in a jar, that's ugly but good enough for this
File location = new File(getClass().getProtectionDomain().getCodeSource().getLocation().getFile());
Path classFile = new File(location, getClass().getName().replace('.', '/') + ".class").toPath();
System.out.format("Current class file -> %s%n", classFile);
// now get the uid and guid of the current file
int uidFile = (Integer)Files.getAttribute(classFile, "unix:uid");
int gidFile = (Integer)Files.getAttribute(classFile, "unix:gid");
System.out.format("File uid/gid -> %d:%d%n", uidFile, gidFile);
if (system.getUid() != uidFile || system.getGid() != gidFile) {
System.out.println("uid/gid of current user and class do not match. This should FAIL");
} else {
System.out.println("uid/gid of current user and class do match. This should SUCCEED");
}
// let's install the agent...moment of truth
try {
Instrumentation instrumentation = ByteBuddyAgent.install();
System.out.println("SUCCESS");
} catch (Exception e) {
System.out.println("FAILED");
throw e;
}
}
public static void main(String[] args) throws Exception {
Bug test = new Bug();
test.check();
}
}Let's compile it via On the NFS shared folder, I have: -rw-rw-r-- 1 501 dialout 3305 Feb 27 02:48 Bug.class
-rw-rw-r-- 1 501 dialout 2007 Feb 27 02:48 Bug.java
-rw-r--r-- 1 501 dialout 256776 Feb 27 02:48 byte-buddy-agent-1.14.11.jarIf I execute via Current pid: 2331442
Current command line: /usr/lib/jvm/java-21-corretto/amazon-corretto-21.0.1.12.1-linux-aarch64/bin/java -cp byte-buddy-agent-1.14.11.jar:. Bug
User uid:gid -> 1000:1000
Current class file -> /web/tmp/mockito/data/Bug.class
File uid/gid -> 501:20
uid/gid of current user and class do not match. This should FAIL
2024-02-27 03:01:19
Full thread dump OpenJDK 64-Bit Server VM (21.0.1+12-LTS mixed mode, sharing):
[...multiple thread dump ...]whereas on a normal directory I have the vagrant/vagrant user/group When I execute I get: Current pid: 2331599
Current command line: /usr/lib/jvm/java-21-corretto/amazon-corretto-21.0.1.12.1-linux-aarch64/bin/java -cp byte-buddy-agent-1.14.11.jar:. Bug
User uid:gid -> 1000:1000
Current class file -> /home/vagrant/mockito/bug/Bug.class
File uid/gid -> 1000:1000
uid/gid of current user and class do match. This should SUCCEED
WARNING: A Java agent has been loaded dynamically (/home/vagrant/mockito/bug/byte-buddy-agent-1.14.11.jar)
WARNING: If a serviceability tool is in use, please run with -XX:+EnableDynamicAgentLoading to hide this warning
WARNING: If a serviceability tool is not in use, please run with -Djdk.instrument.traceUsage for more information
WARNING: Dynamic loading of agents will be disallowed by default in a future release
SUCCESSff I'm looking at the openjdk code it looks like it is more the temporary file that gets created and then there is a mismatch uid happening. I don't see any easy way to identify this, would be nice if bytebuddy was doing but, otherwise mockito could try creating a file in the working directory and check for resulting uid/gid compare to the user one. |
|
@sbailliez that's interesting... does setting the default permissions of that directory or setting Likely something @TimvdLippe would be able to advise on anyway! |
|
The JVM checks in the working directory for the file it creates named My tmp directory is always in a "good" directory that will behave properly as it is not on nfs. Running with the current working directory on NFS: Eventually in the log, there is a line: If I cd in a non-nfs working directory while pointing to the files on nfs volume (for me |
|
I updated the code to check for the real problem, it creates a temporary file in the current working directory and checks for the user id the same way the jvm does: import net.bytebuddy.agent.ByteBuddyAgent;
import com.sun.security.auth.module.UnixSystem;
import java.lang.instrument.Instrumentation;
import java.nio.file.Files;
import java.nio.file.Path;
import java.io.File;
public class Bug {
public void check() throws Exception {
System.out.println("Current pid: " + ProcessHandle.current().pid());
System.out.println("Current command line: " + ProcessHandle.current().info().commandLine().orElse(null));
UnixSystem system = new com.sun.security.auth.module.UnixSystem();
System.out.format("User uid:gid -> %d:%d%n", system.getUid(), system.getGid());
// get the class file path, won't work as is in a jar, that's ugly but good enough for this
File checkFile = new File(".check_pid%d".formatted(ProcessHandle.current().pid())).getAbsoluteFile();
if (checkFile.createNewFile()) {
System.out.format("Created check file -> %s%n", checkFile);
}
// now get the uid and guid of the current file
int uidFile = (Integer)Files.getAttribute(checkFile.toPath(), "unix:uid");
System.out.format("File uid -> %d%n", uidFile);
if (system.getUid() != uidFile) {
System.out.format("Check file %s has different uid %d (vs %d). This should FAIL%n", checkFile, uidFile, system.getUid());
} else {
System.out.format("Check file %s has identical uid %s. This should SUCCEED%n", checkFile, uidFile);
}
// let's install the agent...moment of truth
try {
Instrumentation instrumentation = ByteBuddyAgent.install();
System.out.println("SUCCESS");
} catch (Exception e) {
System.out.println("FAILED");
throw e;
}
}
public static void main(String[] args) throws Exception {
Bug test = new Bug();
test.check();
}
}When I execute from a working directory on nfs I get the uid mismatch as described above: |
|
To me this indeed sounds like a feature improvement for ByteBuddy to improve error reporting here. As Mockito, we consume the ByteBuddy package for instrumentation. If instrumentation fails, we rely on ByteBuddy to provide that error. I see you filed raphw/byte-buddy#1602 which sounds like a good feature improvement. |
It appears that GraalVM JDK 17 does not allow inline mocks of final classes to be made using mockito-inline or by adding the mockito inline extension file.
Not sure if this is the same issue as #2315 or #2122, so I have opened this to try and get some further clarification.
I have included the full code to reproduce, as well as the full exception trace itself below:
src/test/java/mockito/bug/FinalClassMockTest.javamockito/bug/FinalClass.javapom.xmlFull error:
The text was updated successfully, but these errors were encountered: