Blackbird fails with LinkageError when the same class is used across two separate classloaders

[davidconnard]

The new Blackbird module has an issue when it is used to serialise the same class, but across two separate classloaders.

eg. we are serialising class a.b.c from an ObjectMapper that is configured to use Blackbird, from within a library. CrossLoader access does it's thing, and:

• attempts to instantiate class a.b.c$$JacksonBlackbirdAccess
• ClassNotFoundException is thrown, and this instance of CrossLoader attempts to define it

Next, we attempt to deserialise from an ObjectMapper that is configured to use Blackbird, but from within the main application. The classloader is different. CrossLoader again attempts to do it's thing, and:

• attempts to instantiate class a.b.c$$JacksonBlackbirdAccess
• ClassNotFoundException is thrown again (because this classloader doesn't have it, and nor does the parent classloader), and this other instance of CrossLoader attempts to define it
• defineClass fails, as the parent ClassLoader already has it defined

Seems to me like your classname - JacksonBlackbirdAccess - cannot be static, but must include an object reference to the current classloader? ie. allow each copy of JacksonBlackbird CrossLoader to have it's own separate definition of your helper class.

an example stacktrace:

org.opentest4j.AssertionFailedError: Unexpected exception thrown: java.lang.LinkageError: loader 'app' (instance of jdk.internal.loader.ClassLoaders$AppClassLoader) attempted duplicate class definition for a.b.c.$$JacksonBlackbirdAccess.
	at app//org.junit.jupiter.api.AssertDoesNotThrow.createAssertionFailedError(AssertDoesNotThrow.java:83)
	at app//org.junit.jupiter.api.AssertDoesNotThrow.assertDoesNotThrow(AssertDoesNotThrow.java:54)
	at app//org.junit.jupiter.api.AssertDoesNotThrow.assertDoesNotThrow(AssertDoesNotThrow.java:37)
	at app//org.junit.jupiter.api.Assertions.assertDoesNotThrow(Assertions.java:3060)
	at app//a.b.c.runSomeTest(c.java:78)
	at java.base@11.0.3/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base@11.0.3/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base@11.0.3/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base@11.0.3/java.lang.reflect.Method.invoke(Method.java:566)
	at app//org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:688)
	at app//org.junit.jupiter.engine.execution.MethodInvocation.proceed(MethodInvocation.java:60)
	at app//org.junit.jupiter.engine.execution.InvocationInterceptorChain$ValidatingInvocation.proceed(InvocationInterceptorChain.java:131)
	at app//org.junit.jupiter.engine.extension.TimeoutExtension.intercept(TimeoutExtension.java:149)
	at app//org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestableMethod(TimeoutExtension.java:140)
	at app//org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestTemplateMethod(TimeoutExtension.java:92)
	at app//org.junit.jupiter.engine.execution.ExecutableInvoker$ReflectiveInterceptorCall.lambda$ofVoidMethod$0(ExecutableInvoker.java:115)
	at app//org.junit.jupiter.engine.execution.ExecutableInvoker.lambda$invoke$0(ExecutableInvoker.java:105)
	at app//org.junit.jupiter.engine.execution.InvocationInterceptorChain$InterceptedInvocation.proceed(InvocationInterceptorChain.java:106)
	at app//org.junit.jupiter.engine.execution.InvocationInterceptorChain.proceed(InvocationInterceptorChain.java:64)
	at app//org.junit.jupiter.engine.execution.InvocationInterceptorChain.chainAndInvoke(InvocationInterceptorChain.java:45)
	at app//org.junit.jupiter.engine.execution.InvocationInterceptorChain.invoke(InvocationInterceptorChain.java:37)
	at app//org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:104)
	at app//org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:98)
	at app//org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.lambda$invokeTestMethod$6(TestMethodTestDescriptor.java:210)
	at app//org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at app//org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.invokeTestMethod(TestMethodTestDescriptor.java:206)
	at app//org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:131)
	at app//org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:65)
	at app//org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:139)
	at app//org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at app//org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$7(NodeTestTask.java:129)
	at app//org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137)
	at app//org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:127)
	at app//org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at app//org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:126)
	at app//org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:84)
	at app//org.junit.platform.engine.support.hierarchical.ForkJoinPoolHierarchicalTestExecutorService$ExclusiveTask.compute(ForkJoinPoolHierarchicalTestExecutorService.java:185)
	at java.base@11.0.3/java.util.concurrent.RecursiveAction.exec(RecursiveAction.java:189)
	at java.base@11.0.3/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290)
	at java.base@11.0.3/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1020)
	at java.base@11.0.3/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1656)
	at java.base@11.0.3/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1594)
	at java.base@11.0.3/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:177)
Caused by: java.lang.LinkageError: loader 'app' (instance of jdk.internal.loader.ClassLoaders$AppClassLoader) attempted duplicate class definition for a.b.c.$$JacksonBlackbirdAccess.
	at java.base/java.lang.ClassLoader.defineClass1(Native Method)
	at java.base/java.lang.System$2.defineClass(System.java:2123)
	at java.base/java.lang.invoke.MethodHandles$Lookup.defineClass(MethodHandles.java:962)
	at com.fasterxml.jackson.module.blackbird.CrossLoaderAccess.accessClassIn(CrossLoaderAccess.java:125)
	at com.fasterxml.jackson.module.blackbird.CrossLoaderAccess.grantAccess(CrossLoaderAccess.java:94)
	at com.fasterxml.jackson.module.blackbird.CrossLoaderAccess.apply(CrossLoaderAccess.java:83)
	at com.fasterxml.jackson.module.blackbird.CrossLoaderAccess.apply(CrossLoaderAccess.java:11)
	at com.fasterxml.jackson.module.blackbird.ser.BBSerializerModifier.createProperty(BBSerializerModifier.java:109)
	at com.fasterxml.jackson.module.blackbird.ser.BBSerializerModifier.lambda$findProperties$0(BBSerializerModifier.java:67)
	at com.fasterxml.jackson.module.blackbird.util.Unchecked.lambda$runnable$0(Unchecked.java:31)
	at com.fasterxml.jackson.module.blackbird.ser.BBSerializerModifier.findProperties(BBSerializerModifier.java:68)
	at com.fasterxml.jackson.module.blackbird.ser.BBSerializerModifier.changeProperties(BBSerializerModifier.java:52)
	at com.fasterxml.jackson.databind.ser.BeanSerializerFactory.constructBeanOrAddOnSerializer(BeanSerializerFactory.java:414)
	at com.fasterxml.jackson.databind.ser.BeanSerializerFactory.findBeanOrAddOnSerializer(BeanSerializerFactory.java:294)
	at com.fasterxml.jackson.databind.ser.BeanSerializerFactory._createSerializer2(BeanSerializerFactory.java:239)
	at com.fasterxml.jackson.databind.ser.BeanSerializerFactory.createSerializer(BeanSerializerFactory.java:173)
	at com.fasterxml.jackson.databind.SerializerProvider._createUntypedSerializer(SerializerProvider.java:1495)
	at com.fasterxml.jackson.databind.SerializerProvider._createAndCacheUntypedSerializer(SerializerProvider.java:1443)
	at com.fasterxml.jackson.databind.SerializerProvider.findValueSerializer(SerializerProvider.java:544)
	at com.fasterxml.jackson.databind.SerializerProvider.findTypedValueSerializer(SerializerProvider.java:822)
	at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(DefaultSerializerProvider.java:308)
	at com.fasterxml.jackson.databind.ObjectMapper._writeValueAndClose(ObjectMapper.java:4568)
	at com.fasterxml.jackson.databind.ObjectMapper.writeValueAsString(ObjectMapper.java:3821)

[chengchen]

Experienced exactly the same 👍
It seems to be rather a race condition when trying to define the class for the first time.

This "lookup and define" operation should probably be synchronized at some point (maybe a lookup cache is needed to avoid performance penalty):

    private static Class<?> accessClassIn(MethodHandles.Lookup lookup) throws IOException, ReflectiveOperationException {
        Package pkg = lookup.lookupClass().getPackage();
        try {
            return Class.forName(pkg.getName() + "." + CLASS_NAME, true, lookup.lookupClass().getClassLoader());
        } catch (ClassNotFoundException ign) { }
        String fqcn = pkg.getName()
                .replace('.', '/')
            + "/" + CLASS_NAME;
        ByteArrayOutputStream classBytes = new ByteArrayOutputStream(HEADER.length + FOOTER.length + fqcn.length() + 16);
        DataOutputStream dataOut = new DataOutputStream(classBytes);
        for (int b : HEADER) {
            dataOut.writeByte(b);
        }
        dataOut.writeUTF(fqcn);
        for (int b : FOOTER) {
            dataOut.writeByte(b);
        }
        try {
            return (Class<?>) DEFINE_CLASS.invokeExact(lookup, classBytes.toByteArray());
        } catch (RuntimeException | Error | IOException | ReflectiveOperationException e) {
            throw e;
        } catch (Throwable e) {
            throw new RuntimeException(e);
        }
    }

[josephlbarnett]

can confirm we're seeing this with the same classloader being used, just a race condition between the lookup and define when running tests in parallel. Is there a workaround beyond disabling the Blackbird module?

[chengchen]

Hi @josephlbarnett You could downgrade to the previous version or override this problematic class in your application's class path (by correctly implementing the lookup and define).