Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Avoid BlockHound NoSuchTypeException when context propagation dependency is unavailable #3337

Merged

Conversation

pderop
Copy link
Member

@pderop pderop commented Jan 29, 2023

When the context-propagation jar is unavailable (which can happen since the dependency is optional), then BlockHound reports the following ERROR log:

BlockHoundTest > testAsByteBuffer() STANDARD_ERROR
    [Byte Buddy] ERROR reactor.core.publisher.ContextPropagation [jdk.internal.loader.ClassLoaders$AppClassLoader@531d72ca, unnamed module @49c9930c, Thread[Test worker,5,main], loaded=false]
    reactor.blockhound.shaded.net.bytebuddy.pool.TypePool$Resolution$NoSuchTypeException: Cannot resolve type description for io.micrometer.context.ContextRegistry
        at reactor.blockhound.shaded.net.bytebuddy.pool.TypePool$Resolution$Illegal.resolve(TypePool.java:167)
        at reactor.blockhound.shaded.net.bytebuddy.pool.TypePool$Default$LazyTypeDescription$TokenizedGenericType.toErasure(TypePool.java:6877)
        at reactor.blockhound.shaded.net.bytebuddy.pool.TypePool$Default$LazyTypeDescription$LazyTypeList.get(TypePool.java:6684)
        at reactor.blockhound.shaded.net.bytebuddy.pool.TypePool$Default$LazyTypeDescription$LazyTypeList.get(TypePool.java:6657)
        at java.base/java.util.AbstractList$Itr.next(AbstractList.java:371)
        at reactor.blockhound.shaded.net.bytebuddy.description.method.MethodDescription$AbstractBase.getDescriptor(MethodDescription.java:524)
        at reactor.blockhound.shaded.net.bytebuddy.asm.AsmVisitorWrapper$ForDeclaredMethods.wrap(AsmVisitorWrapper.java:493)
        at reactor.blockhound.shaded.net.bytebuddy.asm.AsmVisitorWrapper$Compound.wrap(AsmVisitorWrapper.java:746)
        at reactor.blockhound.shaded.net.bytebuddy.dynamic.scaffold.TypeWriter$Default$ForInlining$WithDecorationOnly$DecorationClassVisitor.visit(TypeWriter.java:5808)
        at reactor.blockhound.shaded.net.bytebuddy.jar.asm.ClassReader.accept(ClassReader.java:569)
        at reactor.blockhound.shaded.net.bytebuddy.jar.asm.ClassReader.accept(ClassReader.java:424)
        at reactor.blockhound.shaded.net.bytebuddy.dynamic.scaffold.TypeWriter$Default$ForInlining.create(TypeWriter.java:4014)
        at reactor.blockhound.shaded.net.bytebuddy.dynamic.scaffold.TypeWriter$Default.make(TypeWriter.java:2224)
        at reactor.blockhound.shaded.net.bytebuddy.dynamic.DynamicType$Builder$AbstractBase$UsingTypeWriter.make(DynamicType.java:4057)
        at reactor.blockhound.shaded.net.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer.doTransform(AgentBuilder.java:12106)
        at reactor.blockhound.shaded.net.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer.transform(AgentBuilder.java:12041)
        at reactor.blockhound.shaded.net.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer.access$1800(AgentBuilder.java:11758)
        at reactor.blockhound.shaded.net.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer$Java9CapableVmDispatcher.run(AgentBuilder.java:12521)
        at reactor.blockhound.shaded.net.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer$Java9CapableVmDispatcher.run(AgentBuilder.java:12453)
        at java.base/java.security.AccessController.doPrivileged(AccessController.java:399)
        at reactor.blockhound.shaded.net.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer.doPrivileged(AgentBuilder.java)
        at reactor.blockhound.shaded.net.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer.transform(AgentBuilder.java:11984)
        at reactor.blockhound.shaded.net.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer$ByteBuddy$ModuleSupport.transform(Unknown Source)
        at java.instrument/sun.instrument.TransformerManager.transform(TransformerManager.java:188)
        at java.instrument/sun.instrument.InstrumentationImpl.transform(InstrumentationImpl.java:541)
        at java.base/java.lang.ClassLoader.defineClass1(Native Method)
        at java.base/java.lang.ClassLoader.defineClass(ClassLoader.java:1012)
        at java.base/java.security.SecureClassLoader.defineClass(SecureClassLoader.java:150)
        at java.base/jdk.internal.loader.BuiltinClassLoader.defineClass(BuiltinClassLoader.java:862)
        at java.base/jdk.internal.loader.BuiltinClassLoader.findClassOnClassPathOrNull(BuiltinClassLoader.java:760)
        at java.base/jdk.internal.loader.BuiltinClassLoader.loadClassOrNull(BuiltinClassLoader.java:681)
        at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:639)
        at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:188)
        at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:520)
        at reactor.core.publisher.FluxHandle.subscribeOrReturn(FluxHandle.java:48)
        at reactor.core.publisher.Mono.subscribe(Mono.java:4460)
        at reactor.core.publisher.Mono.subscribeWith(Mono.java:4541)
        at reactor.core.publisher.Mono.subscribe(Mono.java:4442)
        at reactor.core.publisher.Mono.subscribe(Mono.java:4378)
        at reactor.core.publisher.Mono.subscribe(Mono.java:4325)
        at reactor.netty5.BlockHoundTest.testAsByteBuffer(BlockHoundTest.java:31)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:568)
        at org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:727)
        at org.junit.jupiter.engine.execution.MethodInvocation.proceed(MethodInvocation.java:60)
        at org.junit.jupiter.engine.execution.InvocationInterceptorChain$ValidatingInvocation.proceed(InvocationInterceptorChain.java:131)
        at org.junit.jupiter.engine.extension.TimeoutExtension.intercept(TimeoutExtension.java:156)
        at org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestableMethod(TimeoutExtension.java:147)
        at org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestMethod(TimeoutExtension.java:86)
        at org.junit.jupiter.engine.execution.InterceptingExecutableInvoker$ReflectiveInterceptorCall.lambda$ofVoidMethod$0(InterceptingExecutableInvoker.java:103)
        at org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.lambda$invoke$0(InterceptingExecutableInvoker.java:93)
        at org.junit.jupiter.engine.execution.InvocationInterceptorChain$InterceptedInvocation.proceed(InvocationInterceptorChain.java:106)
        at org.junit.jupiter.engine.execution.InvocationInterceptorChain.proceed(InvocationInterceptorChain.java:64)
        at org.junit.jupiter.engine.execution.InvocationInterceptorChain.chainAndInvoke(InvocationInterceptorChain.java:45)
        at org.junit.jupiter.engine.execution.InvocationInterceptorChain.invoke(InvocationInterceptorChain.java:37)
        at org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.invoke(InterceptingExecutableInvoker.java:92)
        at org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.invoke(InterceptingExecutableInvoker.java:86)
        at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.lambda$invokeTestMethod$7(TestMethodTestDescriptor.java:217)
        at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
        at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.invokeTestMethod(TestMethodTestDescriptor.java:213)
        at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:138)
        at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:68)
        at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:151)
        at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
        at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:141)
        at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137)
        at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$9(NodeTestTask.java:139)
        at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
        at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:138)
        at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:95)
        at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
        at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:41)
        at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:155)
        at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
        at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:141)
        at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137)
        at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$9(NodeTestTask.java:139)
        at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
        at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:138)
        at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:95)
        at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
        at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:41)
        at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:155)
        at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
        at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:141)
        at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137)
        at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$9(NodeTestTask.java:139)
        at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
        at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:138)
        at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:95)
        at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.submit(SameThreadHierarchicalTestExecutorService.java:35)
        at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor.execute(HierarchicalTestExecutor.java:57)
        at org.junit.platform.engine.support.hierarchical.HierarchicalTestEngine.execute(HierarchicalTestEngine.java:54)
        at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:107)
        at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:88)
        at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:54)
        at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:67)
        at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:52)
        at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:114)
        at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:86)
        at org.junit.platform.launcher.core.DefaultLauncherSession$DelegatingLauncher.execute(DefaultLauncherSession.java:86)
        at org.junit.platform.launcher.core.SessionPerRequestLauncher.execute(SessionPerRequestLauncher.java:53)
        at org.gradle.api.internal.tasks.testing.junitplatform.JUnitPlatformTestClassProcessor$CollectAllTestClassesExecutor.processAllTestClasses(JUnitPlatformTestClassProcessor.java:99)
        at org.gradle.api.internal.tasks.testing.junitplatform.JUnitPlatformTestClassProcessor$CollectAllTestClassesExecutor.access$000(JUnitPlatformTestClassProcessor.java:79)
        at org.gradle.api.internal.tasks.testing.junitplatform.JUnitPlatformTestClassProcessor.stop(JUnitPlatformTestClassProcessor.java:75)
        at org.gradle.api.internal.tasks.testing.SuiteTestClassProcessor.stop(SuiteTestClassProcessor.java:62)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:568)
        at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:36)
        at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
        at org.gradle.internal.dispatch.ContextClassLoaderDispatch.dispatch(ContextClassLoaderDispatch.java:33)
        at org.gradle.internal.dispatch.ProxyDispatchAdapter$DispatchingInvocationHandler.invoke(ProxyDispatchAdapter.java:94)
        at jdk.proxy1/jdk.proxy1.$Proxy2.stop(Unknown Source)
        at org.gradle.api.internal.tasks.testing.worker.TestWorker$3.run(TestWorker.java:193)
        at org.gradle.api.internal.tasks.testing.worker.TestWorker.executeAndMaintainThreadName(TestWorker.java:129)
        at org.gradle.api.internal.tasks.testing.worker.TestWorker.execute(TestWorker.java:100)
        at org.gradle.api.internal.tasks.testing.worker.TestWorker.execute(TestWorker.java:60)
        at org.gradle.process.internal.worker.child.ActionExecutionWorker.execute(ActionExecutionWorker.java:56)
        at org.gradle.process.internal.worker.child.SystemApplicationClassLoaderWorker.call(SystemApplicationClassLoaderWorker.java:113)
        at org.gradle.process.internal.worker.child.SystemApplicationClassLoaderWorker.call(SystemApplicationClassLoaderWorker.java:65)
        at worker.org.gradle.process.internal.worker.GradleWorkerMain.run(GradleWorkerMain.java:69)
        at worker.org.gradle.process.internal.worker.GradleWorkerMain.main(GradleWorkerMain.java:74)

This log does not break anything, it is only about logging, but it can be avoided using the proposed patch.

A user has also reported this issue here: reactor/BlockHound#311
Can be also reproduced when building reactor-netty under the netty5 branch, when running this test:

./gradlew reactor-netty5-core:test --tests BufferFluxTest.testAsByteBuffer   -i

This problem is actually related to the following old issues:

reactor/BlockHound#71
raphw/byte-buddy#780

In a nutshell, Byte Buddy requires all type information for a transformed type to be available, else the above error log may be displayed. The log may happen occasionally with very rare byte code combinations
when the class is being defined (initial classloading). Please check #780 for the whole details.

So, what happens is that when the reactor-core ContextPropagation static initializer is instrumented, all bytes code from the static init is instrumented, including the part which is accessing to the context-propagation API, especially this problematic part which involves a lambda; so when the context-propagation is not there, byte buddy is reporting the missing io.micrometer.context.ContextRegistry type.

To avoid this problem, the proposed patch replaces the ContextPropagation static initializer with a basic Class.forName("io.micrometer.context.ContextRegistry") and all the init code which is manipulating the context propagation API is moved to the lazy Holder inner class which will be lazily invoked only if the context propagation is available.

If this PR is rejected, it is possible to enhance the BlockHound in order to not log the missing type. However, we can't ignore such missing types by default because in cases where the missing type is not optional, then that would be dangerous to just hide the log. So what can be done is to enhance the BlockHound builder API so user can specify a callback in order to ignore (or log) a missing type. And in this case, the Reactor Core BlockHound integration plugin will have to be modified in order to just ignore the missing type for context propagation using the new BlockHound missing type callback.

@pderop pderop requested a review from a team as a code owner January 29, 2023 22:25
@pderop pderop self-assigned this Jan 30, 2023
@pderop pderop removed the request for review from a team January 30, 2023 08:40
@pderop
Copy link
Member Author

pderop commented Jan 30, 2023

@reactor/core-team ,

can you please take a look ? thanks.

What is really needed is to avoid using a lambda when initializing the contextCaptureFunction from the static initializer.
@pderop
Copy link
Member Author

pderop commented Jan 30, 2023

@reactor/core-team ,

I have simplified the impact of the PR, and the key point is to avoid using a lambda in this part of the static initializer.

please take a look.

@pderop
Copy link
Member Author

pderop commented Feb 3, 2023

@chemicL , thank you for your review !
I am going to squash & merge this.

@pderop pderop added this to the 3.5.3 milestone Feb 3, 2023
@pderop
Copy link
Member Author

pderop commented Feb 3, 2023

I wonder which label should be used for this issue, because there is no bug fixed per se, it's a work around to avoid the logging issue from raphw/byte-buddy#780

Should I set the label to area/reactor-tools ? or for/other-project, or type/enhancement ?

@chemicL chemicL added the type/enhancement A general enhancement label Feb 3, 2023
@chemicL
Copy link
Member

chemicL commented Feb 3, 2023

I picked type/enhancement. for/other-project would mean the issue is irrelevant, while it actually is. Also, it's not for reactor-tools but is a change in core, so in my opinion we can classify it as a general enhancement that improves the UX.

@pderop pderop merged commit ecb6cd2 into reactor:main Feb 3, 2023
@pderop
Copy link
Member Author

pderop commented Feb 15, 2023

After the recent necessary changes made in the ContextPropagation (latest commit done in 3.5.3 version), the exception is resurfacing.
I will work on this and will hopefully see if something can be done in the ContextPropagation class, so we can avoid the exception.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type/enhancement A general enhancement
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants