saxon test cannot be compiled to native with Camel 4 and Quarkus 3

[ppalaga]

[INFO] [stdout] ========================================================================================================================
[INFO] [stdout] GraalVM Native Image: Generating 'camel-quarkus-integration-test-saxon-3.0.0-SNAPSHOT-runner' (executable)...
[INFO] [stdout] ========================================================================================================================
[INFO] [stdout] [1/7] Initializing...                                                                                    (9.6s @ 0.31GB)
[INFO] [stdout]  Version info: 'GraalVM 22.3.1.0-Final Java 17 Mandrel Distribution'
[INFO] [stdout]  Java version info: '17.0.6+10'
[INFO] [stdout]  C compiler: gcc (redhat, x86_64, 8.5.0)
[INFO] [stdout]  Garbage collector: Serial GC
[INFO] [stdout]  3 user-specific feature(s)
[INFO] [stdout]  - io.quarkus.runner.Feature: Auto-generated class by Quarkus from the existing extensions
[INFO] [stdout]  - io.quarkus.runtime.graal.DisableLoggingFeature: Disables INFO logging during the analysis phase for the [org.jboss.threads] categories
[INFO] [stdout]  - io.quarkus.runtime.graal.ResourcesFeature: Register each line in META-INF/quarkus-native-resources.txt as a resource on Substrate VM
[INFO] [stdout] [2/7] Performing analysis...  []                                                                        (10.7s @ 2.48GB)
[INFO] [stdout]   10,391 (89.60%) of 11,597 classes reachable
[INFO] [stdout]   14,194 (61.09%) of 23,236 fields reachable
[INFO] [stdout]   45,968 (82.24%) of 55,896 methods reachable
[INFO] [stdout]      851 classes,     0 fields, and     0 methods registered for reflection
[INFO] [stdout] 
[WARN] [stderr]         at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.util.AnalysisError.parsingError(AnalysisError.java:153)
[WARN] [stderr]         at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.flow.MethodTypeFlow.createFlowsGraph(MethodTypeFlow.java:104)
[WARN] [stderr]         at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.flow.MethodTypeFlow.ensureFlowsGraphCreated(MethodTypeFlow.java:83)
[WARN] [stderr]         at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.flow.MethodTypeFlow.getOrCreateMethodFlowsGraph(MethodTypeFlow.java:65)
[WARN] [stderr]         at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.typestate.DefaultVirtualInvokeTypeFlow.onObservedUpdate(DefaultVirtualInvokeTypeFlow.java:109)
[WARN] [stderr]         at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.flow.TypeFlow.update(TypeFlow.java:562)
[WARN] [stderr]         at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.PointsToAnalysis$1.run(PointsToAnalysis.java:488)
[WARN] [stderr]         at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.util.CompletionExecutor.executeCommand(CompletionExecutor.java:193)
[WARN] [stderr]         at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.util.CompletionExecutor.lambda$executeService$0(CompletionExecutor.java:177)
[WARN] [stderr]         at java.base/java.util.concurrent.ForkJoinTask$RunnableExecuteAction.exec(ForkJoinTask.java:1395)
[WARN] [stderr]         at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373)
[WARN] [stderr]         at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182)
[WARN] [stderr]         at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655)
[WARN] [stderr]         at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622)
[WARN] [stderr]         at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165)
[WARN] [stderr] Caused by: org.graalvm.compiler.java.BytecodeParser$BytecodeParserError: com.oracle.graal.pointsto.constraints.UnresolvedElementException: Discovered unresolved method during parsing: org.apache.hc.client5.http.entity.BrotliInputStreamFactory.getInstance(). This error is reported at image build time because class org.apache.hc.client5.http.impl.classic.ContentCompressionExec is registered for linking at image build time by command line
[WARN] [stderr]         at parsing org.apache.hc.client5.http.impl.classic.ContentCompressionExec.<init>(ContentCompressionExec.java:109)
[WARN] [stderr]         at jdk.internal.vm.compiler/org.graalvm.compiler.java.BytecodeParser.throwParserError(BytecodeParser.java:2518)
[WARN] [stderr]         at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.phases.SharedGraphBuilderPhase$SharedBytecodeParser.throwParserError(SharedGraphBuilderPhase.java:110)
[WARN] [stderr]         at jdk.internal.vm.compiler/org.graalvm.compiler.java.BytecodeParser.iterateBytecodesForBlock(BytecodeParser.java:3393)
[WARN] [stderr]         at jdk.internal.vm.compiler/org.graalvm.compiler.java.BytecodeParser.handleBytecodeBlock(BytecodeParser.java:3345)
[WARN] [stderr]         at jdk.internal.vm.compiler/org.graalvm.compiler.java.BytecodeParser.processBlock(BytecodeParser.java:3190)
[WARN] [stderr]         at jdk.internal.vm.compiler/org.graalvm.compiler.java.BytecodeParser.build(BytecodeParser.java:1138)
[WARN] [stderr]         at jdk.internal.vm.compiler/org.graalvm.compiler.java.BytecodeParser.buildRootMethod(BytecodeParser.java:1030)
[WARN] [stderr] Fatal error: com.oracle.graal.pointsto.util.AnalysisError$ParsingError: Error encountered while parsing org.apache.hc.client5.http.impl.classic.HttpClientBuilder.build() 
[WARN] [stderr]         at jdk.internal.vm.compiler/org.graalvm.compiler.java.GraphBuilderPhase$Instance.run(GraphBuilderPhase.java:97)
[WARN] [stderr] Parsing context:
[WARN] [stderr]    at org.apache.hc.client5.http.impl.classic.HttpClientBuilder.build(HttpClientBuilder.java:731)
[WARN] [stderr]    at org.apache.hc.client5.http.impl.classic.HttpClients.createDefault(HttpClients.java:57)
[WARN] [stderr]    at org.xmlresolver.ResourceConnection.<init>(ResourceConnection.java:52)
[WARN] [stderr]    at org.xmlresolver.ResourceConnection.<init>(ResourceConnection.java:42)
[WARN] [stderr] 
[WARN] [stderr]         at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.phases.SharedGraphBuilderPhase.run(SharedGraphBuilderPhase.java:84)
[WARN] [stderr]         at jdk.internal.vm.compiler/org.graalvm.compiler.phases.Phase.run(Phase.java:49)
[WARN] [stderr]         at jdk.internal.vm.compiler/org.graalvm.compiler.phases.BasePhase.apply(BasePhase.java:446)
[WARN] [stderr]         at jdk.internal.vm.compiler/org.graalvm.compiler.phases.Phase.apply(Phase.java:42)
[WARN] [stderr]         at jdk.internal.vm.compiler/org.graalvm.compiler.phases.Phase.apply(Phase.java:38)
[WARN] [stderr]         at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.flow.AnalysisParsedGraph.parseBytecode(AnalysisParsedGraph.java:135)
[WARN] [stderr]         at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.meta.AnalysisMethod.ensureGraphParsed(AnalysisMethod.java:685)
[WARN] [stderr]         at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.phases.InlineBeforeAnalysisGraphDecoder.lookupEncodedGraph(InlineBeforeAnalysis.java:180)
[WARN] [stderr]         at jdk.internal.vm.compiler/org.graalvm.compiler.replacements.PEGraphDecoder.doInline(PEGraphDecoder.java:1162)
[WARN] [stderr]         at jdk.internal.vm.compiler/org.graalvm.compiler.replacements.PEGraphDecoder.tryInline(PEGraphDecoder.java:1145)
[WARN] [stderr]         at jdk.internal.vm.compiler/org.graalvm.compiler.replacements.PEGraphDecoder.trySimplifyInvoke(PEGraphDecoder.java:1003)
[WARN] [stderr]         at jdk.internal.vm.compiler/org.graalvm.compiler.replacements.PEGraphDecoder.handleInvoke(PEGraphDecoder.java:957)
[WARN] [stderr]         at jdk.internal.vm.compiler/org.graalvm.compiler.nodes.GraphDecoder.processNextNode(GraphDecoder.java:817)
[WARN] [stderr]         at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.phases.InlineBeforeAnalysisGraphDecoder.processNextNode(InlineBeforeAnalysis.java:240)
[WARN] [stderr]         at jdk.internal.vm.compiler/org.graalvm.compiler.nodes.GraphDecoder.decode(GraphDecoder.java:548)
[WARN] [stderr]         at jdk.internal.vm.compiler/org.graalvm.compiler.replacements.PEGraphDecoder.decode(PEGraphDecoder.java:833)
[WARN] [stderr]         at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.phases.InlineBeforeAnalysis.decodeGraph(InlineBeforeAnalysis.java:98)
[WARN] [stderr] Caused by: com.oracle.graal.pointsto.constraints.UnresolvedElementException: Discovered unresolved method during parsing: org.apache.hc.client5.http.entity.BrotliInputStreamFactory.getInstance(). This error is reported at image build time because class org.apache.hc.client5.http.impl.classic.ContentCompressionExec is registered for linking at image build time by command line
[WARN] [stderr]         at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.phases.SharedGraphBuilderPhase$SharedBytecodeParser.reportUnresolvedElement(SharedGraphBuilderPhase.java:333)
[WARN] [stderr]         at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.phases.SharedGraphBuilderPhase$SharedBytecodeParser.handleUnresolvedMethod(SharedGraphBuilderPhase.java:323)
[WARN] [stderr]         at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.phases.SharedGraphBuilderPhase$SharedBytecodeParser.handleUnresolvedInvoke(SharedGraphBuilderPhase.java:279)
[WARN] [stderr]         at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.flow.MethodTypeFlowBuilder.parse(MethodTypeFlowBuilder.java:179)
[WARN] [stderr]         at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.flow.MethodTypeFlowBuilder.apply(MethodTypeFlowBuilder.java:349)
[WARN] [stderr]         at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.flow.MethodTypeFlow.createFlowsGraph(MethodTypeFlow.java:93)
[WARN] [stderr]         ... 13 more
[WARN] [stderr]         at jdk.internal.vm.compiler/org.graalvm.compiler.java.BytecodeParser.genInvokeStatic(BytecodeParser.java:1676)
[WARN] [stderr]         at jdk.internal.vm.compiler/org.graalvm.compiler.java.BytecodeParser.genInvokeStatic(BytecodeParser.java:1651)
[WARN] [stderr]         at jdk.internal.vm.compiler/org.graalvm.compiler.java.BytecodeParser.processBytecode(BytecodeParser.java:5288)
[WARN] [stderr]         at jdk.internal.vm.compiler/org.graalvm.compiler.java.BytecodeParser.iterateBytecodesForBlock(BytecodeParser.java:3385)
[WARN] [stderr]         ... 38 more

[zhfeng]

It is due to org.brotli:dec is an optional dependency.

[zhfeng]

I think the brotli decompress is introduced by apache/httpcomponents-client@db47570

I tried to add a subsitution on org.apache.hc.client5.http.entity.BrotliInputStreamFactory but the native building throws

Error: Cannot find org.apache.hc.client5.http.entity.BrotliInputStreamFactory.create, org.apache.hc.client5.http.entity.BrotliInputStreamFactory can not be loaded, due to org/brotli/dec/BrotliInputStream not being available in the classpath. Are you missing a dependency in your classpath?
com.oracle.svm.core.util.UserError$UserException: Cannot find org.apache.hc.client5.http.entity.BrotliInputStreamFactory.create, org.apache.hc.client5.http.entity.BrotliInputStreamFactory can not be loaded, due to org/brotli/dec/BrotliInputStream not being available in the classpath. Are you missing a dependency in your classpath?

The only way is to add NativeImageAllowIncompleteClasspathBuildItem

@BuildStep
NativeImageAllowIncompleteClasspathBuildItem brotliDecIsOptional() {
    return new NativeImageAllowIncompleteClasspathBuildItem("camel-quarkus-saxon");
}

@ppalaga @zakkak any idea?

[ppalaga]

Can't we add org.brotli:dec as a non-optional depedency to camel-quarkus-saxon?

[zhfeng]

Yeah, we can and it should work. Let me prepare a PR.

[zakkak]

I tried to add a subsitution on org.apache.hc.client5.http.entity.BrotliInputStreamFactory but the native building throws

Yes, unfortunately in order to substitute a class the class needs to be loaded in its original form. So you need a way to avoid loading the class in the first place instead.

One would expect that GraalVM would be able to (which it does) see that BrotliInputStreamFactory is never reached in this case and not load the class. Unfortunately though due to oracle/graal#4652 this is not happening.

Your two options in this case seem to be:

1. Substitute the constructor of org.apache.hc.client5.http.impl.classic.ContentCompressionExec to remove the reference to BrotliInputStreamFactory.
2. Load brotli as a non-optional dependency as you already did.

[zhfeng]

Thanks @zakkak and it can fix most of such optional dependency issues if oracle/graal#4652 get resolved in the future?

[zakkak]

Only if the optional access is guarded by an if statement whose condition value can be calculated by the static analysis at build time.