Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unable to run a native image on GCP (Cloud Run) #3042

Open
gbhojak opened this issue Apr 10, 2020 · 6 comments
Open

Unable to run a native image on GCP (Cloud Run) #3042

gbhojak opened this issue Apr 10, 2020 · 6 comments
Labels
info: workaround available A workaround is available for the issue relates-to: graal

Comments

@gbhojak
Copy link

gbhojak commented Apr 10, 2020

Hi everyone,

I am trying to compile a simple hello-world application to a native image on GCP.
I have added firebase-admin sdk provided by GCP to access their cloud firestore database.

Would really appreciate any help here. We are trying to use micronaut to create at least 4-5 microservices to run on Cloud Run, hence would like to leverage the fast startup times and reduced memory footprint.

Steps to Reproduce

  1. mn create-app hello-world --features graal-native-image

  2. Added the dependency for firebase-admin sdk to pom.xml

    <dependency>
      <groupId>com.google.firebase</groupId>
      <artifactId>firebase-admin</artifactId>
      <version>6.12.2</version>
    </dependency>
  1. Submitted a build to GCP using docker file created by 'mn' command

I tried to follow the recommendations mentioned here and here

Here are the contents of native-image.properties:

Args = -H:IncludeResources=logback.xml|application.yml|bootstrap.yml|reflect-config.json \
       -H:Name=hello-world \
       -H:Class=com.company.samplepackage.Application \
       -H:+ReportExceptionStackTraces \
       --report-unsupported-elements-at-runtime \
       -H:ReflectionConfigurationFiles=target/classes/reflect-config.json \
       --initialize-at-run-time=io.netty.handler.codec.http2.Http2ClientUpgradeCodec \
       --initialize-at-run-time=com.google.protobuf.ExtensionRegistryFactory

and reflect-config.json


[
  {
    "name": "io.grpc.netty.shaded.io.netty.channel.socket.nio.NioSocketChannel",
    "methods": [
      {
        "name": "<init>",
        "parameterTypes": []
      }
    ]
  }
]

Expected Behaviour

Should be able to invoke the 'hello world' api

Actual Behaviour


A 2020-04-09T23:44:28.888369Z 	... 41 common frames omitted 
A 2020-04-09T23:44:28.888362Z 	at io.micronaut.context.DefaultBeanContext.doCreateBean(DefaultBeanContext.java:1693) 
A 2020-04-09T23:44:28.888352Z 	at com.company.samplepackage.$FirebaseBeanDefinition.build(Unknown Source) 
A 2020-04-09T23:44:28.888330Z 	at com.company.samplepackage.FirebaseBean.<init>(FirebaseBean.java:23) 
A 2020-04-09T23:44:28.888323Z 	at com.google.cloud.ServiceOptions.getService(ServiceOptions.java:518) 
A 2020-04-09T23:44:28.888316Z 	at com.google.cloud.firestore.FirestoreOptions$DefaultFirestoreFactory.create(FirestoreOptions.java:66) 
A 2020-04-09T23:44:28.888305Z 	at com.google.cloud.firestore.FirestoreOptions$DefaultFirestoreFactory.create(FirestoreOptions.java:73) 
A 2020-04-09T23:44:28.888273Z 	at com.google.cloud.firestore.FirestoreImpl.<init>(FirestoreImpl.java:77) 
A 2020-04-09T23:44:28.888266Z 	at com.google.cloud.firestore.FirestoreOptions.getFirestoreRpc(FirestoreOptions.java:385) 
A 2020-04-09T23:44:28.888259Z 	at com.google.cloud.ServiceOptions.getRpc(ServiceOptions.java:538) 
A 2020-04-09T23:44:28.888247Z 	at com.google.cloud.firestore.FirestoreOptions$DefaultFirestoreRpcFactory.create(FirestoreOptions.java:82) 
A 2020-04-09T23:44:28.888239Z 	at com.google.cloud.firestore.FirestoreOptions$DefaultFirestoreRpcFactory.create(FirestoreOptions.java:90) 
A 2020-04-09T23:44:28.888227Z 	at com.google.cloud.firestore.spi.v1.GrpcFirestoreRpc.<init>(GrpcFirestoreRpc.java:122) 
A 2020-04-09T23:44:28.888219Z 	at com.google.api.gax.rpc.ClientContext.create(ClientContext.java:122) 
A 2020-04-09T23:44:28.888203Z 	at com.google.api.gax.rpc.ClientContext.create(ClientContext.java:155) 
A 2020-04-09T23:44:28.888166Z 	at com.google.api.gax.grpc.InstantiatingGrpcChannelProvider.getTransportChannel(InstantiatingGrpcChannelProvider.java:186) 
A 2020-04-09T23:44:28.888155Z 	at com.google.api.gax.grpc.InstantiatingGrpcChannelProvider.createChannel(InstantiatingGrpcChannelProvider.java:194) 
A 2020-04-09T23:44:28.888144Z 	at com.google.api.gax.grpc.InstantiatingGrpcChannelProvider.createSingleChannel(InstantiatingGrpcChannelProvider.java:296) 
A 2020-04-09T23:44:28.888132Z 	at io.grpc.internal.AbstractManagedChannelImplBuilder.build(AbstractManagedChannelImplBuilder.java:507) 
A 2020-04-09T23:44:28.888121Z 	at io.grpc.netty.shaded.io.grpc.netty.NettyChannelBuilder.buildTransportFactory(NettyChannelBuilder.java:406) 
A 2020-04-09T23:44:28.888109Z 	at io.grpc.netty.shaded.io.grpc.netty.GrpcSslContexts.forClient(GrpcSslContexts.java:120) 
A 2020-04-09T23:44:28.888098Z 	at io.grpc.netty.shaded.io.grpc.netty.GrpcSslContexts.configure(GrpcSslContexts.java:171) 
A 2020-04-09T23:44:28.888089Z 	at io.grpc.netty.shaded.io.grpc.netty.GrpcSslContexts.defaultSslProvider(GrpcSslContexts.java:244) 
A 2020-04-09T23:44:28.888075Z 	at java.lang.Class.ensureInitialized(DynamicHub.java:496) 
A 2020-04-09T23:44:28.888068Z 	at com.oracle.svm.core.hub.ClassInitializationInfo.initialize(ClassInitializationInfo.java:270) 
A 2020-04-09T23:44:28.888061Z 	at com.oracle.svm.core.hub.ClassInitializationInfo.invokeClassInitializer(ClassInitializationInfo.java:350) 
A 2020-04-09T23:44:28.888054Z 	at io.grpc.netty.shaded.io.netty.handler.ssl.OpenSsl.<clinit>(OpenSsl.java:121) 
A 2020-04-09T23:44:28.888047Z 	at java.lang.Class.ensureInitialized(DynamicHub.java:496) 
A 2020-04-09T23:44:28.888040Z 	at com.oracle.svm.core.hub.ClassInitializationInfo.initialize(ClassInitializationInfo.java:270) 
A 2020-04-09T23:44:28.888031Z 	at com.oracle.svm.core.hub.ClassInitializationInfo.invokeClassInitializer(ClassInitializationInfo.java:350) 
A 2020-04-09T23:44:28.888023Z 	at io.grpc.netty.shaded.io.netty.internal.tcnative.SSL.<clinit>(SSL.java:67) 
A 2020-04-09T23:44:28.888014Z 	at io.grpc.netty.shaded.io.netty.internal.tcnative.NativeStaticallyReferencedJniMethods.sslOpCipherServerPreference(NativeStaticallyReferencedJniMethods.java) 
A 2020-04-09T23:44:28.888Z 	at com.oracle.svm.jni.JNIGeneratedMethodSupport.nativeCallAddress(JNIGeneratedMethodSupport.java:57) 
A 2020-04-09T23:44:28.887990Z 	at com.oracle.svm.jni.access.JNINativeLinkage.getOrFindEntryPoint(JNINativeLinkage.java:145) 
A 2020-04-09T23:44:28.887982Z Caused by: java.lang.UnsatisfiedLinkError: io.grpc.netty.shaded.io.netty.internal.tcnative.NativeStaticallyReferencedJniMethods.sslOpCipherServerPreference()I [symbol: Java_io_grpc_netty_shaded_io_netty_internal_tcnative_NativeStaticallyReferencedJniMethods_sslOpCipherServerPreference or Java_io_grpc_netty_shaded_io_netty_internal_tcnative_NativeStaticallyReferencedJniMethods_sslOpCipherServerPreference__] 
A 2020-04-09T23:44:28.887964Z 	at com.oracle.svm.core.posix.thread.PosixJavaThreads.pthreadStartRoutine(PosixJavaThreads.java:193) 
A 2020-04-09T23:44:28.887931Z 	at com.oracle.svm.core.thread.JavaThreads.threadStartRoutine(JavaThreads.java:527) 
A 2020-04-09T23:44:28.887923Z 	at java.lang.Thread.run(Thread.java:748) 
A 2020-04-09T23:44:28.887915Z 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) 
A 2020-04-09T23:44:28.887908Z 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) 
A 2020-04-09T23:44:28.887900Z 	at io.reactivex.internal.schedulers.ExecutorScheduler$ExecutorWorker.run(ExecutorScheduler.java:253) 
A 2020-04-09T23:44:28.887892Z 	at io.reactivex.internal.schedulers.ExecutorScheduler$ExecutorWorker$BooleanRunnable.run(ExecutorScheduler.java:288) 
A 2020-04-09T23:44:28.887884Z 	at io.reactivex.internal.operators.flowable.FlowableSubscribeOn$SubscribeOnSubscriber.run(FlowableSubscribeOn.java:82) 
A 2020-04-09T23:44:28.887876Z 	at io.reactivex.Flowable.subscribe(Flowable.java:14865) 
A 2020-04-09T23:44:28.887869Z 	at io.reactivex.Flowable.subscribe(Flowable.java:14918) 
A 2020-04-09T23:44:28.887861Z 	at io.reactivex.internal.operators.flowable.FlowableFromPublisher.subscribeActual(FlowableFromPublisher.java:29) 
A 2020-04-09T23:44:28.887853Z 	at io.micronaut.http.context.ServerRequestTracingPublisher.subscribe(ServerRequestTracingPublisher.java:52) 
A 2020-04-09T23:44:28.887845Z 	at io.micronaut.http.context.ServerRequestContext.with(ServerRequestContext.java:68) 
A 2020-04-09T23:44:28.887837Z 	at io.micronaut.http.context.ServerRequestTracingPublisher.lambda$subscribe$0(ServerRequestTracingPublisher.java:52) 
A 2020-04-09T23:44:28.887828Z 	at io.reactivex.Flowable.subscribe(Flowable.java:14868) 
A 2020-04-09T23:44:28.887821Z 	at io.reactivex.Flowable.subscribe(Flowable.java:14918) 
A 2020-04-09T23:44:28.887814Z 	at io.reactivex.internal.operators.flowable.FlowableSwitchIfEmpty.subscribeActual(FlowableSwitchIfEmpty.java:32) 
A 2020-04-09T23:44:28.887805Z 	at io.reactivex.Flowable.subscribe(Flowable.java:14918) 
A 2020-04-09T23:44:28.887796Z 	at io.reactivex.internal.operators.flowable.FlowableMap.subscribeActual(FlowableMap.java:37) 
A 2020-04-09T23:44:28.887785Z 	at io.reactivex.Flowable.subscribe(Flowable.java:14918) 
A 2020-04-09T23:44:28.887777Z 	at io.reactivex.internal.operators.flowable.FlowableCreate.subscribeActual(FlowableCreate.java:71) 
A 2020-04-09T23:44:28.887692Z 	at io.micronaut.http.server.netty.RoutingInBoundHandler.lambda$buildResultEmitter$16(RoutingInBoundHandler.java:1474) 
A 2020-04-09T23:44:28.887683Z 	at io.micronaut.web.router.RouteMatch.execute(RouteMatch.java:122) 
A 2020-04-09T23:44:28.887674Z 	at io.micronaut.web.router.AbstractRouteMatch.execute(AbstractRouteMatch.java:228) 
A 2020-04-09T23:44:28.887667Z 	at io.micronaut.context.DefaultBeanContext$BeanExecutionHandle.invoke(DefaultBeanContext.java:2993) 
A 2020-04-09T23:44:28.887659Z 	at io.micronaut.context.DefaultBeanContext$BeanExecutionHandle.getTarget(DefaultBeanContext.java:2972) 
A 2020-04-09T23:44:28.887651Z 	at io.micronaut.context.DefaultBeanContext.getBean(DefaultBeanContext.java:618) 
A 2020-04-09T23:44:28.887640Z 	at io.micronaut.context.DefaultBeanContext.getBeanInternal(DefaultBeanContext.java:2058) 
A 2020-04-09T23:44:28.887594Z 	at io.micronaut.context.DefaultBeanContext.getBeanForDefinition(DefaultBeanContext.java:2084) 
A 2020-04-09T23:44:28.887584Z 	at io.micronaut.context.DefaultBeanContext.createAndRegisterSingleton(DefaultBeanContext.java:2393) 
A 2020-04-09T23:44:28.887575Z 	at io.micronaut.context.DefaultBeanContext.createAndRegisterSingletonInternal(DefaultBeanContext.java:2407) 
A 2020-04-09T23:44:28.887567Z 	at io.micronaut.context.DefaultBeanContext.doCreateBean(DefaultBeanContext.java:1693) 
A 2020-04-09T23:44:28.887560Z 	at com.company.samplepackage.$HelloControllerDefinition.build(Unknown Source) 
A 2020-04-09T23:44:28.887552Z 	at com.company.samplepackage.$HelloControllerDefinition.injectBean(Unknown Source) 
A 2020-04-09T23:44:28.887543Z 	at io.micronaut.context.AbstractBeanDefinition.getBeanForField(AbstractBeanDefinition.java:1184) 
A 2020-04-09T23:44:28.887535Z 	at io.micronaut.context.AbstractBeanDefinition.getBeanForField(AbstractBeanDefinition.java:1411) 
A 2020-04-09T23:44:28.887526Z 	at io.micronaut.context.DefaultBeanContext.getBean(DefaultBeanContext.java:1095) 
A 2020-04-09T23:44:28.887501Z 	at io.micronaut.context.DefaultBeanContext.getBeanInternal(DefaultBeanContext.java:2058) 
A 2020-04-09T23:44:28.887192Z 	at io.micronaut.context.DefaultBeanContext.getBeanForDefinition(DefaultBeanContext.java:2084) 
A 2020-04-09T23:44:28.887180Z 	at io.micronaut.context.DefaultBeanContext.createAndRegisterSingleton(DefaultBeanContext.java:2393) 
A 2020-04-09T23:44:28.887167Z 	at io.micronaut.context.DefaultBeanContext.createAndRegisterSingletonInternal(DefaultBeanContext.java:2407) 
A 2020-04-09T23:44:28.887113Z 	at io.micronaut.context.DefaultBeanContext.doCreateBean(DefaultBeanContext.java:1719) 
A 2020-04-09T23:44:28.887104Z Path Taken: HelloController.fb 
A 2020-04-09T23:44:28.887096Z Message: io.grpc.netty.shaded.io.netty.internal.tcnative.NativeStaticallyReferencedJniMethods.sslOpCipherServerPreference()I [symbol: Java_io_grpc_netty_shaded_io_netty_internal_tcnative_NativeStaticallyReferencedJniMethods_sslOpCipherServerPreference or Java_io_grpc_netty_shaded_io_netty_internal_tcnative_NativeStaticallyReferencedJniMethods_sslOpCipherServerPreference__] 
A 2020-04-09T23:44:28.887078Z 
A 2020-04-09T23:44:28.887071Z io.micronaut.context.exceptions.BeanInstantiationException: Error instantiating bean of type  [com.company.mypackage.HelloController] 
A 2020-04-09T23:44:28.887054Z Path Taken: HelloController.fb 
A 2020-04-09T23:44:28.887044Z Message: io.grpc.netty.shaded.io.netty.internal.tcnative.NativeStaticallyReferencedJniMethods.sslOpCipherServerPreference()I [symbol: Java_io_grpc_netty_shaded_io_netty_internal_tcnative_NativeStaticallyReferencedJniMethods_sslOpCipherServerPreference or Java_io_grpc_netty_shaded_io_netty_internal_tcnative_NativeStaticallyReferencedJniMethods_sslOpCipherServerPreference__] 
A 2020-04-09T23:44:28.887027Z 
A 2020-04-09T23:44:28.886998Z �[36m23:44:28.886�[0;39m �[1;30m[pool-2-thread-2]�[0;39m �[1;31mERROR�[0;39m �[35mi.m.h.s.netty.RoutingInBoundHandler�[0;39m - Unexpected error occurred: Error instantiating bean of type  [com.company.mypackage.HelloController] 
A 2020-04-09T23:44:28.776407Z �[36m23:44:28.775�[0;39m �[1;30m[pool-2-thread-2]�[0;39m �[34mINFO �[0;39m �[35mi.g.n.s.i.n.u.i.PlatformDependent�[0;39m - Your platform does not provide complete low-level API for accessing direct buffers reliably. Unless explicitly requested, heap buffer will always be preferred to avoid potential system instability. 
A 2020-04-09T23:44:28.738912Z Initialized the Firebase App. [DEFAULT] 


Environment Information

  • Operating System: GCP Cloud Run
  • Micronaut Version: 1.3.4
  • JDK Version: java8
  • graalvm version: graalvm-ce:20.0.0-java8
@renannprado
Copy link
Contributor

@gbhojak to narrow down a little bit: looking at your stacktrace it seems this error is related to gRPC (which is what the library uses underneath).

@gbhojak
Copy link
Author

gbhojak commented Apr 13, 2020

@gbhojak to narrow down a little bit: looking at your stacktrace it seems this error is related to gRPC (which is what the library uses underneath).

Hi @renannprado , thank you for responding back. It is related to use of gRPC. Looking at 2516, it does look like micronaut app should compile in native mode while using Cloud Firestore sdk. I am not sure if I am missing anything. Would appreciate any help with it. Thank you.

@dzou
Copy link

dzou commented Jun 5, 2020

@gbhojak - Hey did you ever figure this out? I am encountering the same error.

@gbhojak
Copy link
Author

gbhojak commented Jun 6, 2020

Hi @dzou , I ended up not using micronaut coz of this issue. I developed the microservice in Go language instead.

@emilvberglind
Copy link

emilvberglind commented Oct 19, 2020

I ran into the same issue, and it is related to the included version of grpc-netty-shaded. What I wound up doing, at least for now, was to exclude that transitive artifact and instead use grpc-okhttp. So, my pom.xml looks like this:

    <dependency>
      <groupId>com.google.firebase</groupId>
      <artifactId>firebase-admin</artifactId>
      <version>7.0.0</version>
            <exclusions>
              <exclusion>
                <groupId>io.grpc</groupId>
                <artifactId>grpc-netty-shaded</artifactId>
              </exclusion>
            </exclusions>
    </dependency>

    <!--Using OkHttp over netty because the shaded netty included in grpc does not work with native image-->
    <dependency>
      <groupId>io.grpc</groupId>
      <artifactId>grpc-okhttp</artifactId>
      <version>1.30.2</version>
    </dependency>

@graemerocher graemerocher added the info: workaround available A workaround is available for the issue label Oct 19, 2020
@TMSantos
Copy link

TMSantos commented Sep 14, 2023

@emilvberglind does it work for you with exclusion? It compiles, but FCM always returns 400 at cloud Run, running in JVM works fine.

I'm having same error as here GoogleCloudPlatform/native-image-support-java#116

 <!-- FCM -->
    <dependency>
      <groupId>com.google.firebase</groupId>
      <artifactId>firebase-admin</artifactId>
      <version>9.2.0</version>
      <exclusions>
        <exclusion>
          <groupId>io.grpc</groupId>
          <artifactId>grpc-netty-shaded</artifactId>
        </exclusion>
      </exclusions>
    </dependency>

    <!--Using OkHttp over netty because the shaded netty included in grpc does not work with native image-->
    <dependency>
      <groupId>io.grpc</groupId>
      <artifactId>grpc-okhttp</artifactId>
      <version>1.51.1</version>
    </dependency>

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
info: workaround available A workaround is available for the issue relates-to: graal
Projects
None yet
Development

No branches or pull requests

7 participants