Exception java.lang.IllegalStateException: Could not find policy 'pick_first' #5493

pereira-a · 2019-03-24T19:20:35Z

Please answer these questions before submitting your issue.

What version of gRPC are you using?

1.18.0

When executing the Quickstart class of Google Firestore Service I get this exception.
https://cloud.google.com/firestore/docs/quickstart-servers

Mar 24, 2019 7:16:18 PM io.grpc.internal.ManagedChannelImpl$1 uncaughtException
SEVERE: [Channel<1>: (firestore.googleapis.com:443)] Uncaught exception in the SynchronizationContext. Panic!
java.lang.IllegalStateException: Could not find policy 'pick_first'. Make sure its implementation is either registered to LoadBalancerRegistry or included in META-INF/services/io.grpc.LoadBalancerProvider from your jar files.
        at io.grpc.internal.AutoConfiguredLoadBalancerFactory$AutoConfiguredLoadBalancer.<init>(AutoConfiguredLoadBalancerFactory.java:93)
        at io.grpc.internal.AutoConfiguredLoadBalancerFactory.newLoadBalancer(AutoConfiguredLoadBalancerFactory.java:64)
        at io.grpc.internal.ManagedChannelImpl.exitIdleMode(ManagedChannelImpl.java:357)
        at io.grpc.internal.ManagedChannelImpl$ChannelTransportProvider$1ExitIdleModeForTransport.run(ManagedChannelImpl.java:455)
        at io.grpc.SynchronizationContext.drain(SynchronizationContext.java:101)
        at io.grpc.SynchronizationContext.execute(SynchronizationContext.java:130)
        at io.grpc.internal.ManagedChannelImpl$ChannelTransportProvider.get(ManagedChannelImpl.java:459)
        at io.grpc.internal.ClientCallImpl.start(ClientCallImpl.java:241)
        at io.grpc.internal.CensusTracingModule$TracingClientInterceptor$1.start(CensusTracingModule.java:392)
        at io.grpc.internal.CensusStatsModule$StatsClientInterceptor$1.start(CensusStatsModule.java:694)
        at io.grpc.ForwardingClientCall.start(ForwardingClientCall.java:32)
        at com.google.api.gax.grpc.GrpcHeaderInterceptor$1.start(GrpcHeaderInterceptor.java:95)
        at io.grpc.stub.ClientCalls.startCall(ClientCalls.java:308)
        at io.grpc.stub.ClientCalls.asyncUnaryRequestCall(ClientCalls.java:280)
        at io.grpc.stub.ClientCalls.futureUnaryCall(ClientCalls.java:189)
        at com.google.api.gax.grpc.GrpcDirectCallable.futureCall(GrpcDirectCallable.java:58)
        at com.google.api.gax.grpc.GrpcExceptionCallable.futureCall(GrpcExceptionCallable.java:64)
        at com.google.api.gax.tracing.TracedUnaryCallable.futureCall(TracedUnaryCallable.java:75)
        at com.google.api.gax.rpc.UnaryCallable$1.futureCall(UnaryCallable.java:126)
        at com.google.api.gax.rpc.UnaryCallable.futureCall(UnaryCallable.java:87)
        at com.google.cloud.firestore.FirestoreImpl.sendRequest(FirestoreImpl.java:413)
        at com.google.cloud.firestore.UpdateBuilder.commit(UpdateBuilder.java:621)
        at com.google.cloud.firestore.WriteBatch.commit(WriteBatch.java:41)
        at com.google.cloud.firestore.DocumentReference.set(DocumentReference.java:172)
        at Main.main(Main.java:26)

java.util.concurrent.ExecutionException: com.google.api.gax.rpc.InternalException: io.grpc.StatusRuntimeException: INTERNAL: Panic! This is a bug!
        at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:531)
        at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:492)
        at com.google.common.util.concurrent.AbstractFuture$TrustedFuture.get(AbstractFuture.java:83)
        at com.google.common.util.concurrent.ForwardingFuture.get(ForwardingFuture.java:62)
        at Main.main(Main.java:30)
Caused by: com.google.api.gax.rpc.InternalException: io.grpc.StatusRuntimeException: INTERNAL: Panic! This is a bug!
        at com.google.api.gax.rpc.ApiExceptionFactory.createException(ApiExceptionFactory.java:67)
        at com.google.api.gax.grpc.GrpcApiExceptionFactory.create(GrpcApiExceptionFactory.java:72)
        at com.google.api.gax.grpc.GrpcApiExceptionFactory.create(GrpcApiExceptionFactory.java:60)
        at com.google.api.gax.grpc.GrpcExceptionCallable$ExceptionTransformingFuture.onFailure(GrpcExceptionCallable.java:97)
        at com.google.api.core.ApiFutures$1.onFailure(ApiFutures.java:68)
        at com.google.common.util.concurrent.Futures$CallbackListener.run(Futures.java:1052)
        at com.google.common.util.concurrent.MoreExecutors$DirectExecutor.execute(MoreExecutors.java:398)
        at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1030)
        at com.google.common.util.concurrent.AbstractFuture.addListener(AbstractFuture.java:675)
        at com.google.common.util.concurrent.ForwardingListenableFuture.addListener(ForwardingListenableFuture.java:45)
        at com.google.api.core.ApiFutureToListenableFuture.addListener(ApiFutureToListenableFuture.java:52)
        at com.google.common.util.concurrent.Futures.addCallback(Futures.java:1033)
        at com.google.api.core.ApiFutures.addCallback(ApiFutures.java:63)
        at com.google.api.gax.grpc.GrpcExceptionCallable.futureCall(GrpcExceptionCallable.java:67)
        at com.google.api.gax.tracing.TracedUnaryCallable.futureCall(TracedUnaryCallable.java:75)
        at com.google.api.gax.rpc.UnaryCallable$1.futureCall(UnaryCallable.java:126)
        at com.google.api.gax.rpc.UnaryCallable.futureCall(UnaryCallable.java:87)
        at com.google.cloud.firestore.FirestoreImpl.sendRequest(FirestoreImpl.java:413)
        at com.google.cloud.firestore.UpdateBuilder.commit(UpdateBuilder.java:621)
        at com.google.cloud.firestore.WriteBatch.commit(WriteBatch.java:41)
        at com.google.cloud.firestore.DocumentReference.set(DocumentReference.java:172)
        at Main.main(Main.java:26)
Caused by: io.grpc.StatusRuntimeException: INTERNAL: Panic! This is a bug!
        at io.grpc.Status.asRuntimeException(Status.java:530)
        at io.grpc.stub.ClientCalls$UnaryStreamToFuture.onClose(ClientCalls.java:482)
        at io.grpc.PartialForwardingClientCallListener.onClose(PartialForwardingClientCallListener.java:39)
        at io.grpc.ForwardingClientCallListener.onClose(ForwardingClientCallListener.java:23)
        at io.grpc.ForwardingClientCallListener$SimpleForwardingClientCallListener.onClose(ForwardingClientCallListener.java:40)
        at io.grpc.internal.CensusStatsModule$StatsClientInterceptor$1$1.onClose(CensusStatsModule.java:699)
        at io.grpc.PartialForwardingClientCallListener.onClose(PartialForwardingClientCallListener.java:39)
        at io.grpc.ForwardingClientCallListener.onClose(ForwardingClientCallListener.java:23)
        at io.grpc.ForwardingClientCallListener$SimpleForwardingClientCallListener.onClose(ForwardingClientCallListener.java:40)
        at io.grpc.internal.CensusTracingModule$TracingClientInterceptor$1$1.onClose(CensusTracingModule.java:397)
        at io.grpc.internal.ClientCallImpl.closeObserver(ClientCallImpl.java:459)
        at io.grpc.internal.ClientCallImpl.access$300(ClientCallImpl.java:63)
        at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl.close(ClientCallImpl.java:546)
        at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl.access$600(ClientCallImpl.java:467)
        at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInContext(ClientCallImpl.java:584)
        at io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37)
        at io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:123)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.IllegalStateException: Could not find policy 'pick_first'. Make sure its implementation is either registered to LoadBalancerRegistry or included in META-INF/services/io.grpc.LoadBalancerProvider from your jar files.
        at io.grpc.internal.AutoConfiguredLoadBalancerFactory$AutoConfiguredLoadBalancer.<init>(AutoConfiguredLoadBalancerFactory.java:93)
        at io.grpc.internal.AutoConfiguredLoadBalancerFactory.newLoadBalancer(AutoConfiguredLoadBalancerFactory.java:64)
        at io.grpc.internal.ManagedChannelImpl.exitIdleMode(ManagedChannelImpl.java:357)
        at io.grpc.internal.ManagedChannelImpl$ChannelTransportProvider$1ExitIdleModeForTransport.run(ManagedChannelImpl.java:455)
        at io.grpc.SynchronizationContext.drain(SynchronizationContext.java:101)
        at io.grpc.SynchronizationContext.execute(SynchronizationContext.java:130)
        at io.grpc.internal.ManagedChannelImpl$ChannelTransportProvider.get(ManagedChannelImpl.java:459)
        at io.grpc.internal.ClientCallImpl.start(ClientCallImpl.java:241)
        at io.grpc.internal.CensusTracingModule$TracingClientInterceptor$1.start(CensusTracingModule.java:392)
        at io.grpc.internal.CensusStatsModule$StatsClientInterceptor$1.start(CensusStatsModule.java:694)
        at io.grpc.ForwardingClientCall.start(ForwardingClientCall.java:32)
        at com.google.api.gax.grpc.GrpcHeaderInterceptor$1.start(GrpcHeaderInterceptor.java:95)
        at io.grpc.stub.ClientCalls.startCall(ClientCalls.java:308)
        at io.grpc.stub.ClientCalls.asyncUnaryRequestCall(ClientCalls.java:280)
        at io.grpc.stub.ClientCalls.futureUnaryCall(ClientCalls.java:189)
        at com.google.api.gax.grpc.GrpcDirectCallable.futureCall(GrpcDirectCallable.java:58)
        at com.google.api.gax.grpc.GrpcExceptionCallable.futureCall(GrpcExceptionCallable.java:64)
        at com.google.api.gax.tracing.TracedUnaryCallable.futureCall(TracedUnaryCallable.java:75)
        at com.google.api.gax.rpc.UnaryCallable$1.futureCall(UnaryCallable.java:126)
        at com.google.api.gax.rpc.UnaryCallable.futureCall(UnaryCallable.java:87)
        at com.google.cloud.firestore.FirestoreImpl.sendRequest(FirestoreImpl.java:413)
        at com.google.cloud.firestore.UpdateBuilder.commit(UpdateBuilder.java:621)
        at com.google.cloud.firestore.WriteBatch.commit(WriteBatch.java:41)
        at com.google.cloud.firestore.DocumentReference.set(DocumentReference.java:172)
        at Main.main(Main.java:26)

The text was updated successfully, but these errors were encountered:

creamsoup · 2019-03-25T16:36:07Z

@SkunFly can you provide how you configure loadbalancer when you create channel? did you try what error message suggest?

pereira-a · 2019-03-25T17:34:00Z

@SkunFly can you provide how you configure loadbalancer when you create channel? did you try what error message suggest?

I don't configure anything. I simply follow the guide and it should work. Maybe I'm missing something but I think not. If you try to replicate this you will face the same error. https://cloud.google.com/firestore/docs/quickstart-servers

I don't know if this is the correct place to open an issue, because I'm doing a guide about Firestore. But the stack trace, the exception message and the context of the error motivated me to open an issue here.

creamsoup · 2019-03-25T18:15:46Z

Can you run the complete quickstart example? if the issue is still exists, file against firestore? (or the example repository)
Even if this is a gRPC issue, they also need to aware of this issue. They can fix the issue or find a workaround until the proper fix is released if this is not just a tutorial issue.

pereira-a · 2019-03-25T19:34:34Z

Can you run the complete quickstart example? if the issue is still exists, file against firestore? (or the example repository)

When I execute the provided class "Quickstart" it gives me the above error.

Even if this is a gRPC issue, they also need to aware of this issue. They can fix the issue or find a workaround until the proper fix is released if this is not just a tutorial issue.

I already opened an issue in their platform, but at the moment they gave no response.

dapengzhang0 · 2019-03-25T20:11:12Z

@SkunFly Where is your client app running? GCP or Android?

pereira-a · 2019-03-25T20:21:46Z

@SkunFly Where is your client app running? GCP or Android?

GCP

mpdn · 2019-03-29T13:07:30Z

Hitting the same issue on a local machine. Same error and I do not have a loadbalancer configured either.

creamsoup · 2019-03-30T01:12:07Z

@Noctune, are you running the same code(firestore quickstart)?

mpdn · 2019-03-30T10:48:52Z

@creamsoup Nope. This is plain GRPC. I will try to see if I can find a minimal example that causes it, but it seems difficult to replicate.

mpdn · 2019-04-01T09:14:02Z

Turns out it was due to the Gradle Shadow plugin not merging service files by default. I just had to add this to my build.gradle and it works for me:

shadowJar {
    mergeServiceFiles()
}

creamsoup · 2019-04-01T16:18:44Z

@Noctune, thanks for finding the fix. do you want to make a PR?

mpdn · 2019-04-01T16:24:35Z

@creamsoup No, I had to add that snippet to my project's build.gradle, not grpc-java's. It is all on my end. :)

creamsoup · 2019-04-02T16:42:04Z

okay. that's common way to lose the META-INF/services. the original issue seems different. I filed an issue behalf of @SkunFly.

saturnism · 2019-04-02T19:15:00Z

i'm seeing a similar issue using Maven Shade plugin. I can confirm I do have META-INF/services/io.grpc.* files though.

saturnism · 2019-04-02T19:19:29Z

For Maven Shade plugin, was able to fix it by configuring to use ServicesResourceTransformer.

creamsoup · 2019-04-22T17:21:10Z

If you are shading or building uber jar solution mentioned above should fix the issue. for @SkunFly, see issue on the firestore.

ejona86 · 2019-05-07T22:43:19Z

Maven Shade plugin has been working since 2017: #3141 (comment)

creamsoup self-assigned this Mar 25, 2019

carl-mastrangelo added this to the Next milestone Mar 26, 2019

creamsoup closed this as completed Apr 22, 2019

ejona86 removed this from the Next milestone May 7, 2019

lock bot locked as resolved and limited conversation to collaborators Aug 5, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Exception java.lang.IllegalStateException: Could not find policy 'pick_first' #5493

Exception java.lang.IllegalStateException: Could not find policy 'pick_first' #5493

pereira-a commented Mar 24, 2019 •

edited

creamsoup commented Mar 25, 2019

pereira-a commented Mar 25, 2019

creamsoup commented Mar 25, 2019

pereira-a commented Mar 25, 2019

dapengzhang0 commented Mar 25, 2019

pereira-a commented Mar 25, 2019

mpdn commented Mar 29, 2019

creamsoup commented Mar 30, 2019

mpdn commented Mar 30, 2019

mpdn commented Apr 1, 2019 •

edited

creamsoup commented Apr 1, 2019

mpdn commented Apr 1, 2019

creamsoup commented Apr 2, 2019

saturnism commented Apr 2, 2019

saturnism commented Apr 2, 2019

creamsoup commented Apr 22, 2019

ejona86 commented May 7, 2019

Exception java.lang.IllegalStateException: Could not find policy 'pick_first' #5493

Exception java.lang.IllegalStateException: Could not find policy 'pick_first' #5493

Comments

pereira-a commented Mar 24, 2019 • edited

What version of gRPC are you using?

creamsoup commented Mar 25, 2019

pereira-a commented Mar 25, 2019

creamsoup commented Mar 25, 2019

pereira-a commented Mar 25, 2019

dapengzhang0 commented Mar 25, 2019

pereira-a commented Mar 25, 2019

mpdn commented Mar 29, 2019

creamsoup commented Mar 30, 2019

mpdn commented Mar 30, 2019

mpdn commented Apr 1, 2019 • edited

creamsoup commented Apr 1, 2019

mpdn commented Apr 1, 2019

creamsoup commented Apr 2, 2019

saturnism commented Apr 2, 2019

saturnism commented Apr 2, 2019

creamsoup commented Apr 22, 2019

ejona86 commented May 7, 2019

pereira-a commented Mar 24, 2019 •

edited

mpdn commented Apr 1, 2019 •

edited