Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unhandled Hz (C/S) exception when shutting down instance #17070

Closed
shashanka981 opened this issue Jun 10, 2020 · 3 comments · Fixed by #17153
Closed

Unhandled Hz (C/S) exception when shutting down instance #17070

shashanka981 opened this issue Jun 10, 2020 · 3 comments · Fixed by #17153
Assignees
Labels
Source: Community PR or issue was opened by a community user Team: Client Type: Defect
Milestone

Comments

@shashanka981
Copy link

shashanka981 commented Jun 10, 2020

When Hazelcast is run in Client/Server mode, an exception is reported on the hazelcast client instance that is being shut down. The message shown is:

(proxy.ClientReliableTopicProxy) WARN    - hz.client_0 [PegaCluster] [3.11.5] Terminating MessageListener com.org.platform.cluster.internal.HazelcastClientClusterProvider$MembershipEventListener@3aa798ff on topic: test/member/membership/event/topic. Reason: Unhandled exception, message: Client is shutting down
com.hazelcast.client.HazelcastClientNotActiveException: Client is shutting down
	at com.hazelcast.client.spi.impl.ClientInvocation.notifyException(ClientInvocation.java:215) ~[hazelcast-enterprise-client-3.11.5.jar:3.11.5]
	at com.hazelcast.client.spi.impl.AbstractClientInvocationService.shutdown(AbstractClientInvocationService.java:197) ~[hazelcast-enterprise-client-3.11.5.jar:3.11.5]
	at com.hazelcast.client.impl.clientside.HazelcastClientInstanceImpl.doShutdown(HazelcastClientInstanceImpl.java:962) ~[hazelcast-enterprise-client-3.11.5.jar:3.11.5]
	at com.hazelcast.client.impl.clientside.LifecycleServiceImpl.shutdown(LifecycleServiceImpl.java:152) ~[hazelcast-enterprise-client-3.11.5.jar:3.11.5]
	at com.hazelcast.client.impl.clientside.HazelcastClientProxy.shutdown(HazelcastClientProxy.java:297) ~[hazelcast-enterprise-client-3.11.5.jar:3.11.5]
	at 

Caused by: com.hazelcast.client.HazelcastClientNotActiveException: Client is shutting down
	... 34 more

Version of hazelcast used for both client and server is hazelcast enterprise 3.11.5
Listener referred in the issue is actually registered during client intialization as below

instance.getReliableTopic("test/member/membership/event/topic").addMessageListener();

Wanted to know if this is an issue or expected behaviour ? If it is an issue, what is the impact of this exception and how can we fix it

@sancar
Copy link
Contributor

sancar commented Jun 10, 2020

@shashanka981 This is an expected behaviour. Any method can throw HazelcastClientNotActiveException , if the client is being shutdown or it was already shutdown.
If you don't expect your client to shutdown quickly, you can configure so via connectionAttemptLimit. see https://docs.hazelcast.org/docs/3.11.5/manual/html-single/index.html#handling-failures

@sancar sancar self-assigned this Jun 10, 2020
@sancar sancar added Team: Client Type: Question Community Questions and removed Type: Defect labels Jun 10, 2020
@shashanka981
Copy link
Author

@sancar to clarify, this exception occurs not because client is not able to connect to cluster members, but when client is undergoing shutdown (via explicit call to HazelcastInstance.shutdown() invocation as part of applcation server shutdown hook.

Why does Hazelcast explicitly logs the message as a warning during shutdown ? What does the below message mean to a developer working with hazelcast ?

"Terminating MessageListener com.org.platform.cluster.internal.HazelcastClientClusterProvider$MembershipEventListener@3aa798ff on topic: test/member/membership/event/topic. Reason: Unhandled exception, message: Client is shutting down
com.hazelcast.client.HazelcastClientNotActiveException: Client is shutting down"

@sancar
Copy link
Contributor

sancar commented Jun 12, 2020

Why does Hazelcast explicitly logs the message as a warning during shutdown ? What does the below message mean to a developer working with hazelcast ?

ReliableTopic MessageListener supposed to handle the internal exceptions where it can and log them as finest if it can not handle. For example HazelcastInstanceNotActiveException and DistributedObjectDestroyedException are the ones that it can not handle and logged as finest . It seems that HazelcastClientNotActiveException should be in that list and skipped.

So to our users, it should be hidden. Since the client is shutting down, there is nothing user can do anymore, it does not make sense toWARN the user. As a workaround, you can safely ignore the log.

I am marking this issue as defect. We are maintaining 3.12.x and 4.x series at the moment. We will provide the fix for these versions.

@sancar sancar added Type: Defect good first issue Good for newcomers and removed Type: Question Community Questions labels Jun 12, 2020
@sancar sancar added this to the 3.12.8 milestone Jun 12, 2020
@sancar sancar removed their assignment Jun 12, 2020
@mmedenjak mmedenjak modified the milestones: 3.12.8, 3.12.9 Jun 19, 2020
@mmedenjak mmedenjak added the Source: Community PR or issue was opened by a community user label Jun 25, 2020
sancar pushed a commit to sancar/hazelcast that referenced this issue Jul 1, 2020
ReliableTopicMessageListener should not warn the user when
the client is shutting down. InstanceNotActiveException is
already in finest. This pr makes ClientNotActiveException log level
finest too.

fixes hazelcast#17070
sancar pushed a commit to sancar/hazelcast that referenced this issue Jul 1, 2020
ReliableTopicMessageListener should not warn the user when
the client is shutting down. InstanceNotActiveException is
already in finest. This pr makes ClientNotActiveException log level
finest too.

fixes hazelcast#17070

(cherry picked from commit d792c28)
sancar pushed a commit to sancar/hazelcast that referenced this issue Jul 1, 2020
ReliableTopicMessageListener should not warn the user when
the client is shutting down. InstanceNotActiveException is
already in finest. This pr makes ClientNotActiveException log level
finest too.

fixes hazelcast#17070
@sancar sancar removed the good first issue Good for newcomers label Jul 1, 2020
@sancar sancar self-assigned this Jul 1, 2020
sancar pushed a commit that referenced this issue Jul 2, 2020
ReliableTopicMessageListener should not warn the user when
the client is shutting down. InstanceNotActiveException is
already in finest. This pr makes ClientNotActiveException log level
finest too.

fixes #17070
sancar pushed a commit that referenced this issue Jul 2, 2020
ReliableTopicMessageListener should not warn the user when
the client is shutting down. InstanceNotActiveException is
already in finest. This pr makes ClientNotActiveException log level
finest too.

fixes #17070
sancar pushed a commit that referenced this issue Jul 2, 2020
ReliableTopicMessageListener should not warn the user when
the client is shutting down. InstanceNotActiveException is
already in finest. This pr makes ClientNotActiveException log level
finest too.

fixes #17070

(cherry picked from commit d792c28)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Source: Community PR or issue was opened by a community user Team: Client Type: Defect
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants