hazelcast-client 3.10.5 not auto-discovering on kubernetes

[dacleyra]

[9/13/18 19:20:16:014 UTC] 00000067 com.hazelcast.core.LifecycleService                          I file:/opt/ibm/wlp/usr/shared/config//hazelcast/hazelcast-client.xml [dev] [3.10.5] HazelcastClient 3.10.5 (20180913 - 6ffa2ee) is STARTED
[9/13/18 19:20:16:593 UTC] 000000bc com.hazelcast.kubernetes.RetryUtils                          W Couldn't connect to the Kubernetes master, [1] retrying in 1 seconds...
[9/13/18 19:20:18:107 UTC] 000000bc com.hazelcast.kubernetes.RetryUtils                          W Couldn't connect to the Kubernetes master, [2] retrying in 2 seconds...
[9/13/18 19:20:20:369 UTC] 000000bc com.hazelcast.kubernetes.RetryUtils                          W Couldn't connect to the Kubernetes master, [3] retrying in 3 seconds...
[9/13/18 19:20:23:755 UTC] 000000bc com.hazelcast.kubernetes.RetryUtils                          W Couldn't connect to the Kubernetes master, [4] retrying in 5 seconds...
[9/13/18 19:20:28:827 UTC] 000000bc com.hazelcast.kubernetes.RetryUtils                          W Couldn't connect to the Kubernetes master, [5] retrying in 7 seconds...
[9/13/18 19:20:36:431 UTC] 000000bc com.hazelcast.kubernetes.RetryUtils                          W Couldn't connect to the Kubernetes master, [6] retrying in 11 seconds...
[9/13/18 19:20:47:833 UTC] 000000bc com.hazelcast.kubernetes.RetryUtils                          W Couldn't connect to the Kubernetes master, [7] retrying in 17 seconds...
[9/13/18 19:21:04:926 UTC] 000000bc com.hazelcast.kubernetes.RetryUtils                          W Couldn't connect to the Kubernetes master, [8] retrying in 25 seconds...
[9/13/18 19:21:30:562 UTC] 000000bc com.hazelcast.kubernetes.RetryUtils                          W Couldn't connect to the Kubernetes master, [9] retrying in 38 seconds...
[9/13/18 19:22:09:009 UTC] 000000bc com.hazelcast.kubernetes.RetryUtils                          W Couldn't connect to the Kubernetes master, [10] retrying in 57 seconds...

[9/13/18 19:23:06:677 UTC] 000000bc com.hazelcast.client.connection.ClientConnectionManager      W file:/opt/ibm/wlp/usr/shared/config//hazelcast/hazelcast-client.xml [dev] [3.10.5] Exception from AddressProvider: com.hazelcast.client.spi.impl.discovery.DiscoveryAddressProvider@9418ef16
com.hazelcast.kubernetes.KubernetesClientException: Failure in KubernetesClient
        at com.hazelcast.kubernetes.DefaultKubernetesClient.callGet(DefaultKubernetesClient.java:109)
        at com.hazelcast.kubernetes.DefaultKubernetesClient.endpoints(DefaultKubernetesClient.java:74)
        at com.hazelcast.kubernetes.RetryKubernetesClient$1.call(RetryKubernetesClient.java:46)
        at com.hazelcast.kubernetes.RetryKubernetesClient$1.call(RetryKubernetesClient.java:42)
        at com.hazelcast.kubernetes.RetryUtils.retry(RetryUtils.java:52)
        at com.hazelcast.kubernetes.RetryKubernetesClient.endpoints(RetryKubernetesClient.java:42)
        at com.hazelcast.kubernetes.ServiceEndpointResolver.resolve(ServiceEndpointResolver.java:78)
        at com.hazelcast.kubernetes.HazelcastKubernetesDiscoveryStrategy.discoverNodes(HazelcastKubernetesDiscoveryStrategy.java:105)
        at com.hazelcast.spi.discovery.impl.DefaultDiscoveryService.discoverNodes(DefaultDiscoveryService.java:71)
        at com.hazelcast.client.spi.impl.discovery.DiscoveryAddressProvider.loadAddresses(DiscoveryAddressProvider.java:44)
        at com.hazelcast.client.connection.nio.ClientConnectionManagerImpl.getPossibleMemberAddresses(ClientConnectionManagerImpl.java:918)
        at com.hazelcast.client.connection.nio.ClientConnectionManagerImpl.connectToClusterInternal(ClientConnectionManagerImpl.java:836)
        at com.hazelcast.client.connection.nio.ClientConnectionManagerImpl.access$1200(ClientConnectionManagerImpl.java:108)
        at com.hazelcast.client.connection.nio.ClientConnectionManagerImpl$2.call(ClientConnectionManagerImpl.java:881)
        at com.hazelcast.client.connection.nio.ClientConnectionManagerImpl$2.call(ClientConnectionManagerImpl.java:877)
        at java.util.concurrent.FutureTask.run(FutureTask.java:277)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1160)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
        at java.lang.Thread.run(Thread.java:812)
        at com.hazelcast.util.executor.HazelcastManagedThread.executeRun(HazelcastManagedThread.java:64)
        at com.hazelcast.util.executor.HazelcastManagedThread.run(HazelcastManagedThread.java:80)
Caused by: javax.net.ssl.SSLException: Received fatal alert: protocol_version
        at com.ibm.jsse2.k.a(k.java:5)
        at com.ibm.jsse2.k.a(k.java:23)
        at com.ibm.jsse2.av.b(av.java:343)
        at com.ibm.jsse2.av.a(av.java:981)
        at com.ibm.jsse2.av.i(av.java:869)
        at com.ibm.jsse2.av.a(av.java:19)
        at com.ibm.jsse2.av.startHandshake(av.java:672)
        at com.ibm.net.ssl.www2.protocol.https.c.afterConnect(c.java:46)
        at com.ibm.net.ssl.www2.protocol.https.d.connect(d.java:35)
        at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1576)
        at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1504)
        at java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:491)
        at com.ibm.net.ssl.www2.protocol.https.b.getResponseCode(b.java:14)
        at com.hazelcast.kubernetes.DefaultKubernetesClient.callGet(DefaultKubernetesClient.java:103)
        ... 20 more

    <hazelcast-client xsi:schemaLocation="http://www.hazelcast.com/schema/client-config hazelcast-client-config-3.9.xsd"
               xmlns="http://www.hazelcast.com/schema/client-config"
               xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
      <properties>
        <property name="hazelcast.discovery.enabled">true</property>
      </properties>
      <network>
        <redo-operation>true</redo-operation>
        <discovery-strategies>
          <discovery-strategy enabled="true" class="com.hazelcast.kubernetes.HazelcastKubernetesDiscoveryStrategy">
          </discovery-strategy>
        </discovery-strategies>
      </network>
    </hazelcast-client>

This is not a problem in 3.10.4

@mesutcelik notes
hazelcast/hazelcast:3.10.5 --> hazelcast-kubernetes-1.2
hazelcast/hazelcast:3.10.4 --> hazelcast-kubernetes-1.1

Building a customized version of 3.10.5, can confirm the difference is caused by
HZ_KUBE_VERSION=1.2 vs. 1.1.0

https://github.com/hazelcast/hazelcast-docker/blob/v3.10.4/hazelcast-oss/Dockerfile#L6