Everything is healthy but I get connection refused.

[simon-zumbrunnen]

I created a new talos cluster with three nodes. The dashboard says running, ready and healthy for all three nodes. The vip shows up on one of the nodes:

IP 192.168.1.31/24, 192.168.1.40/32

I can also ping the vip successfully. All nodes show the following warning:

 user: warning: [2024-05-11T11:17:36.5218914Z]: [talos] kubernetes endpoint watch error {"component": "controller-runtime", "controller":  
 "k8s.EndpointController", "error": "failed to list *v1.Endpoints: Get \x5c"https://192.168.1.40/api/v1/namespaces/default/endpoints?      
 fieldSelector=metadata.name%3Dkubernetes&limit=500&resourceVersion=0\x5c": dial tcp 192.168.1.40:443: connect: connection refused"} 

The health check shows this:

talosctl health --endpoints 192.168.1.31 --nodes 192.168.1.31
discovered nodes: ["192.168.1.32" "192.168.1.31" "192.168.1.33"]
waiting for etcd to be healthy: ...
waiting for etcd to be healthy: OK
waiting for etcd members to be consistent across nodes: ...
waiting for etcd members to be consistent across nodes: OK
waiting for etcd members to be control plane nodes: ...
waiting for etcd members to be control plane nodes: OK
waiting for apid to be ready: ...
waiting for apid to be ready: OK
waiting for all nodes memory sizes: ...
waiting for all nodes memory sizes: OK
waiting for all nodes disk sizes: ...
waiting for all nodes disk sizes: OK
waiting for kubelet to be healthy: ...
waiting for kubelet to be healthy: OK
waiting for all nodes to finish boot sequence: ...
waiting for all nodes to finish boot sequence: OK
waiting for all k8s nodes to report: ...
waiting for all k8s nodes to report: Get "https://192.168.1.40/api/v1/nodes": dial tcp 192.168.1.40:443: connect: connection refused

What am I doing wrong? How can I diagnose this further?

[simon-zumbrunnen]

I found the problem. The API server runs on port 6443. I forgot to add the port when I created the config.