You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
What happened?
Deploying a Prometheus instance with mTLS configured and spec.web.tlsConfig.clientAuthType set to "RequireAndVerifyClientCert" results in the Deployment never reaching readiness. The operator sets the probes behind the authenticated endpoints, which prohibits kubelet from reaching them, since the readiness probes do not support setting client certificates at this point.
$ kubectl logs pod/prometheus-example-0...ts=2023-03-16T18:03:23.514Z caller=stdlib.go:105 level=error component=web caller="http: TLS handshake error from [::1" msg="]:38722: tls: client didn't provide a certificate"
$ kubectl get events...17s Warning Unhealthy pod/prometheus-test-0 Startup probe failed: Get "https://10.244.0.37:9090/-/ready": remote error: tls: bad certificate
Did you expect to see something different?
The readiness probe should not fail in this scenario. The readiness probe shouldn't be set up behind an authenticated endpoint.
How to reproduce it (as minimally and precisely as possible):
Prerequisites: prometheus-operator deployed in your cluster
Create a TLS certificate and key for the server and a CA certificate for client certificate authentication to the server.
Create a example-prometheus-client-ca ConfigMap with ca.crt key set to the CA certificate for client authentication.
Create a TLS Secret example-prometheus-serving-certs containing the serving certificate and key under keys tls.crt and tls.key correspondingly.
What happened?
Deploying a Prometheus instance with mTLS configured and
spec.web.tlsConfig.clientAuthType
set to"RequireAndVerifyClientCert"
results in the Deployment never reaching readiness. The operator sets the probes behind the authenticated endpoints, which prohibits kubelet from reaching them, since the readiness probes do not support setting client certificates at this point.Did you expect to see something different?
The readiness probe should not fail in this scenario. The readiness probe shouldn't be set up behind an authenticated endpoint.
How to reproduce it (as minimally and precisely as possible):
Prerequisites: prometheus-operator deployed in your cluster
example-prometheus-client-ca
ConfigMap withca.crt
key set to the CA certificate for client authentication.example-prometheus-serving-certs
containing the serving certificate and key under keystls.crt
andtls.key
correspondingly.Environment
Prometheus Operator version:
v0.61.1
Kubernetes version information:
Kubernetes cluster kind:
insert how you created your cluster: kops, bootkube, etc.
Manifests:
as above
Prometheus Operator Logs:
N/A
Anything else we need to know?:
The text was updated successfully, but these errors were encountered: