You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Set LimitNOFILE = 1000 to BlackBox service configuration and restart it
Scrape 16 oauth2 metrics per minute and wait 1 hour
What did you expect to see?
Something like that when we restarted Blackbox service
[root@ip-XX-XXX-XXX-XXX ec2-user]# curl 'http://localhost:9115/probe?module=oauth2&debug=true&target=https://prometheus.my_domain.com/'
Logs for the probe:
ts=2022-07-01T10:18:54.098569302Z caller=main.go:351 module=oauth2 target=https://prometheus.my_domain.com/ level=info msg="Beginning probe" probe=http timeout_seconds=60
ts=2022-07-01T10:18:54.098676009Z caller=http.go:336 module=oauth2 target=https://prometheus.my_domain.com/ level=info msg="Resolving target address" target=prometheus.my_domain.com ip_protocol=ip6
ts=2022-07-01T10:18:54.111697107Z caller=http.go:336 module=oauth2 target=https://prometheus.my_domain.com/ level=info msg="Resolved target address" target=prometheus.my_domain.com ip=XX.XXX.XX.XX
ts=2022-07-01T10:18:54.111811482Z caller=client.go:252 module=oauth2 target=https://prometheus.my_domain.com/ level=info msg="Making HTTP request" url=https://XX.XXX.XX.XX/ host=prometheus.my_domain.com
ts=2022-07-01T10:18:54.558533162Z caller=client.go:498 module=oauth2 target=https://prometheus.my_domain.com/ level=info msg="Received redirect" location="https://my_domain.com/authorization.oauth2?approval_prompt=force&client_id=my_client_id&redirect_uri=https%3A%2F%2Fprometheus.my_domain.com%2Foauth2%2Fcallback&response_type=code&scope=openid+email+profile&state=PKnvKS6t75NMU8wQRrNDX5kAmOWxhNJadQQm7tjtTY0%3A%2F"
ts=2022-07-01T10:18:54.558646018Z caller=client.go:252 module=oauth2 target=https://prometheus.my_domain.com/ level=info msg="Making HTTP request" url="https://my_domain.com/authorization.oauth2?approval_prompt=force&client_id=my_client_id&redirect_uri=https%3A%2F%2Fprometheus.my_domain.com%2Foauth2%2Fcallback&response_type=code&scope=openid+email+profile&state=PKnvKS6t75NMU8wQRrNDX5kAmOWxhNJadQQm7tjtTY0%3A%2F" host=
ts=2022-07-01T10:18:54.558675648Z caller=client.go:252 module=oauth2 target=https://prometheus.my_domain.com/ level=info msg="Address does not match first address, not sending TLS ServerName" first=XX.XXX.XX.XX address=my_domain.com
ts=2022-07-01T10:18:55.228423457Z caller=main.go:144 module=oauth2 target=https://prometheus.my_domain.com/ level=info msg="Received HTTP response" status_code=200
ts=2022-07-01T10:18:55.228539117Z caller=main.go:144 module=oauth2 target=https://prometheus.my_domain.com/ level=info msg="Response timings for roundtrip" roundtrip=0 start=2022-07-01T10:18:54.500873437Z dnsDone=2022-07-01T10:18:54.500873437Z connectDone=2022-07-01T10:18:54.514487527Z gotConn=2022-07-01T10:18:54.543274184Z responseStart=2022-07-01T10:18:54.558467253Z tlsStart=2022-07-01T10:18:54.514551374Z tlsDone=2022-07-01T10:18:54.543225678Z end=0001-01-01T00:00:00Z
ts=2022-07-01T10:18:55.228576239Z caller=main.go:144 module=oauth2 target=https://prometheus.my_domain.com/ level=info msg="Response timings for roundtrip" roundtrip=1 start=2022-07-01T10:18:54.880164122Z dnsDone=2022-07-01T10:18:54.881096846Z connectDone=2022-07-01T10:18:54.955550621Z gotConn=2022-07-01T10:18:55.1050576Z responseStart=2022-07-01T10:18:55.228317479Z tlsStart=2022-07-01T10:18:54.955618587Z tlsDone=2022-07-01T10:18:55.105037353Z end=2022-07-01T10:18:55.228506351Z
ts=2022-07-01T10:18:55.228659807Z caller=main.go:351 module=oauth2 target=https://prometheus.my_domain.com/ level=info msg="Probe succeeded" duration_seconds=1.130064076
Metrics that would have been returned:
# HELP probe_dns_lookup_time_seconds Returns the time taken for probe dns lookup in seconds
# TYPE probe_dns_lookup_time_seconds gauge
probe_dns_lookup_time_seconds 0.013047818
# HELP probe_duration_seconds Returns how long the probe took to complete in seconds
# TYPE probe_duration_seconds gauge
probe_duration_seconds 1.130064076
# HELP probe_failed_due_to_regex Indicates if probe failed due to regex
# TYPE probe_failed_due_to_regex gauge
probe_failed_due_to_regex 0
# HELP probe_http_content_length Length of http content response
# TYPE probe_http_content_length gauge
probe_http_content_length 10814
# HELP probe_http_duration_seconds Duration of http request by phase, summed over all redirects
# TYPE probe_http_duration_seconds gauge
probe_http_duration_seconds{phase="connect"} 0.088067846
probe_http_duration_seconds{phase="processing"} 0.138452967
probe_http_duration_seconds{phase="resolve"} 0.013980550999999999
probe_http_duration_seconds{phase="tls"} 0.178093063
probe_http_duration_seconds{phase="transfer"} 0.000188865
# HELP probe_http_redirects The number of redirects
# TYPE probe_http_redirects gauge
probe_http_redirects 1
# HELP probe_http_ssl Indicates if SSL was used for the final redirect
# TYPE probe_http_ssl gauge
probe_http_ssl 1
# HELP probe_http_status_code Response HTTP status code
# TYPE probe_http_status_code gauge
probe_http_status_code 200
# HELP probe_http_uncompressed_body_length Length of uncompressed response body
# TYPE probe_http_uncompressed_body_length gauge
probe_http_uncompressed_body_length 10814
# HELP probe_http_version Returns the version of HTTP of the probe response
# TYPE probe_http_version gauge
probe_http_version 1.1
# HELP probe_ip_addr_hash Specifies the hash of IP address. It's useful to detect if the IP address changes.
# TYPE probe_ip_addr_hash gauge
probe_ip_addr_hash 2.397761283e+09
# HELP probe_ip_protocol Specifies whether probe ip protocol is IP4 or IP6
# TYPE probe_ip_protocol gauge
probe_ip_protocol 4
# HELP probe_ssl_earliest_cert_expiry Returns earliest SSL cert expiry in unixtime
# TYPE probe_ssl_earliest_cert_expiry gauge
probe_ssl_earliest_cert_expiry 1.677531509e+09
# HELP probe_ssl_last_chain_expiry_timestamp_seconds Returns last SSL chain expiry in timestamp seconds
# TYPE probe_ssl_last_chain_expiry_timestamp_seconds gauge
probe_ssl_last_chain_expiry_timestamp_seconds 1.677531509e+09
# HELP probe_ssl_last_chain_info Contains SSL leaf certificate information
# TYPE probe_ssl_last_chain_info gauge
probe_ssl_last_chain_info{fingerprint_sha256="25fe13eacb5db65b46a604426a8ef04d8bc9d2b629d8fa4ec280d4ad19c04947"} 1
# HELP probe_success Displays whether or not the probe was a success
# TYPE probe_success gauge
probe_success 1
# HELP probe_tls_version_info Contains the TLS version used
# TYPE probe_tls_version_info gauge
probe_tls_version_info{version="TLS 1.2"} 1
Module configuration:
prober: http
timeout: 1m0s
http:
ip_protocol_fallback: true
oauth2:
client_id: my_client_id
client_secret: <secret>
client_secret_file: ""
scopes:
- email
- profile
- openid
token_url: https://my_domain.com/token.oauth2
endpoint_params:
grant_type: password
password: my_password
username: my_username
follow_redirects: true
enable_http2: true
tcp:
ip_protocol_fallback: true
icmp:
ip_protocol_fallback: true
dns:
ip_protocol_fallback: true
recursion_desired: true
Host operating system: output of
uname -a
Linux ip-XX-XXX-XXX-XXX.us-west-2.compute.internal 5.10.106-102.504.amzn2.x86_64 #1 SMP Tue Mar 29 23:15:13 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
blackbox_exporter version: output of
blackbox_exporter --version
What is the blackbox.yml module config.
What is the prometheus.yml scrape config.
What logging output did you get from adding
&debug=true
to the probe URL?No response from Blackbox service
What did you do that produced an error?
Set LimitNOFILE = 1000 to BlackBox service configuration and restart it
Scrape 16 oauth2 metrics per minute and wait 1 hour
What did you expect to see?
Something like that when we restarted Blackbox service
What did you see instead?
We see no response from Blackbox service
We see a lot of etableshed connection to the SSO auth server
We see the following errors
The text was updated successfully, but these errors were encountered: