Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Document the bridge that allows for client_golang instrumented code to push with OpenTelemetry OTLP #1495

Open
bwplotka opened this issue Apr 10, 2024 · 3 comments

Comments

@bwplotka
Copy link
Member

bwplotka commented Apr 10, 2024

Thanks to amazing community behind OpenTelemetry Go SDK (special kudos to @dashpole), users of client_golang can optionally import a new Go module, a special bridge [1][2] and push data from prometheus.Registry using OTLP push.

This is useful if you have your application metrics instrumented with Prometheus client_golang, but wish to use OTLP push instead of (or additionally to) promhttp /metrics pull method, because perhaps it's a batch job or you want to use push based collection mechanisms with e.g. OTel collector. Notably, initial benchmarks shows that, currently, this way is more efficient than e.g. pulling metrics using Otel collector Prometheus receiver and converting to OTLP data model. This situation, however, can change as the source of the efficiency issues can be, in theory, optimized away.

Opinionated disclaimer: Using OTLP push (or generally push) has its own disadvantages. Converting Prometheus metrics to OTLP format has another set of disadvantages. Our recommendation is to prefer pull mechanisms for metric pipelines for reliability, centralised discovery, efficiency and simple client failover scenarios. Standard promhttp /metrics pull mechanism remains the most efficient and reliable way to directly get metrics from your application to Prometheus or Prometheus compatible backend.

Nevertheless for those who wants to leverage pure OTLP, this issue is to track adding relevant explanation to our docs. This is also on part with Java Prometheus client offering OTLP

@kakkoyun
Copy link
Member

Opinionated disclaimer: Using OTLP push (or generally push) has its own disadvantages. Converting Prometheus metrics to OTLP format has another set of disadvantages. Our recommendation is to prefer pull mechanisms for metric pipelines for reliability, centralised discovery, efficiency and simple client failover scenarios. Standard promhttp /metrics pull mechanism remains the most efficient and reliable way to directly get metrics from your application to Prometheus or Prometheus compatible backend.

I can't emphasize this point more

@kakkoyun
Copy link
Member

but wish to use OTLP push instead of (or additionally to) promhttp /metrics pull method, because perhaps it's a batch job or you want to use push based collection mechanisms with e.g. OTel collector

In theory, this approach would eliminate the use of https://github.com/weaveworks/prom-aggregation-gateway?tab=readme-ov-file (or maintained version https://github.com/zapier/prom-aggregation-gateway)

@dashpole
Copy link

@bwplotka were you thinking of an example, or just documentation? If documentation, where do you think this belongs?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants