1.13 Migration Guide
This guide helps users upgrading from Micrometer 1.12.x to Micrometer 1.13.x by explaining changes users should be aware of and potentially take action on. See also the release page for included changes.
The micrometer-registry-prometheus module has upgraded to the Prometheus Java client 1.x version, which required some breaking changes from the prior 0.x version. The breaking changes may not affect all users depending on whether you are directly interacting with the API in PrometheusMeterRegistry or a framework is handling that configuration for you. As a fallback to ease upgrading, the previous code based on the Prometheus Java client 0.x is available in a new module named micrometer-registry-prometheus-simpleclient (deprecated). The plan is to eventually remove support for the micrometer-registry-prometheus-simpleclient so it should only be used as an interim solution until you can adapt to the necessary changes in micrometer-registry-prometheus related to the upgrade to Prometheus Java client 1.x.
If you want to migrate to the Prometheus Java client 1.x with Micrometer 1.13.0:
- runtimeOnly 'io.micrometer:micrometer-registry-prometheus:1.12.6'
+ runtimeOnly 'io.micrometer:micrometer-registry-prometheus:1.13.0'
If you want to stay on the Prometheus Java client 0.x but you want to migrate to Micrometer 1.13.0:
- runtimeOnly 'io.micrometer:micrometer-registry-prometheus:1.12.6'
+ runtimeOnly 'io.micrometer:micrometer-registry-prometheus-simpleclient:1.13.0'
Frameworks that integrate with Micrometer and specifically the Micrometer Prometheus registry will need to update their support to adapt to the changes in micrometer-registry-prometheus for the Prometheus Java client 1.x upgrade.
Spring Boot offers dependency management as a feature, and best compatibility with integrations is ensured when you let Spring Boot manage the version of dependencies rather than override them (especially to new minor/major versions), including Micrometer. For example, Spring Boot's auto-configuration for the Prometheus registry will not work if you override the managed version of Micrometer to 1.13.0 when using Spring Boot 3.2 with micrometer-registry-prometheus (note that Spring Boot 3.2.x manages the version of Micrometer to 1.12.x), but it will work if you switch to the module for the Prometheus Java client 0.x (micrometer-registry-prometheus-simpleclient). Alternatively, upgrading to Spring Boot 3.3.x will result in the Micrometer version being managed to 1.13.x and auto-configuration working with either micrometer-registry-prometheus or micrometer-registry-prometheus-simpleclient (or both).
The micrometer-registry-prometheus module now uses the base package io.micrometer.prometheusmetrics rather than the previous io.micrometer.prometheus. If you are using any of the classes in the module in your code, you will need to update the import of the class.
For example
import io.micrometer.prometheus.PrometheusMeterRegistry;should be replaced to
import io.micrometer.prometheusmetrics.PrometheusMeterRegistry;and so on.
Some types exposed in the micrometer-registry-prometheus module's API were from the Prometheus Java client and they have changed. Accordingly, the API in PrometheusMeterRegistry has changed. Specifically, where a CollectorRegistry was used before, now a PrometheusRegistry is. The constructor that took an io.prometheus.client.exemplars.ExemplarSampler for exemplar support now takes a io.prometheus.metrics.tracer.common.SpanContext instead.
For example creating a PrometheusMeterRegistry using the Prometheus Java client 0.x can look like this:
import io.micrometer.prometheus.*;
PrometheusMeterRegistry prometheusMeterRegistry = new PrometheusMeterRegistry(
PrometheusConfig.DEFAULT,
new CollectorRegistry(),
Clock.SYSTEM,
new DefaultExemplarSampler(new DemoSpanContextSupplier())
);the same with the Prometheus Java client 1.x can look like this:
import io.micrometer.prometheusmetrics.*;
PrometheusMeterRegistry prometheusMeterRegistry = new PrometheusMeterRegistry(
PrometheusConfig.DEFAULT,
new PrometheusRegistry(),
Clock.SYSTEM,
new DemoSpanContext()
);HistogramFlavor as well as "native" support for VictoriaMetrics were removed since Prometheus Java client 1.x prevents changing the histogram bucket label (le vs. vmrange). On the other hand, according to the VictoriaMetrics docs, VictoriaMetrics can be used as drop-in replacement for Prometheus and should be able to scrape targets that Prometheus can.
Micrometer supports the new config mechanism of Prometheus Java client 1.x. You can read more about this in our docs. From migration perspective this can be an important detail because of the constructor of PrometheusMeterRegistry that took an io.prometheus.client.exemplars.ExemplarSampler for exemplar support now takes a io.prometheus.metrics.tracer.common.SpanContext instead which can prevent you customizing certain behaviors, like disabling Exemplars for _count. With the Prometheus Config support, you can modify the exemplar- and exporter properties of the Prometheus Java client 1.x, for example, you can disable exemplars on _count:
PrometheusConfig config = new PrometheusConfig() {
@Override
public String get(String key) {
return null;
}
@Override
public Properties prometheusProperties() {
Properties properties = new Properties();
properties.setProperty("io.prometheus.exporter.exemplarsOnAllMetricTypes", "false");
return properties;
}
};
PrometheusMeterRegistry registry = new PrometheusMeterRegistry(config, new PrometheusRegistry(), Clock.SYSTEM, spanContext);Because of behavioral differences and extra validation in Prometheus Java client 1.x (micrometer-registry-prometheus), there are changes in the scrape output compared to Prometheus Java client 0.x (micrometer-registry-prometheus-simpleclient).
LongTaskTimer is now registered as a GaugeHistogram type metric with the Prometheus registry rather than a Histogram/Summary type.
The _count and _sum time series are now _gcount and _gsum and they will no longer have _active and _duration respectively appended to the base metric name. See the following sample.
Old Client (0.x) (micrometer-registry-prometheus-simpleclient)
# HELP spring_security_authorizations_active_seconds
# TYPE spring_security_authorizations_active_seconds summary
spring_security_authorizations_active_seconds_active_count{application="my-app",spring_security_authentication_type="n/a",spring_security_authorization_decision="unknown",spring_security_object="request",} 0.0
spring_security_authorizations_active_seconds_duration_sum{application="my-app",spring_security_authentication_type="n/a",spring_security_authorization_decision="unknown",spring_security_object="request",} 0.0
New Client (1.x) (micrometer-registry-prometheus)
# HELP spring_security_authorizations_active_seconds
# TYPE spring_security_authorizations_active_seconds gaugehistogram
spring_security_authorizations_active_seconds_gcount{application="my-app",spring_security_authentication_type="n/a",spring_security_authorization_decision="unknown",spring_security_object="request"} 0
spring_security_authorizations_active_seconds_gsum{application="my-app",spring_security_authentication_type="n/a",spring_security_authorization_decision="unknown",spring_security_object="request"} 0.0
Gauge (Micrometer) suffixed with .info is now registered as Prometheus Info type metric rather than Prometheus Gauge type. This does not effect the name of the time series (what you query) but the type and the names in # TYPE and # HELP are different now. See the following sample.
Old Client (0.x) (micrometer-registry-prometheus-simpleclient)
# TYPE jvm_info gauge
# HELP jvm_info JVM version info
jvm_info{runtime="OpenJDK Runtime Environment",vendor="BellSoft",version="17.0.8.1+1-LTS"} 1.0
New Client (1.x) (micrometer-registry-prometheus)
# TYPE jvm info
# HELP jvm JVM version info
jvm_info{runtime="OpenJDK Runtime Environment",vendor="BellSoft",version="17.0.8.1+1-LTS"} 1
As you can see, the time-series (jvm_info{...}) did not change but the type (gauge -> info) and the names (jvm_info -> jvm) did.
Name suffixes are validated by Prometheus Java client 1.x, the following suffixes are now invalid and will be removed from the Meter names by PrometheusMeterRegistry: .info, .total, .created, .bucket (or _info, _total, _created, _bucket). These suffixes are used by Prometheus Java client 1.x and in certain situations they will be appended to the name. This can have some interesting consequences:
- ✅ If you create a Micrometer
Gaugenamedtest.info, the name of the Prometheus time series will betest_infosince the registry removes but the Prometheus client adds back the_infosuffix (see the previous section aboutInfovs.Gaugetype). - ✅ If you create a Micrometer
Counternamedtest.total, the name of the Prometheus time series will betest_totalsince the registry removes but the Prometheus client adds back the_totalsuffix. ⚠️ If you create a MicrometerCounternamedtest.info, the name of the Prometheus time series will betest_totalsince the registry removes the_infosuffix and the Prometheus client adds the_totalsuffix forcounters.⚠️ If you create a MicrometerGaugenamedtest.total, the name of the Prometheusgaugewill betestsince the registry removes the_totalsuffix and the Prometheus client does not add_infosuffix forgauges.⚠️ Similarly,_createdand_bucketsuffixes will be removed, otherwise the the Prometheus client will fail.
As you can see from the above, using .total, .created, and .bucket suffixes should be avoided to avoid unexpected outputs. A valid scenario could be using the .info suffix with Gauge (and only with Gauge) if you want the Prometheus type to be info instead of gauge (see the previous section about info vs. gauge type). For example this sample: registry.counter("test.info").increment(); produces the following output using the new Prometheus Java client (1.x) (micrometer-registry-prometheus):
# TYPE test counter
# HELP test
test_total 1.0
Please notice that the name of the registered Counter was test.info while the name of the time series is test_total.
Names are validated by Prometheus Java client 1.x and using single character Meter names (e.g.: registry.counter("c").increment();) will result in an exception on Prometheus Java client side:
Exception in thread "main" java.lang.IllegalArgumentException: 'c': Illegal metric name. The metric name contains unsupported characters Call PrometheusNaming.sanitizeMetricName(name) to avoid this error.
On the other hand, single character Tag names and values (Prometheus Labels) are still valid: registry.counter("test", "a", "b").increment();.
The number format of _count and _bucket (Summary/Histogram) was changed in Prometheus Java client 1.x. Since these values are always integers, the fraction was dropped. Theoretically this should not cause issues, the server that scrapes the output should be able to parse both. See the following example using Micrometer Timer (Timer.builder("test").register(registry).record(100, MILLISECONDS);).
Old Client (0.x) (micrometer-registry-prometheus-simpleclient)
# TYPE test_seconds summary
# HELP test_seconds
test_seconds_count 1.0
test_seconds_sum 0.1
New Client (1.x) (micrometer-registry-prometheus)
# TYPE test_seconds summary
# HELP test_seconds
test_seconds_count 1
test_seconds_sum 0.1
Please notice the value of test_seconds_count changing from 1.0 to 1.
You can experience the same behavior with _bucket if you enable .publishPercentileHistogram():
Old Client (0.x) (micrometer-registry-prometheus-simpleclient)
# TYPE test_seconds histogram
# HELP test_seconds
test_seconds_bucket{le="0.001"} 0.0
[...]
New Client (1.x) (micrometer-registry-prometheus)
# TYPE test_seconds histogram
# HELP test_seconds
test_seconds_bucket{le="0.001"} 0
[...]
The Prometheus Text Format (but not the OpenMetrics format) contained trailing commas (,) in the label list of every time series that contained labels. These trailing commas were removed in Prometheus Java client 1.x. Theoretically this should not cause issues, the server that scrapes the output should be able to parse both especially in case of modern Prometheus servers since they ask for the OpenMetrics format where trailing commas were missing originally so nothing needed to be removed. See the following example using a Micrometer Counter (registry.counter("test.counter", "a", "b").increment();).
Old Client (0.x) (micrometer-registry-prometheus-simpleclient) using Prometheus Text Format
# HELP test_counter_total
# TYPE test_counter_total counter
test_counter_total{a="b",} 1.0
New Client (1.x) (micrometer-registry-prometheus) using Prometheus Text Format
# HELP test_counter_total
# TYPE test_counter_total counter
test_counter_total{a="b"} 1.0
Please notice the end of the time series changing from {a="b",} 1.0 to {a="b"} 1.0.
The value of the Exemplar used in _count (Summary/Histogram) was changed. In the old client (0.x) (micrometer-registry-prometheus-simpleclient), the value was always 1.0 since for every recording, the counter was incremented by 1. In the new client (1.x) (micrometer-registry-prometheus), the Exemplar value of _count is the recorded value. For example in the case of recording latency, if the last recorded (and sampled) value was 250ms (0.25s), the Exemplar value is 0.25 (recorded latency) instead of 1.0 (counter increment). See the following example using Micrometer Timer (Timer.builder("test").register(registry).record(250, MILLISECONDS);).
Old Client (0.x) (micrometer-registry-prometheus-simpleclient)
# TYPE test_seconds summary
# HELP test_seconds
test_seconds_count 1.0 # {span_id="321",trace_id="123"} 1.0 1715384800.092
test_seconds_sum 0.25
New Client (1.x) (micrometer-registry-prometheus)
# TYPE test_seconds summary
# HELP test_seconds
test_seconds_count 1 # {span_id="321",trace_id="123"} 0.25 1715384800.092
test_seconds_sum 0.25
Please notice the value of Exemplar changing from 1.0 to 0.25 in # {span_id="321",trace_id="123"} 0.25.