Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DefaultWebMvcTagsProvider Code Flow. #2767

Closed
fitchner opened this issue Aug 31, 2021 · 3 comments
Closed

DefaultWebMvcTagsProvider Code Flow. #2767

fitchner opened this issue Aug 31, 2021 · 3 comments
Labels
bug A general bug registry: prometheus A Prometheus Registry related issue waiting for feedback We need additional information before we can continue

Comments

@fitchner
Copy link

Describe the bug
The code workflow when adding tags to Prometheus registry is erreneous

Environment

Springboot: 2.4.1

  • Micrometer version [1.7.0]
  • Micrometer registry [prometheus]
  • OS: [Windows 10]
  • Java version: [
    openjdk version "11.0.11" 2021-04-20 LTS
    OpenJDK Runtime Environment SapMachine (build 11.0.11+9-LTS-sapmachine)
    OpenJDK 64-Bit Server VM SapMachine (build 11.0.11+9-LTS-sapmachine, mixed mode)]

To Reproduce
How to reproduce the bug:
Sample Code used for WebMvcTagsContributor Bean:

@Bean
    public WebMvcTagsContributor webMvcTagsContributor() {
        return new WebMvcTagsContributor() {
            @Override
            public Iterable<Tag> getTags(HttpServletRequest request, HttpServletResponse response, Object handler, Throwable exception) {
                return  Tags.of("test","TEST");
            }

            @Override
            public Iterable<Tag> getLongRequestTags(HttpServletRequest request, Object handler) {
                return null;
            }
        };
    }

BreakPoints Useful when debugging: applyToCollector

applytoCollector-2

CollectorResistry.register(Collector)

Expected behavior
The DefaultMvcProvider populates the tags before the code execution reaches PrometheusRegistry.

Additional context
Bug in PrometheusMeterRegistry

ERROR:
Collector already registered that provides name: http_server_requests_seconds

By surfing on web for quick fixes on the internet for this issue found the following articles:
https://stackoverflow.com/questions/52893435/including-spring-boot-endpoint-path-variable-as-a-metric-dimension
#2399
prometheus/client_java#279

Although These articles discuss the error faced by me they are caused by a different requirement. i.e to have different tags for the same metric.

But we need the same tags for the metric throughout the runtime of application.

Dependencies Used:
io.micrometer:micrometer-registry-prometheus:1.6.2

Then i started to debug the application and found the following.

When Using "WebMvcTagsContributor" Bean or extending "WebMvcTagsProvider" configuration
The call to applyToCollector checks if the metric has a MicrometerCollector entry for the metric and if it doesn't, it would add it to the collector.
When checking the MicrometerCollectorentry the default tags are being used micrometerCollector.register(registry)
There are default tags assosciated to the metric here are without invocation of "DefaultWebMvcTagsProvider.getTags()", Not sure how this is being populated.
This method returns Collector.register(CollectorRegistry) -> CollectorResistry.register(Collector) Here the Collecter is populated with default tags.
Ater this execution is complete.

It follows to "DefaultWebMvcTagsProvider.getTags()", here the additional tags that are provided in the bean are populated.
Following the above tag reassignment it again proceeds to applyToCollector

Since the entry for MicrometerCollector is populated the method would direct to
if clause existingCollector.getTagKeys().equals(tagKeys)applytoCollector-2 There is a mismatch because of the nely added tags from defaultmvctags provider.

And for this reason the CollectorMap in PrometheusMeterRegistry is not updated whereas the collector is populated with default tags, when a new request arrives PrometheusMeterRegistry will try to populate the collecor with an existing entry.

This causes an exception like following:
Collector already registered that provides name: http_server_requests_seconds

@fitchner fitchner added the bug A general bug label Aug 31, 2021
@shakuzen shakuzen added the registry: prometheus A Prometheus Registry related issue label Aug 31, 2021
@shakuzen
Copy link
Member

shakuzen commented Sep 2, 2021

Please try with the latest patch versions to see if the issue hasn't already been fixed. I wasn't able to reproduce it on the latest patch versions. I tried in a sample app using Spring Boot 2.5.4 or 2.4.10 with a bean as you provided and the tag test was on all of the http_server_requests metrics in the Prometheus endpoint, as I would expect. I did not see any errors.

@shakuzen shakuzen added the waiting for feedback We need additional information before we can continue label Sep 2, 2021
@fitchner
Copy link
Author

fitchner commented Sep 3, 2021

Thank You for the update: @shakuzen
Looked into the issue further
Seems like there were two dependencies providing tags for http.server.metrics
DefaultWebMvcTagsProvider and DefaultJerseyTagsProvider this was coming from io.micrometer:micrometer-jersey2 dependency

Closing the issue as it was not a problem with the class.

@fitchner fitchner closed this as completed Sep 3, 2021
@shakuzen
Copy link
Member

shakuzen commented Sep 3, 2021

Thanks for the update. By the way, here is the Spring Boot documentation on Jersey server metrics auto-configuration.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug A general bug registry: prometheus A Prometheus Registry related issue waiting for feedback We need additional information before we can continue
Projects
None yet
Development

No branches or pull requests

2 participants