Metrics static methods use a different registry than the injected MeterRegistry

[Davio]

I don't know whether this belongs on the Spring Boot side, but basically since version 1.7.0 of Micrometer and 2.5.0 of Spring Boot, all of my custom metrics had disappeared.

After some research, it seems like Metrics.counter("test").increment() adds the counter to the global registry which is a CompositeMeterRegistry. This counter does not get displayed at endpoint /actuator/metrics nor /actuator/prometheus.

When I use an injected MeterRegistry, I get a registry of type PrometheusMeterRegistry (I assume because of my micrometer-registry-prometheus dependency). If I register my counter with this registry, it is both shown in the metrics and prometheus endpoint.

Is there some configuration which I'm missing or is this a (known) bug?

[checketts]

This sounds like a new bug.

Out of curiosity, do you see any messages about <bean> is not eligible for getting processed by all BeanPostProcessors?

[Davio]

Yes, several, currently these are the relevant beans mentioned which are not eligible:

• Bean 'management.metrics-org.springframework.boot.actuate.autoconfigure.metrics.MetricsProperties' of type [org.springframework.boot.actuate.autoconfigure.metrics.MetricsProperties]
• Bean 'org.springframework.boot.actuate.autoconfigure.metrics.data.RepositoryMetricsAutoConfiguration' of type [org.springframework.boot.actuate.autoconfigure.metrics.data.RepositoryMetricsAutoConfiguration]
• Bean 'org.springframework.boot.actuate.autoconfigure.metrics.export.prometheus.PrometheusMetricsExportAutoConfiguration' of type [org.springframework.boot.actuate.autoconfigure.metrics.export.prometheus.PrometheusMetricsExportAutoConfiguration]
• Bean 'management.metrics.export.prometheus-org.springframework.boot.actuate.autoconfigure.metrics.export.prometheus.PrometheusProperties' of type [org.springframework.boot.actuate.autoconfigure.metrics.export.prometheus.PrometheusProperties]
• Bean 'prometheusConfig' of type [org.springframework.boot.actuate.autoconfigure.metrics.export.prometheus.PrometheusPropertiesConfigAdapter]
• Bean 'collectorRegistry' of type [io.prometheus.client.CollectorRegistry]
• Bean 'org.springframework.boot.actuate.autoconfigure.metrics.MetricsAutoConfiguration' of type [org.springframework.boot.actuate.autoconfigure.metrics.MetricsAutoConfiguration]
• Bean 'micrometerClock' of type [io.micrometer.core.instrument.Clock$1]
• Bean 'prometheusMeterRegistry' of type [io.micrometer.prometheus.PrometheusMeterRegistry]
• Bean 'repositoryTagsProvider' of type [org.springframework.boot.actuate.metrics.data.DefaultRepositoryTagsProvider]
• Bean 'metricsRepositoryMethodInvocationListener' of type [org.springframework.boot.actuate.metrics.data.MetricsRepositoryMethodInvocationListener]

[checketts]

That is the smoking gun. By PrometheusMeterRegistry being unavailable for bean post-processing then MeterRegistryPostProcessor can't configure it (and add it to the global registry)

You might have a dependency or code that is instantiating the Prometheus registry very early. See spring-projects/spring-framework#24092 for an overview (and note the linked Micrometer issues to see some more details)

Unfortunately this sounds like a Spring Boot issue.

[shakuzen]

since version 1.7.0 of Micrometer and 2.5.0 of Spring Boot, all of my custom metrics had disappeared.

Sounds like spring-projects/spring-boot#26630, which is fixed in the upcoming Spring Boot 2.5.1 release, scheduled for June 10. I would suggest trying your application on Spring Boot 2.5.1 snapshots to ensure the issue is fixed for you.

Under the assumption that is the cause, I'll close this as a duplicate of that issue. If that turns out to not be the case, we can reopen and investigate further.

[Davio]

I can confirm the workaround mentioned in spring-projects/spring-boot#26630 (which originates from https://stackoverflow.com/questions/57607445/spring-actuator-jvm-metrics-not-showing-when-globalmethodsecurity-is-enabled/57908577#57908577) fixes my issue.

So until 2.5.1 is released, I have implemented this workaround.

This means there is nothing more to be done in this issue, so I will close it and I hope that whoever finds this issue also finds the mentioned workaround.

[shakuzen]

Were you able to try snapshots without a workaround? It would be good to check that everything works as expected with the fix and no workarounds.

[Davio]

I tested this with Spring Boot 2.5.1 and sadly, the problem still exists. Using the Metrics.counter static method doesn't show my metric in the prometheus endpoint when I removed the workaround.

When I re-added the workaround mentioned in the related issue:

// Workaround for https://github.com/spring-projects/spring-boot/issues/26630
@Bean
@ConditionalOnBean(name = {"meterRegistryPostProcessor", "prometheusMeterRegistry"})
public InitializingBean forcePrometheusPostProcessor(BeanPostProcessor meterRegistryPostProcessor,
                                                     PrometheusMeterRegistry registry) {
    return () -> meterRegistryPostProcessor.postProcessAfterInitialization(registry, "");
}

...it worked again.

[giuseppemilicia]

I tested this with Spring Boot 2.5.1 and sadly, the problem still exists. Using the Metrics.counter static method doesn't show my metric in the prometheus endpoint when I removed the workaround.

When I re-added the workaround mentioned in the related issue:

// Workaround for https://github.com/spring-projects/spring-boot/issues/26630
@Bean
@ConditionalOnBean(name = {"meterRegistryPostProcessor", "prometheusMeterRegistry"})
public InitializingBean forcePrometheusPostProcessor(BeanPostProcessor meterRegistryPostProcessor,
                                                     PrometheusMeterRegistry registry) {
    return () -> meterRegistryPostProcessor.postProcessAfterInitialization(registry, "");
}

...it worked again.

Same issue also with Spring Boot 2.5.3

[shakuzen]

Can you provide a minimal sample project for us to run that reproduces the issue?

[shakuzen]

Closing until we can get a minimal sample project that reproduces the issue for us to investigate further.