Skip to content

Spring Web RestTemplate http.client.requests Metrics No Longer Have client.name Tag #29839

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
mchin7 opened this issue Jan 18, 2023 · 3 comments
Assignees
Labels
in: web Issues in web modules (web, webmvc, webflux, websocket) theme: observability An issue related to observability and tracing type: enhancement A general enhancement
Milestone

Comments

@mchin7
Copy link

mchin7 commented Jan 18, 2023

Affects: 6.0.3


Problem Description

Ever since Spring Framework 6, the client.name tag from http.client.requests metrics are missing.

Steps to Reproduce

  1. Create a Spring Boot 3.0 application (which comes with Spring Framework 6)
  2. Add spring-boot-starter-actuator and spring-boot-starter-web dependencies
  3. Run a task that periodically send requests to external endpoint using RestTemplate
  4. Visit /actuator/metrics/http.client.requests endpoint and observe that the client.name tag is missing

Sanity Check

  1. Create a Spring Boot 2.7.6 application (which comes with Spring Framework 5)
  2. Add spring-boot-starter-actuator and spring-boot-starter-web dependencies
  3. Run a task that periodically send requests to external endpoint using RestTemplate
  4. Visit /actuator/metrics/http.client.requests endpoint and observe that the client.name tag is present

I've created a repository to help in reproducing the issue. Check out https://github.com/mchin7/actuator-missing-tag-demo

Sorry if this is issue is not a Spring Framework issue, any redirection to the correct project is highly appreciated!

@spring-projects-issues spring-projects-issues added the status: waiting-for-triage An issue we've not yet triaged or decided on label Jan 18, 2023
@bclozel bclozel self-assigned this Jan 18, 2023
@bclozel
Copy link
Member

bclozel commented Jan 18, 2023

The "client.name" is contributed by the client observation as a high cardinality KeyValue, as we can't guarantee that the value is bounded for the application. As a result, this metadata ships with traces, but not metrics right now.

We consider moving it back to a low cardinality KeyValue if we believe that it's highly unlikely that an application queries enough different hosts to make the metrics system fail. As far as I remember, we did not configure a MeterFilter for this tag in Spring Boot 2.x to avoid cardinality explosion.

Changing it back to a low cardinality would not break applications as the value would be still present in traces, but would add it back into metrics. WDYT @marcingrzejszczak ?

@bclozel bclozel added the theme: observability An issue related to observability and tracing label Jan 18, 2023
@jonatan-ivanov
Copy link
Member

FWIW, I think this should be a low cardinality key-value normally. Those users who has apps with lots of http clients in them (still low cardinality) but want to lower the number of time series they have, they can add a MeterFilter.

@bclozel bclozel added type: enhancement A general enhancement in: web Issues in web modules (web, webmvc, webflux, websocket) and removed status: waiting-for-triage An issue we've not yet triaged or decided on labels Jan 18, 2023
@bclozel bclozel added this to the 6.0.5 milestone Jan 18, 2023
@mchin7
Copy link
Author

mchin7 commented Jan 19, 2023

Thank you so much for the prompt response and the detailed explanation on the issue.

For folks facing the same issue and are unable to revert to earlier version, a workaround is to simply overrides the DefaultClientRequestObservationConvention#getLowCardinalityKeyValues to also return the client.name tag by calling clientName(context) method.

Then, simply register it as a bean in Spring context and it should be picked up.

See example here mchin7/actuator-missing-tag-demo@f0414a4

izeye added a commit to izeye/spring-framework that referenced this issue Jan 28, 2023

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
bclozel pushed a commit that referenced this issue Jan 30, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
in: web Issues in web modules (web, webmvc, webflux, websocket) theme: observability An issue related to observability and tracing type: enhancement A general enhancement
Projects
None yet
Development

No branches or pull requests

4 participants