Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Prometheus fix inf bucket count #15287

Merged

Conversation

balintzs
Copy link
Contributor

Description:
+Inf bucket counts are handled incorrectly in both prometheusexporter and prometheusremotewriteexporter.

In prometheus this bug was taken care of by the following PR: prometheus/client_golang#1148
For prometheusremotewriteexporter we need to make the same fix and assign value of the total count to the +Inf bucket as well.

This issue caused New Relic to drop our histogram datapoints that had invalid values. It was a problem with 0.61.0 as well, but became much worse with 0.62.0. We built a custom image updating github.com/prometheus/client_golang to the SHA version (dcea97eee2b3257f34fd3203cb922eedeabb42a6) that contained our fix and the issue disappeared:
Screenshot 2022-10-19 at 12 48 49

Link to tracking Issue: #4975

Copy link
Contributor

@dashpole dashpole left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

Are you planning to fix the prometheus exporter separately?

@balintzs
Copy link
Contributor Author

Are you planning to fix the prometheus exporter separately?

This change fixed it: prometheus/client_golang#1148

Or is there something I missed?

@dashpole
Copy link
Contributor

I just noticed a change in pkg/translator/prometheusremotewrite/helper.go, but not in the prometheus exporter. Is the dependency bump all that is required for that exporter?

@dashpole dashpole added exporter/prometheus bug Something isn't working ready to merge Code review completed; ready to merge by maintainers labels Oct 20, 2022
@balintzs
Copy link
Contributor Author

I just noticed a change in pkg/translator/prometheusremotewrite/helper.go, but not in the prometheus exporter. Is the dependency bump all that is required for that exporter?

Yes, as far as I can tell, that is all that is required.

@codeboten codeboten merged commit 519a2ae into open-telemetry:main Oct 20, 2022
@balintzs balintzs deleted the prometheus-fix-inf-bucket-count branch October 26, 2022 08:40
shalper2 pushed a commit to shalper2/opentelemetry-collector-contrib that referenced this pull request Dec 6, 2022
 +Inf bucket counts are handled incorrectly in both prometheusexporter and prometheusremotewriteexporter.
@plantfansam plantfansam mentioned this pull request Jul 21, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working exporter/prometheus ready to merge Code review completed; ready to merge by maintainers
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants