Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow specifying metrics to skip in promtool lint #10574

Closed
chancez opened this issue Apr 11, 2022 · 4 comments
Closed

Allow specifying metrics to skip in promtool lint #10574

chancez opened this issue Apr 11, 2022 · 4 comments

Comments

@chancez
Copy link

chancez commented Apr 11, 2022

Proposal

Use case. Why is this important?

An application may be exposing metrics that are defined by third party libraries, which the developer has no control over, so it would be useful to be able to configure promtool to skip these metrics when linting.

One example is the GoCollector metrics from prometheus/client_golang fails linting:

go_gc_heap_allocs_by_size_bytes_total non-counter metrics should not have "_total" suffix
go_gc_heap_frees_by_size_bytes_total non-counter metrics should not have "_total" suffix
go_gc_pauses_seconds_total non-counter metrics should not have "_total" suffix
@chancez
Copy link
Author

chancez commented Apr 11, 2022

Also it seems like the metrics that don't conform are new in 1.12.x. So I think it may have been part of prometheus/client_golang#955

@bwplotka
Copy link
Member

Yea, let's fix it on client_golang side, thanks for reporting!

nkinkade added a commit to m-lab/pusher that referenced this issue May 11, 2022
I was running into this with newer versions:

prometheus/prometheus#10574
nkinkade added a commit to m-lab/pusher that referenced this issue May 12, 2022
* Removes old, empty directories

Pusher did not remove old directories. Every extant directory creates an
inotify watch in pusher. Over time and/or for experiments (e.g., wehe)
that create a large amount of directories, this can create a very large
number of useless watches, which eat up memory, and eventually may run
up against fs.inotify.max_user_watches, causing other containers on the
system to fail.

This commit also fixes an apparent variable name bug in which the value
for maxFileAge was put into a variable named minFileAge.

* Removes old, empty directories when walking dir

Previously, directories were not touched. On long-running containers
directories would begin to pile up causing a large amount of useless
pusher inotify directories watches.

* Uses Go v1.18 to build Docker image, and Alpine 3.15

* Updates Go modules

* Reverts prometheus/client_golang to v1.11.0

I was running into this with newer versions:

prometheus/prometheus#10574

* Updates Travis to use Go v1.18

* Reverts Go to v1.17 (from v1.18)

The travis build was failing with something like this:

mattn/goveralls#207

... and the Docker build was failing with this error:

Step 8/10 : COPY --from=build /go/bin/pusher /
COPY failed: stat go/bin/pusher: file does not exist

* Adds a clarifying comment about removing dirs

* Simplifies dir removal logic, removes branching
@roidelapluie
Copy link
Member

@bwplotka what is the status here?

@bwplotka
Copy link
Member

bwplotka commented Jul 6, 2022

It is fixed in latest version, plus we disabled those new metrics by default.

I believe this means we can close this issue for now. We can reopen when there will be more arguments for such feature. Thanks!

@bwplotka bwplotka closed this as not planned Won't fix, can't repro, duplicate, stale Jul 6, 2022
@prometheus prometheus locked as resolved and limited conversation to collaborators Jan 2, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

3 participants