Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CFP: Support cilium metrics allowlist #27471

Closed
CatherineF-dev opened this issue Aug 12, 2023 · 5 comments
Closed

CFP: Support cilium metrics allowlist #27471

CatherineF-dev opened this issue Aug 12, 2023 · 5 comments
Labels
area/metrics Impacts statistics / metrics gathering, eg via Prometheus. info-completed The GH issue has received a reply from the author kind/cfp kind/feature This introduces new functionality. stale The stale bot thinks this issue is old. Add "pinned" label to prevent this from becoming stale.

Comments

@CatherineF-dev
Copy link

CatherineF-dev commented Aug 12, 2023

Cilium Feature Proposal

Thanks for taking time to make a feature proposal for Cilium! If you have usage questions, please try the slack channel and see the FAQ first.

Is your proposed feature related to a problem?
#20810

If so, please describe the problem
Some cilium metrics cardinality are very high. Current open-telemetry collector will read the whole metrics payload, which can cause open-telemetry collector OOM.

Describe the feature you'd like
Add a flag --metrics-allowlist to only expose certain cilium metrics

Include any specific requirements you need

(Optional) Describe your proposed solution

Please complete this section if you have ideas / suggestions on how to implement the feature. We strongly recommend discussing your approach with Cilium committers before spending lots of time implementing a change.

For longer proposals, you are welcome to link to an external doc (e.g. a Google doc). We have a Cilium Feature Proposal template to help you structure your proposal - if you would like to use it, please make a copy and ensure it's publicly visible, and then add the link here.

Once the CFP is close to being finalized, please add it as a PR to the design-cfps repo for final approval.

Tasks

No tasks being tracked yet.
@CatherineF-dev CatherineF-dev added the kind/feature This introduces new functionality. label Aug 12, 2023
@ti-mo ti-mo added area/metrics Impacts statistics / metrics gathering, eg via Prometheus. kind/cfp need-more-info More information is required to further debug or fix the issue. labels Aug 14, 2023
@ti-mo
Copy link
Contributor

ti-mo commented Aug 14, 2023

Hi @CatherineF-dev, thanks for the proposal! Could you elaborate on what exactly --metrics-allowlist's effect would be? I'm not sure if an agent flag is useful for this beyond specifying 2-3 metrics, or this will grow huge. (there are limits on how long arguments can be)

Could you flesh this out a bit more?

@CatherineF-dev
Copy link
Author

Explicitly list all exposed metrics in --metrics-allowlist.

For example, cilium_k8s_client_api_latency_time_seconds_bucket causes some OOMs. After updating --metrics-allowlist, this metric will not be exposed. Not only this metric, found other cilium metrics cardinality is high as well.

@github-actions github-actions bot added info-completed The GH issue has received a reply from the author and removed need-more-info More information is required to further debug or fix the issue. labels Aug 14, 2023
@CatherineF-dev
Copy link
Author

After PR prometheus/client_golang#1327, cilium will be able to use metrics-allowlist.

@github-actions
Copy link

This issue has been automatically marked as stale because it has not
had recent activity. It will be closed if no further activity occurs.

@github-actions github-actions bot added the stale The stale bot thinks this issue is old. Add "pinned" label to prevent this from becoming stale. label Oct 14, 2023
@github-actions
Copy link

This issue has not seen any activity since it was marked stale.
Closing.

@github-actions github-actions bot closed this as not planned Won't fix, can't repro, duplicate, stale Oct 28, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/metrics Impacts statistics / metrics gathering, eg via Prometheus. info-completed The GH issue has received a reply from the author kind/cfp kind/feature This introduces new functionality. stale The stale bot thinks this issue is old. Add "pinned" label to prevent this from becoming stale.
Projects
None yet
Development

No branches or pull requests

2 participants