Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[kube-prometheus-stack] container resource requests and limits incorrectly aggregated #4453

Open
sjiekak opened this issue Apr 15, 2024 · 0 comments
Labels
bug Something isn't working

Comments

@sjiekak
Copy link

sjiekak commented Apr 15, 2024

Describe the bug a clear and concise description of what the bug is.

"Kubernetes / Compute Resources / Pod" grafana dashboard is showing incorrect container requests\limit values when the pod is scheduled on different instances

limits:
  cpu: 500m
  memory: 250Mi
requests:
  cpu: 100m
  memory: 100Mi

Is being displayed as if the values were doubled

What's your helm version?

3.14.3

What's your kubectl version?

1.26.13

Which chart?

kube-prometheus-stack

What's the chart version?

5.18.1

What happened?

displayed memory and cpu requests are equivalent to multiple of configured values

limits:
  cpu: 1
  memory:  500Mi
requests:
  cpu: 200m
  memory: 200Mi

What you expected to happen?

displayed values should match what is configured

limits:
  cpu: 500m
  memory: 250Mi
requests:
  cpu: 100m
  memory: 100Mi

How to reproduce it?

kube_pod_container_resource_requests{job="kube-state-metrics", cluster="$cluster", namespace="$namespace", pod="$pod", resource="cpu"}

give two metrics, with the same pod in different instance

kube_pod_container_resource_requests{cluster="production-worker-1", container="elasticsearch-exporter", endpoint="http", environment="prod", instance="1.1.1.1:8080", job="kube-state-metrics", namespace="sample-namespace", node="gke-cluster", pod="elasticsearch-prometheus-exporter-same-hash", prometheus="prometheus-operator/prometheus-operator-prometheus", resource="cpu", service="prometheus-operator-kube-state-metrics", unit="core"}
kube_pod_container_resource_requests{cluster="production-worker-1", container="elasticsearch-exporter", endpoint="http", environment="prod", instance="2.2.2.2:8080", job="kube-state-metrics", namespace="sample-namespace", node="gke-cluster", pod="elasticsearch-prometheus-exporter-same-hash", prometheus="prometheus-operator/prometheus-operator-prometheus", resource="cpu", service="prometheus-operator-kube-state-metrics", unit="core"}

with value 0.1

sum( kube_pod_container_resource_requests{job="kube-state-metrics", cluster="$cluster", namespace="$namespace", pod="$pod", resource="cpu"})

then returns 0.2

Enter the changed values of values.yaml?

NONE

Enter the command that you execute and failing/misfunctioning.

helm install

Anything else we need to know?

avg should be used instead of sum

@sjiekak sjiekak added the bug Something isn't working label Apr 15, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant