Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

✨ Add rest client request duration and size metrics #2217

Merged
merged 1 commit into from Mar 1, 2023
Merged

✨ Add rest client request duration and size metrics #2217

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
59 changes: 53 additions & 6 deletions pkg/metrics/client_go_adapter.go
View file
Open in desktop
Expand Up @@ -62,11 +62,44 @@ var (
Buckets: prometheus.ExponentialBuckets(0.001, 2, 10),
}, []string{"verb", "url"})
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if we should drop RequestLatency in l.41-62 (in a separate PR) given:

	// Deprecated: This metric is deprecated for removal in a future release: using the URL as a
	// dimension results in cardinality explosion for some consumers. It was deprecated upstream
	// in k8s v1.14 and hidden in v1.17 via https://github.com/kubernetes/kubernetes/pull/83836.


requestResult = prometheus.NewCounterVec(prometheus.CounterOpts{
Subsystem: RestClientSubsystem,
Name: ResultKey,
Help: "Number of HTTP requests, partitioned by status code, method, and host.",
}, []string{"code", "method", "host"})
// requestLatency is a Prometheus Histogram metric type partitioned by
// "verb", and "host" labels. It is used for the rest client latency metrics.
requestLatency = prometheus.NewHistogramVec(
prometheus.HistogramOpts{
Name: "rest_client_request_duration_seconds",
Help: "Request latency in seconds. Broken down by verb, and host.",
Buckets: []float64{0.005, 0.025, 0.1, 0.25, 0.5, 1.0, 2.0, 4.0, 8.0, 15.0, 30.0, 60.0},
},
[]string{"verb", "host"},
)

requestSize = prometheus.NewHistogramVec(
prometheus.HistogramOpts{
Name: "rest_client_request_size_bytes",
Help: "Request size in bytes. Broken down by verb and host.",
// 64 bytes to 16MB
Buckets: []float64{64, 256, 512, 1024, 4096, 16384, 65536, 262144, 1048576, 4194304, 16777216},
},
[]string{"verb", "host"},
)

responseSize = prometheus.NewHistogramVec(
prometheus.HistogramOpts{
Name: "rest_client_response_size_bytes",
Help: "Response size in bytes. Broken down by verb and host.",
// 64 bytes to 16MB
Buckets: []float64{64, 256, 512, 1024, 4096, 16384, 65536, 262144, 1048576, 4194304, 16777216},
},
[]string{"verb", "host"},
)

requestResult = prometheus.NewCounterVec(
prometheus.CounterOpts{
Name: "rest_client_requests_total",
Copy link
Member Author

@sbueringer sbueringer Mar 1, 2023 •
edited

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just aligned this to the way the metrics are defined just with name instead of subsystem + name here: https://github.com/kubernetes/component-base/blob/master/metrics/prometheus/restclient/metrics.go#L78

Help: "Number of HTTP requests, partitioned by status code, method, and host.",
},
[]string{"code", "method", "host"},
)
)

func init() {
Expand All @@ -76,11 +109,17 @@ func init() {
// registerClientMetrics sets up the client latency metrics from client-go.
func registerClientMetrics() {
// register the metrics with our registry
Registry.MustRegister(requestLatency)
Registry.MustRegister(requestSize)
Registry.MustRegister(responseSize)
Registry.MustRegister(requestResult)

// register the metrics with client-go
clientmetrics.Register(clientmetrics.RegisterOpts{
RequestResult: &resultAdapter{metric: requestResult},
RequestLatency: &LatencyAdapter{metric: requestLatency},
RequestSize: &sizeAdapter{metric: requestSize},
ResponseSize: &sizeAdapter{metric: responseSize},
RequestResult: &resultAdapter{metric: requestResult},
})
}

Expand All @@ -102,6 +141,14 @@ func (l *LatencyAdapter) Observe(_ context.Context, verb string, u url.URL, late
l.metric.WithLabelValues(verb, u.String()).Observe(latency.Seconds())
}

type sizeAdapter struct {
metric *prometheus.HistogramVec
}

func (s *sizeAdapter) Observe(ctx context.Context, verb string, host string, size float64) {
s.metric.WithLabelValues(verb, host).Observe(size)
}

type resultAdapter struct {
metric *prometheus.CounterVec
}
Expand Down