Spring Kafka Grafana Dashboard

[Migueljfs]

Hello,

we are using Spring Kafka in some of our applications, we are monitoring them with Micrometer. I was wondering if there's any official grafana dashboard or a community made one that is used.

After enabling kafka observability in Micrometer, we are now swamped with thousands of metrics and it's really hard to create a dashboard from scratch

[gpando]

https://github.com/oslabs-beta/zurau/tree/main/grafana/dashboards
https://grafana.com/grafana/dashboards/?search=kafka

[Migueljfs]

Hey @gpando thanks for helping out!

Unfortunately those dashboards are all around monitoring Kafka itself (the cluster, brokers, zookeepers, etc).

What I was looking for was dashboards for Kafka Spring Metrics (consumer, streams, etc) for our Java Applications

Here are a couple of metrics I'm seeing:

kafka_stream_state_delete_latency_avg
kafka_consumer_request_total
kafka_producer_request_rate
kafka_producer_node_request_size_max
kafka_stream_processor_node_record_e2e_latency_max
kafka_consumer_fetch_manager_bytes_consumed_total

There are thousands of metrics like this, all related to an "application"

[garyrussell]

Those metrics are all from the kafka-clients; Spring for Apache Kafka, itself, only has a couple of timers:

spring-kafka/spring-kafka/src/test/java/org/springframework/kafka/annotation/EnableKafkaIntegrationTests.java

Lines 290 to 304 in bc27453

	assertThat(this.meterRegistry.get("spring.kafka.template")
	.tag("name", "template")
	.tag("extraTag", "bar")
	.tag("result", "success")
	.timer()
	.count())
	.isGreaterThan(0L);
	assertThat(this.meterRegistry.get("spring.kafka.listener")
	.tag("name", "bar-0")
	.tag("extraTag", "foo")
	.tag("result", "success")
	.timer()
	.count())
	.isGreaterThan(0L);

[Migueljfs]

I see, then I'm asking in the wrong place.

Thank you!

[TrueBurn]

@Migueljfs did you ever get a dashboard for the kafka spring consumer or producer?