How can I log each attempt in non-blocking retries?

[Casper-dss]

Hi

I want to log each attempts, as a warningss in non-blocking retries. I know I can do it by using a producerinterceptor, and then check if the topic contains "retry". Then i can use the headers from the record, to enrich my log line. But is this the correct approach? Is there a way to hook in at exception time?

/Casper

[sobychacko]

Could you check this answer given by Gary? https://stackoverflow.com/a/76811267/2070861
The answer discusses how to log each attempt in the retry. You can log each attempt as a warning in the KafkaListener. You can access the attempt header in the DltHandler method also, if you want to log when the record is sent to the DLT. How is your requirement different from what is given in this answer?

[Casper-dss]

We have configured our retry strategy through a RetryTopicConfiguration, "attaching" retries to topics through this

.includeTopics(Arrays.asList(TaskTopics.VIRKSOMHED_DATA_TASK, Topics.VIRKSOMHED_DATA_REQUEST))

What i want, is to log every time, a record is sent to the retry topic. So for every retry attempt a warning is found in the log.

[sobychacko]

If I understand correctly, rather than logging in the KafkaListener, you would instead log it when the record is sent to the retry topic, correct?

[artembilan]

Isn’t send done by KafkaTemplate? Well, you already said about ProducerInterceptor. So, that’s probably the best place to do your logic . Not sure what better could be proposed…
You can enable observability on the template and trace those spans.

[Casper-dss]

That is correct. On blocking retries a retrylistener can be used. I would like to use kind of same approach on nonblocking.

[sobychacko]

If you want to log the sending to the retry topic, as Artem mentioned above, ProducerInterceptor is your best option. Currently, the framework does not provide special abstractions for retry topic send operations. Although a bit low-level, ProducerInterceptor gets the job done.

[Casper-dss]

I am missing RetryListener option on the RetryTopicConfiguration. The old RetryTemplate had it.

[sobychacko]

We will explore whether that is a plausible approach. The async nature of the non-blocking retries may make it difficult, but it may be possible.

[Casper-dss]

I will go for the solution, with the ProducerInterceptor.