Integrated Metrics

[danielrehmann]

Recently there was an issue with a OutboxProcessor not working in one of our services. Currently we don't have any metrics regarding our outbox libraries and offset in our outbox table.

I was wondering if we could introduce meter registry or something that allows us to add metrics like outbox table offset.

I have to admit I am unsure if this should be part of this library or projects need to implement it themselves so I wanted to open a discussion first.

[magro]

I definitely like the idea of instrumentation / observability improvements. Maybe we collect some metrics that would be good to have? E.g.

• num_processed_records (the number of OutboxRecord entries that have set processed)
• num_unprocessed_records (the number of OutboxRecord entries that have not set processed)

?

[danielrehmann]

I think for me the num_unprocessed_records containing the amount of outbox entities that not have been sent yet would be essential. Thats a metric that can be easily monitored and alerted on.

num_processed_records is a nice to have where we could actually monitor the throughput of our Outbox.

[magro]

I'm also curious to hear more about the issue and root cause, which lead to "a OutboxProcessor not working". What exactly means "not working", and what was the cause for that? Is there anything in this library that could be improved to prevent this situation in the future?

[hnrkdmsk]

I'm not so familiar with this topic, @ChulioZet can you take over please? 🤗

[magro]

I could reproduce the issue and just submitted a PR with the fix: #183. To make megalinter/editorconfig rules happy I had to replace tabs with spaces for the OutboxProcessor - if you want to see just the relevant change please activate "Hide whitespace" in the diff view settings, or see commit 61c393.

[ChulioZet]

Perfect, that sounds good. Then I won't open an issue or anything right now. If you need any additional info or something else, just reach out. 🙂

[magro]

If you have analyzed the issue it would still be valuable to compare the issue that I reproduced and fixed here with your analysis / observations, @ChulioZet.

[ChulioZet]

I haven't analyzed too much yet. So I'd lean towards trusting your fix for now (if the reviewing maintainers are fine with it 🙂) until we find out that the issue arises again.