Custom Outbox Implementation #2647
Replies: 3 comments 1 reply
-
Hi @zhuo-yan-khoh we would be very supportive of you doing that and contributing back to us. We like to have components that have been battle-tested in production. The key complexity is that to provide transactional messaging you need a transaction between the document so you will probably want to use MongoDb multi-document transactions to support that. That lets you poll your outbox document for outstanding work. (There are alternatives: using a transaction log to build the outbox for example). Our DynamoDb support is probably the closest we currently have to MongoDb. We have started to have a model that splits Paramore.Brighter.[db] from Paramore.Brighter.[box] so that if you want to support an Inbox as well as an Outbox, common functionality can live in one place. |
Beta Was this translation helpful? Give feedback.
-
@preardon fyi |
Beta Was this translation helpful? Give feedback.
-
We use transactions for DynamoDb although they are less efficient, because it is a simpler programming model. The alternative is usually to listen to the transaction log, and turn those transaction log entries into messages in the outbox, although you require the relevant context. Our model doesn't support that so well because it tends to assume a spanning transaction is available. IIRC from Jimmy's presentation he used that model because he lacked transactions to work around it.
On Wednesday, 17 May 2023 at 18:15:53 BST, Dan Lewis ***@***.***> wrote:
I have recently joined a project where the architecture is event-driven services + mongodb and have also been looking into the potential need for implementing an outbox pattern for this. Unfortunately, I don't have much MongoDB experience so I'm ramping up on that as well!
The potential libraries that sprang to mind to check were Brighter and MassTransit, and I'd been meaning to try Brighter out for a while now which has brought me to this issue!
In terms of implementation, I remembered @jimmybogard 's article series from a few years back Life Beyond Distributed Transactions: An Apostate's Implementation. In Jimmy's example, he stores the outbox directly on the documents themself. I wonder if that's because multi-document transactions were still relatively new when the blog series was created?
Can you see any advantage for one over the other, obvious one for me is not "polluting" the main document with the event data?
I also believe MassTransit's implementation uses a separate collection for the outbox documents in mongo.
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you were mentioned.Message ID: ***@***.***>
|
Beta Was this translation helpful? Give feedback.
-
Hi @iancooper, I want to create custom outbox using mongodb/documentdb as outbox. What do I need to pay attention to when creating a custom outbox? Or how should I provide outbox implementation so that it works seamlessly in Brighter context.
Beta Was this translation helpful? Give feedback.
All reactions