pkg/kgo: allow marking using an offset #672
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Currently marking is only available via records, expose a new function, MarkCommitOffsets, to allow marking using offsets only.
|
This looks good to me. Are you ok waiting a bit -- maybe a while? This is the only thing that necessitates a minor release right now and I'd like to bundle it with another feature or two, or maybe wait for the Kafka 3.7 release if that's coming out soon enough. FWIW it looks like 3.7 should release sooner than later: https://cwiki.apache.org/confluence/display/KAFKA/Release+Plan+3.7.0 |
|
Yes, that's okay, it'd be great if it makes it some time soon but I'm not in a rush, thank you |
|
Timeline update: I plan to work on 3.7 changes this week or weekend and then merge this into the next minor release. If there's a delay it's because I'm a bit busy at work. |
|
Sorry for the delay, the last 3mo were much busier than I anticipated. Releasing imminently! |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Currently when using
AutoCommitMarksoption the only function available for marking isMarkCommitRecords(rs ...*Record), this requires you to keepRecords in memory until they're marked (possibly for prolonged periods of time) unnecessarily since the marking itself doesn't really need anything other than the offsets.I added a new function
MarkCommitOffsetsto do the same asMarkCommitRecordsbut accept a map ofEpochOffsets instead, let me know what you think, thank you