Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature/Question: consume a batch of messages #132

Open
chrisgoddard opened this issue Aug 31, 2023 · 3 comments
Open

Feature/Question: consume a batch of messages #132

chrisgoddard opened this issue Aug 31, 2023 · 3 comments
Labels
enhancement New feature or request

Comments

@chrisgoddard
Copy link

I'm not sure if this is already possible, but I'm wondering if there is a way to consume a batch of messages off a topic (I'm working with Kafka).

I want to consume either 5 minutes of messages or 50,000 messages (whichever comes first) and then process that collection of messages as a batch.

Right now, each handler is executed once for each message so I don't see a straightforward way to pull multiple messages in one function.

Is there a way to do this currently? If not, do you think this is something that could be possible?

@chrisgoddard chrisgoddard added the enhancement New feature or request label Aug 31, 2023
@Lancetnik
Copy link
Owner

@chrisgoddard sorry, but in the current implementation there is no way to consume batches
But you can find this functionality in the V2, just wait 2 weeks for release

@farzbood
Copy link

farzbood commented Sep 8, 2023

It would be great to add support for Redis-Streams too.

@Lancetnik
Copy link
Owner

@chrisgoddard please, check FastStream for this feature
Tomorrow will be a great release
All your Propan code should works with FastStream correctly: just migrate @handle to @subscriber and change some imports paths

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

3 participants