Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Option to send messages in bulk #65

Closed
tankorbox opened this issue Dec 14, 2018 · 4 comments
Closed

Option to send messages in bulk #65

tankorbox opened this issue Dec 14, 2018 · 4 comments

Comments

@tankorbox
Copy link

I see your idea in _messages on ChannelWrapper is data preserving. But i realized that the performance is low (in my system only ~300msgs/s rate from producer to RabbitMQ) - One message can only be sent after previous message has callback return. Any option to send message in bulk asynchorous without waiting others. Or you can suggest me some properly way to do this. Thank you so much.

@tankorbox tankorbox changed the title Option to send message in bulk Option to send messages in bulk Dec 14, 2018
@jwalton
Copy link
Owner

jwalton commented Dec 14, 2018

You can send multiple messages without waiting for the callback to return. For example:

for(const message of messages) {
    await channel.publish(...);
}

Will send each message sequentially, waiting for each to be finished, but:

for(const message of messages) {
    channel.publish(...).catch(err => console.log("Bad things happened. :(");
}

Will fire off all your messages at once, since we're not awaiting each one. You could also do something like:

await Promise.all(messages.map(message => channel.publish(...)));

Which will send a big block of messages all at once, wait for all of them to be successfully delivered, and error back if any one of them fails (although, you don't know which ones failed in this case).

@tankorbox
Copy link
Author

Thank you so much for responses. I successfully apply new implement to send multiple messages. The app's performance improved perfectly. But I have a problem. when a large number of request call sendToQueue, node app reached max heap and die. Would you mind raising any solution for this case?
And I have one question about your code implementation. Why did you call publishQueueMessage recursive. It may be more heavier than using loop??

@jwalton
Copy link
Owner

jwalton commented Dec 23, 2018

_publishQueuedMessages isn't really recursive - it calls itself from inside the Promise().then(), which is always resolved after a call to setImmediate(), so it's not as if this is going to blow the stack or anything. But, reaching max heap certainly sounds bad. :P

@jwalton
Copy link
Owner

jwalton commented May 1, 2019

Fixed by #84.

benbriadeploy pushed a commit that referenced this issue May 21, 2019
## [2.3.2](v2.3.1...v2.3.2) (2019-05-21)

### Bug Fixes

* Null delta to get semantic-release to pick up [#65](#65).  Fix [#84](#84). ([9737135](9737135))
@jwalton jwalton closed this as completed Aug 21, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants