Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

MapStore Write-Behind Batching produces too small batches #24763

Closed
lukasblu opened this issue Jun 7, 2023 · 0 comments · Fixed by #24672
Closed

MapStore Write-Behind Batching produces too small batches #24763

lukasblu opened this issue Jun 7, 2023 · 0 comments · Fixed by #24672
Labels
Type: Defect

Comments

@lukasblu
Copy link
Contributor

lukasblu commented Jun 7, 2023

Problem
Our write-behind queue is growing very large because it is not processed efficiently. We are trying to use batches of size 1000, but we see that most batches are smaller than 10 items.

Expected behavior
Since we are using coalescing write-behind queues - and therefore only the latest version of a map entry needs to be persisted, we would expect that the batches are 1000 items in size if the write-behind queue is growing large.

To Reproduce

  1. To reproduce the issue, you will need a workload that mixes store and delete operations. We use around 80% store and 20% delete operations.
  2. Configure a map which has a coalescing write-behind map store configured and uses batches of size 1000.
  3. Execute the workload and look at the MapStore.storeAll() and MapStore.deleteAll() callbacks.

I already opened a PR which contains a test that shows the problem:
#24672

Additional context
In the above mentioned PR I already implemented a fix which seems easy enough - please check, approve, and merge :-)
@vbekiaris vbekiaris mentioned this issue Jun 16, 2023
Merged
vbekiaris pushed a commit that referenced this issue Jun 22, 2023
@lukasblu
Feature: Improve batching in write-behind processor. (#24672)
a4a2070 
This PR improves the batching in the write-behind processor when used
with coalescing write-behind queues.

Without this PR and having many store and delete operations mixed in the
write-behind queue, you will end up with many small callbacks to your
MapStore implementation. This is bad for performance, because it pretty
much disables the advantages of batching.

With this PR and using write coalescing, when having many store and
delete operations mixed in the write-behind queue, you will end up with
just two callbacks to your MapStore implementation - one for all the
store operations and one for all the delete operations.

Fixes #24763
vbekiaris pushed a commit that referenced this issue Jun 22, 2023
@lukasblu
Feature: Improve batching in write-behind processor [5.3.z] (#24870)
0f16645 
This PR improves the batching in the write-behind processor when used with coalescing write-behind queues.

Without this PR and having many store and delete operations mixed in the write-behind queue, you will end up with many small callbacks to your MapStore implementation. This is bad for performance, because it pretty much disables the advantages of batching.

With this PR and using write coalescing, when having many store and delete operations mixed in the write-behind queue, you will end up with just two callbacks to your MapStore implementation - one for all the store operations and one for all the delete operations.

Fixes #24763 on 5.3.z
Backport of #24672
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Type: Defect
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Feature: Improve batching in write-behind processor. lukasblu/hazelcast
1 participant
@lukasblu