Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Improve performance of batch saveOffset #897

Closed
patriknw opened this issue Jun 17, 2022 · 2 comments · Fixed by akka/akka-persistence-r2dbc#255
Closed

Improve performance of batch saveOffset #897

patriknw opened this issue Jun 17, 2022 · 2 comments · Fixed by akka/akka-persistence-r2dbc#255
Assignees
@patriknw
Milestone
1.4.1

Comments

@patriknw
Copy link
Member

Noticed that when many (>100) offsets are stored in the same batch it can be rather slow. We already have a TODO for this:

// TODO Try Batch without bind parameters for better performance. Risk of sql injection for these parameters is low.
@patriknw patriknw self-assigned this Jun 17, 2022
patriknw referenced this issue in akka/akka-persistence-r2dbc Jun 21, 2022
@patriknw
Save offsets in Batch without bind parameters, #253
5f71270
@patriknw patriknw mentioned this issue Jun 21, 2022
Merged
patriknw referenced this issue in akka/akka-persistence-r2dbc Jun 30, 2022
@patriknw
Save offsets in Batch without bind parameters, #253
bf8a0e5
patriknw referenced this issue in akka/akka-persistence-r2dbc Jul 1, 2022
@patriknw
Save offsets in Batch without bind parameters, #253 (#255)
2ea4121 
* for better performance since the add-bind approach has round-trips to the db
* basic validation of string parameters
patriknw referenced this issue in akka/akka-persistence-r2dbc Jul 4, 2022
@patriknw
Revert "Save offsets in Batch without bind parameters, #253 (#255)"
ce8bfd3 
This reverts commit 2ea4121.
@patriknw patriknw reopened this Jul 4, 2022
@patriknw
Copy link
Member Author

patriknw commented Jul 4, 2022

The problem with .add().bind()... approach is that each bind is a round-trip with the database. Can that be fixed in the upstream driver?

@patriknw patriknw mentioned this issue Jul 4, 2022
Merged
@jrudolph
Copy link
Member

jrudolph commented Jul 5, 2022

Created pgjdbc/r2dbc-postgresql#527 to track this in the upstream driver.

johanandren referenced this issue in akka/akka-persistence-r2dbc Jul 5, 2022
@patriknw
Revert "Save offsets in Batch without bind parameters, #253 (#255)" (#…
039e400 
…259)

This reverts commit 2ea4121.
@patriknw patriknw transferred this issue from akka/akka-persistence-r2dbc May 19, 2023
patriknw added a commit that referenced this issue May 19, 2023
@patriknw
perf: More efficient batch insert of offsets, #897
f54500c 
* the r2dbc statement.add() will still result in
  one round trip to the db for each added statement
* using Postgres insert with many values instead
* grouped into batches of 20, and remaining executed as before
@patriknw patriknw mentioned this issue May 19, 2023
Merged
johanandren pushed a commit that referenced this issue May 24, 2023
@patriknw
perf: More efficient batch insert of offsets, #897 (#898)
8729bb2 
* the r2dbc statement.add() will still result in
  one round trip to the db for each added statement
* using Postgres insert with many values instead
* grouped into batches of 20, and remaining executed as before
@johanandren johanandren added this to the 1.4.1 milestone May 24, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@patriknw patriknw

Labels
None yet
Projects
None yet
Milestone
1.4.1
Development

Successfully merging a pull request may close this issue.

Save offsets in Batch without bind parameters, #253 akka/akka-persistence-r2dbc
3 participants
@jrudolph @patriknw @johanandren