Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make batch sizes configurable #4149

Merged
merged 22 commits into from Mar 19, 2024
Merged

Make batch sizes configurable #4149

merged 22 commits into from Mar 19, 2024

Conversation

kropidlowsky
Copy link
Contributor

@kropidlowsky kropidlowsky commented Mar 6, 2024

Description

Replaced batch sizes hard-cored values with configuratable for: sqlite, postgrsql, hana.

Closes #3708.

Readiness checklist

  • I added/updated unit tests (and they pass).
  • I added/updated integration/compatibility tests (and they pass).
  • I added/updated comments and checked rendering.
  • I made spot refactorings.
  • I updated user documentation.
  • I ran task all, and it passed.
  • I ensured that PR title is good enough for the changelog.
  • (for maintainers only) I set Reviewers (@FerretDB/core), Milestone (Next), Labels, Project and project's Sprint fields.
  • I marked all done items in this checklist.

@kropidlowsky kropidlowsky requested review from AlekSi and a team as code owners March 6, 2024 00:14
Copy link

codecov bot commented Mar 6, 2024

Codecov Report

Attention: Patch coverage is 71.05263% with 11 lines in your changes are missing coverage. Please review.

Project coverage is 73.90%. Comparing base (aea56ea) to head (0a86ac7).

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #4149      +/-   ##
==========================================
- Coverage   74.61%   73.90%   -0.72%     
==========================================
  Files         339      339              
  Lines       22862    22876      +14     
==========================================
- Hits        17059    16906     -153     
- Misses       4498     4684     +186     
+ Partials     1305     1286      -19     
Files Coverage Δ
integration/setup/listener.go 70.94% <100.00%> (-3.89%) ⬇️
integration/setup/setup.go 84.74% <ø> (ø)
internal/backends/postgresql/backend.go 71.42% <100.00%> (ø)
internal/backends/postgresql/metadata/registry.go 71.54% <100.00%> (+0.05%) ⬆️
internal/backends/sqlite/backend.go 80.39% <100.00%> (ø)
internal/backends/sqlite/metadata/registry.go 70.58% <100.00%> (+0.10%) ⬆️
internal/handler/handler.go 50.86% <ø> (ø)
internal/handler/msg_insert.go 82.69% <100.00%> (-0.17%) ⬇️
internal/handler/registry/postgresql.go 85.18% <100.00%> (+1.18%) ⬆️
internal/handler/registry/registry.go 44.44% <ø> (ø)
... and 7 more

... and 11 files with indirect coverage changes

Flag Coverage Δ
filter-true 67.38% <71.05%> (-0.81%) ⬇️
hana-1 ?
integration 67.38% <71.05%> (-0.81%) ⬇️
mongodb-1 5.13% <0.00%> (-0.02%) ⬇️
postgresql-1 46.48% <39.47%> (+0.03%) ⬆️
postgresql-2 49.56% <39.47%> (-0.10%) ⬇️
postgresql-3 49.84% <39.47%> (+0.02%) ⬆️
sqlite-1 45.68% <42.10%> (+0.03%) ⬆️
sqlite-2 48.78% <42.10%> (+0.04%) ⬆️
sqlite-3 49.00% <42.10%> (-0.02%) ⬇️
unit 32.51% <48.64%> (-0.01%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

@AlekSi AlekSi requested review from a team, henvic, chilagrow and noisersup March 6, 2024 09:50
@AlekSi AlekSi changed the title Batch sizes configurable Make batch sizes configurable Mar 6, 2024
@AlekSi AlekSi added code/enhancement Some user-visible feature could work better community Issues and PRs assigned to community members labels Mar 6, 2024
@AlekSi AlekSi added this to the Next milestone Mar 6, 2024
@AlekSi AlekSi enabled auto-merge (squash) March 6, 2024 16:36
Copy link
Member

@AlekSi AlekSi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tests fail because we don't pass BatchSize somewhere, and invalid SQL is generated for INSERT

internal/backends/postgresql/collection.go Show resolved Hide resolved
internal/backends/sqlite/collection.go Show resolved Hide resolved
auto-merge was automatically disabled March 11, 2024 23:27

Head branch was pushed to by a user without write access

cmd/ferretdb/main.go Outdated Show resolved Hide resolved
chilagrow
chilagrow previously approved these changes Mar 14, 2024
Copy link
Contributor

@chilagrow chilagrow left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for your contribution!

@rumyantseva rumyantseva enabled auto-merge (squash) March 19, 2024 10:41
rumyantseva
rumyantseva previously approved these changes Mar 19, 2024
Copy link
Member

@rumyantseva rumyantseva left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for contributing!

@AlekSi AlekSi removed this from the Next milestone Mar 19, 2024
@AlekSi AlekSi added this to the Next milestone Mar 19, 2024
@AlekSi AlekSi dismissed stale reviews from chilagrow and rumyantseva via 7aea53b March 19, 2024 13:56
@AlekSi AlekSi disabled auto-merge March 19, 2024 14:36
@AlekSi AlekSi merged commit 60fcd09 into FerretDB:main Mar 19, 2024
28 of 29 checks passed
@AlekSi
Copy link
Member

AlekSi commented Mar 19, 2024

Thank you!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
code/enhancement Some user-visible feature could work better community Issues and PRs assigned to community members
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

Make batch sizes configurable
4 participants