Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

perf(sql): reduce query latency when querying tables under heavy writing #4500

Merged
merged 5 commits into from
May 15, 2024
Merged

perf(sql): reduce query latency when querying tables under heavy writing #4500

merged 5 commits into from
May 15, 2024

Conversation

ideoma
Copy link
Collaborator

@ideoma ideoma commented May 13, 2024

Fixes #3664

Also, re-purposes 64bit in the partition table to be used for partition compression (parquet format).

@ideoma ideoma changed the title feat(perf): reduce query latency when queried tables under heavy writing feat(sql): reduce query latency when queried tables under heavy writing May 13, 2024
@ideoma ideoma added the Performance Performance improvements label May 13, 2024
@ideoma ideoma changed the title feat(sql): reduce query latency when queried tables under heavy writing pefr(sql): reduce query latency when queried tables under heavy writing May 13, 2024
@nwoolmer nwoolmer changed the title pefr(sql): reduce query latency when queried tables under heavy writing perf(sql): reduce query latency when queried tables under heavy writing May 13, 2024
@nwoolmer nwoolmer changed the title perf(sql): reduce query latency when queried tables under heavy writing perf(sql): reduce query latency when querying tables under heavy writing May 13, 2024
@ideoma ideoma marked this pull request as ready for review May 14, 2024 11:04
@ideoma
Copy link
Collaborator Author

ideoma commented May 14, 2024

[PR Coverage check]

😍 pass : 29 / 34 (85.29%)

file detail

path covered line new line coverage
🔵 io/questdb/cairo/TxReader.java 2 4 50.00%
🔵 io/questdb/cairo/ColumnVersionReader.java 12 14 85.71%
🔵 io/questdb/cairo/TableReader.java 15 16 93.75%

@puzpuzpuz puzpuzpuz self-requested a review May 15, 2024 11:39
@bluestreak01 bluestreak01 merged commit c3cc2e4 into master May 15, 2024
24 checks passed
@bluestreak01 bluestreak01 deleted the feat-lazy-reader-partition-reload branch May 15, 2024 15:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@puzpuzpuz puzpuzpuz puzpuzpuz approved these changes

Assignees
No one assigned
Labels
Performance Performance improvements
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

feat(core): Defer column file opening on read
3 participants
@ideoma @puzpuzpuz @bluestreak01