Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Speed up SELECT DISTINCT + LIMIT queries involving designated timestamp column #4172

Open
1 task done
puzpuzpuz opened this issue Jan 29, 2024 · 0 comments · May be fixed by #4180
Open
1 task done

Speed up SELECT DISTINCT + LIMIT queries involving designated timestamp column #4172

puzpuzpuz opened this issue Jan 29, 2024 · 0 comments · May be fixed by #4180
Labels
Performance Performance improvements SQL Issues or changes relating to SQL execution

Comments

@puzpuzpuz
Copy link
Contributor

To reproduce

Let's consider the following trips table query from demo:

select distinct cab_type, vendor_id, pickup_datetime from trips order by pickup_datetime desc limit 100;

Its explain plan is the following.
Screenshot from 2024-01-29 13-41-11

Here, LimitedSizeSortedLightRecordCursorFactory is applied on top of DistinctTimeSeriesRecordCursorFactory which means that LimitedSizeSortedLightRecordCursor#buildChain() iterates through full DistinctTimeSeriesRecordCursorFactory's result set. Instead, we should stop as soon as we got 100 rows.

QuestDB version:

7.3.9

OS, in case of Docker specify Docker and the Host OS:

Linux

File System, in case of Docker specify Host File System:

ext4

Full Name:

Andrei Pechkurov

Affiliation:

QuestDB

Have you followed Linux, MacOs kernel configuration steps to increase Maximum open files and Maximum virtual memory areas limit?

  • Yes, I have

Additional context

No response

@puzpuzpuz puzpuzpuz added SQL Issues or changes relating to SQL execution Performance Performance improvements labels Jan 29, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Performance Performance improvements SQL Issues or changes relating to SQL execution
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant