Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Using a runtime with no lifo slot. #4771

Merged
merged 1 commit into from May 16, 2024
Merged

Using a runtime with no lifo slot. #4771

merged 1 commit into from May 16, 2024

Conversation

fulmicoton
Copy link
Contributor

@fulmicoton fulmicoton commented Mar 21, 2024

As spotted by @PSeitz, the unstealable lifo-slot is causing the
doc processor and the indexer to run on the same thread.

For the moment this is disabled by default and only apply to the
indexer blocking runtime.

lifo slot disabled

➜  quickwit-dynajson cat ~/data/gharchive.json | ./quickwit index ingest --index gharchive4
❯ Ingesting documents from stdin.
⠏ [00:01:56] 1.43 GiB/1.43 GiB (5.7 MiB/s)                                     Ingested ✔ documents successfully.
➜  quickwit-dynajson
➜  quickwit-dynajson ./quickwit index create --index-config ./index.yaml

with lifo slot

➜  quickwit-dynajson cat ~/data/gharchive.json | ./quickwit index ingest --index gharchive4
❯ Ingesting documents from stdin.
⠇ [00:00:23] 459.65 MiB/459.65 MiB (17.1 MiB/s)
⠋ [00:00:23] 459.65 MiB/459.65 MiB (17.1 MiB/s)
⠏ [00:01:39] 1.43 GiB/1.43 GiB (5.7 MiB/s)                 Ingested ✔ documents successfully.

@fulmicoton fulmicoton force-pushed the no-lifo-slots branch 3 times, most recently from 353444d to e2deac0 Compare March 21, 2024 08:23
@fulmicoton fulmicoton force-pushed the no-lifo-slots branch 2 times, most recently from 18533bd to 9262418 Compare April 9, 2024 07:01
@fulmicoton fulmicoton marked this pull request as ready for review April 9, 2024 07:02
As spotted by @PSeitz, the unstealable lifo-slot is causing the
doc processor and the indexer to run on the same thread.

For the moment this is disabled by default and only apply to the
indexer blocking runtime.
@fulmicoton fulmicoton merged commit 2ceab75 into main May 16, 2024
2 of 3 checks passed
@fulmicoton fulmicoton deleted the no-lifo-slots branch May 16, 2024 02:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants