Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Simplify the iterator adaptive splitting strategy #857

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Simplify the iterator adaptive splitting strategy #857

wants to merge 1 commit into from

Commits on Feb 9, 2024

  1. Simplify the iterator adaptive splitting strategy 

    Before, when an iterator job was stolen, we would reset the split count
all the way back to `current_num_threads` to adaptively split jobs more
aggressively when threads seem to need more work. This ends up splitting
a lot farther than a lot of people expect, especially in the tail end of
a computation when threads are fighting over what's left. Excess
splitting can also be harmful for things like `fold` or `map_with` that
want to share state as much as possible.

We can get a much lazier "adaptive" effect by just not updating the
split count when we split a stolen job, effectively giving it only _one_
extra boost of splitting.
    @cuviper
    cuviper committed Feb 9, 2024
    Configuration menu
    Copy the full SHA
    34ed130 View commit details
    Browse the repository at this point in the history