Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ability to resize the async main()'s thread pool from within #6420

Closed
Pr0methean opened this issue Mar 21, 2024 · 2 comments
Closed

Ability to resize the async main()'s thread pool from within #6420

Pr0methean opened this issue Mar 21, 2024 · 2 comments
Labels
A-tokio Area: The main tokio crate C-feature-request Category: A feature request. M-runtime Module: tokio/runtime

Comments

@Pr0methean
Copy link

Pr0methean commented Mar 21, 2024

Is your feature request related to a problem? Please describe.
On AWS's m7g.16xlarge instances, the detected number of available CPU cores is sometimes 63 rather than 64, which is the actual number of VCPUs available. This slows down sufficiently parallel work.

Describe the solution you'd like
A way for a #[tokio::main] async fn main() method to update the desired number of threads in the pool where it was running. The desired and actual numbers of threads would be stored in atomics with at least Ordering::Acquire for increases and Ordering::Release for decreases (and I'm not 100% sure even that is enough; it's possible Ordering::SeqCst would be needed throughout). Whenever a thread was between tasks and the actual number of threads exceeded the desired number, that thread would terminate. Whenever the desired number of threads was less than the actual number, all the existing threads were busy, and a new task was submitted, that task would run right away on a new thread. (In other words, the actual number of threads would be able to exceed the desired number, but not for longer than m + n + O(1) where m and n were the top two task durations.)

Describe alternatives you've considered
When I was using Rayon rather than Tokio, I could call the following from the sync main():

    let mut global_builder = ThreadPoolBuilder::new().use_current_thread();
    match available_parallelism() {
        Ok(parallelism) => {
            let adjusted_parallelism = parallelism.get() + 1;
            if adjusted_parallelism.count_ones() <= 1 {
                warn!("Adjusting CPU count from {} to {}", parallelism, adjusted_parallelism);
                // Compensate for missed CPU core on m7g.16xlarge
                global_builder = global_builder.num_threads(adjusted_parallelism);
            } else {
                info!("Rayon thread pool has {} threads", parallelism);
            }
        }
        Err(e) => warn!("Unable to get available parallelism: {}", e)
    }
    global_builder.build_global()?;

Additional context
Add any other context or screenshots about the feature request here.

@Pr0methean Pr0methean added A-tokio Area: The main tokio crate C-feature-request Category: A feature request. labels Mar 21, 2024
@Pr0methean Pr0methean changed the title Ability to resize the default thread pool from inside async main() Ability to resize the async main()'s thread pool from within Mar 21, 2024
@maminrayej
Copy link
Member

I think something like this can be implemented by the user after #6341 is implemented.

@Darksonn Darksonn added the M-runtime Module: tokio/runtime label Mar 21, 2024
@Darksonn
Copy link
Contributor

Closing as duplicate of #6341.

If you just need to set this in the beginning of main, then you may want to explicitly use the runtime builder rather than the #[tokio::main] macro and set the right number from the beginning.

@Darksonn Darksonn closed this as not planned Won't fix, can't repro, duplicate, stale Mar 21, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-tokio Area: The main tokio crate C-feature-request Category: A feature request. M-runtime Module: tokio/runtime
Projects
None yet
Development

No branches or pull requests

3 participants