How to reduce the number of worker threads created by MsQuic ?

[tldQuantic]

Hi,

I saw this discussion which propose to use QUIC_EXECUTION_PROFILE_TYPE_SCAVENGER:
#1680

I used it and it reduce the number of quic_worker threads created by the RegistrationOpen function (from 16 threads to only one thread).
That's great but they are other threads elsewhere created and I wanted to know if it's possible to reduce them and how to do it.

The second function is MsQuicOpen2 which create 16 cxplat_worker threads.

The last one is the ListenStart function which call QuicLibraryOnListenerRegistered which call MsQuicRegistrationOpen which create another 16 quic_worker threads.

N.B. QuicLibraryOnListenerRegistered call MsQuicRegistrationOpen with this config:
const QUIC_REGISTRATION_CONFIG Config = {
"Stateless",
QUIC_EXECUTION_PROFILE_TYPE_INTERNAL
};

So, by default, 48 threads has been created. With QUIC_EXECUTION_PROFILE_TYPE_SCAVENGER, we reduce them to 33.
My question is, could we do something to reduce all these threads ?

Thanks.

[nibanks]

Hi @tldQuantic, this is a work in progress. For instance, #3034 changes MsQuic to share QUIC and platform work threads when you don't specify QUIC_EXECUTION_PROFILE_TYPE_MAX_THROUGHPUT. It also will allow for you to configure the number of platform threads that get created (completely in future PR). The ultimate goal is to allow the app to configure the number threads and which CPUs to run on OR to even run MsQuic from it's own threads (which might be what you really want?).

[ThusSpokeNomad]

@nibanks

run MsQuic from it's own threads

precisely this. I'm curious what's the state-of-the-art way to control MsQuic execution, potentially call it's flush/tick/update functions manually inside our own threads. for the context, I'm exploring MsQuic in real-time game networking use cases and being able to control MsQuic's thread characteristics would be really beneficial. being able to control when and where to call flush/tick/update to send and receive packets over the wire and handle all callbacks in a specific time and place feels essentials. are there any new APIs or config options to dictate over these? are they documented or if not, are there any plans towards this direction?

[nibanks]

Hi @ThusSpokeNomad. Short answer: we're not there yet. There is some work (maybe a week's) to do to be able to support this, but I haven't had the time or priority to do it. If you're interested, I could probably walk you through it and support you by reviewing any PRs, but I can't commit to much more any time soon. I really want to get here, but I just haven't had a chance.

[ThusSpokeNomad]

hey @nibanks — thank you for the quick response (no pun intended haha).

if you're open to public PRs and other contributions, I'm more than happy to evaluate MsQuic in our real-time game networking use-case to its full (in UE5/C++) and contribute back with some features, bug fixes, feedback and other things — including implementing and/or refactoring this work once I become a bit better at understanding the codebase. how that sounds to you?

If you're interested, I could probably walk you through it and support you by reviewing any PRs

that'd be fantastic kickstarter for me!

[nibanks]

We have weekly meetings on our Discord server at 1 PM eastern time / 10 AM pacific time every Thursday. I'd be happy to explain there. If you prefer, open an issue specifically for this (if one isn't already there) and I can post some stuff.

[ThusSpokeNomad]

I'll try to join the next meeting tomorrow and hopefully we can follow-up on this there. thanks again!