Question: What is the correct way to call asynchronous APIs on main thread from pthreads

[hedwigz]

Hello everyone
I have a multi threaded application in C that needs to access asynchronus API in Javascript. For that I use Asyncify.
The issue is that I need the api to be called from the main thread (in javascript) so for that I use MAIN_THREAD_EM_ASM and Asyncify.handleAsync
Example BAD CODE:

// can't actually be run from multiple threads at the same time - dangerous
MAIN_THREAD_EM_ASM({
    return Asyncify.handleAsync(async () => {
        await someAsyncOperationThatMustRunFromMainThread();
    });
});

The problem is that if two operations happen simultanously then Asyncify goes shenaneganes (it has a state for rewind which is ruined)
So my question is this:
What is the correct way to run an asynchronous operation from multiple pthreads that might happen in the same time, and that are all need to run on the main thread?

I believe this is the section in the docs that speaks about this problem
https://emscripten.org/docs/porting/asyncify.html#reentrancy

Thanks

[sbc100]

@tlively @RReverser ?

[tlively]

I would avoid using Asyncify at all since no Wasm code needs to be suspended to make this work.

Instead of using MAIN_THREAD_EM_ASM, I would use proxying.h[https://emscripten.org/docs/api_reference/proxying.h.html] to set up the proxying to the main thread yourself. Specifically, you should use the emscripten_proxy_sync_with_ctx or ProxyingQueue::proxySyncWithCtx functions to proxy the work to the main thread, where you should call into the async JS without asyncify and arrange it so that you call emscripten_proxy_finish when the async work is done.

As an example, you can see how this code proxies work to another thread, where it is executed asyncronously in JS.

[hedwigz]

Thank you very much sir!
I will give that a go and update here.
I also tried playing with EM_ASM + Asyncify.handleAsync + postMessage to post a message for the main thread to do some operation and report back. it's kinda cumbersome though... but still nice for experimenting/learning

[hedwigz]

OK @tlively I did as you suggested and I'd like to document it here for others who may come. I also have a few questions regarding this (also please lmk if theres a way to simplify this).
First of all, emscripten_proxy_sync_with_ctx receives em_proxying_queue and target thread pthread_t.
I had to 1. create the queue using em_proxying_queue_create() and 2. get the pthread_t of the main thread.
I don't know what is the best method to get the pthread_t of the main thread and I am also using PROXY_TO_PTHREAD so the main function runs on a worker.
What I did is in the main entry point:

int main(int argc, char* argv[])
{
	 MAIN_THREAD_EM_ASM({
		dynCall('vi', $0, [$1]);
	}, setup_main_thread_queue, app_context);
}

And the code for `setup_main_thread_queue:

void setup_main_thread_queue(app_context* app_context) {
	em_proxying_queue* queue = em_proxying_queue_create();
	// pthread_self should be the main thread since I used MAIN_THREAD_EM_ASM
	pthread_t main_thread = pthread_self();
	app_context->main_thread = main_thread;
	app_context->proxying_queue = queue;
}

Next, when I wanted to run the async JS on the main thread, I ran the following code from any of my workers:

emscripten_proxy_sync_with_ctx(app_context->proxying_queue, app_context->main_thread, (void (*)(em_proxying_ctx*, void*))myFunc, NULL);

And this is the complementary code for myFunc:

void myFunc(em_proxying_ctx* ctx, void* args) {
	EM_ASM({
		console.log("myFunc called");
		proxyFinish($0);
	}, ctx);
}

The above code calls proxyFinish which is just a C function that calls emscripten_proxy_finish:

EMSCRIPTEN_KEEPALIVE
void em_proxy_finish(em_proxying_ctx * ctx) {
	emscripten_proxy_finish(ctx);
}

May I suggest adding some documentation around Asyncify that warns against how I used it? IMHO "Asyncify" sounds like the correct tools for running async JS code and it is also quite convenient to just write:

MAIN_THREAD_EM_ASM({
    return Asyncify.handleAsync(async () => {
        await someAsyncOperationThatMustRunFromMainThread();
    });
});

but the result of the above code may be disastrous

[tlively]

This all sounds exactly right. You can simplify it a bit by getting the pthread_t for the main thread with emscripten_main_runtime_thread_id() from emscripten/threading.h.

[hedwigz]

awesome! Thank you very much @tlively 🙏

[sbc100]

@tlively what do you think about creating a variant of MAIN_THREAD_EM_ASM that expects the code to return a promise and the calling thread will block until the promise resolves on the main thread? MAIN_THREAD_EM_ASM_ASYNC?

[tlively]

That sounds like it would be useful 👍