Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Investigate IO_URING support #16232

Open
espoal opened this issue Oct 10, 2022 · 2 comments
Open

Investigate IO_URING support #16232

espoal opened this issue Oct 10, 2022 · 2 comments
Labels
deno_core Changes in "deno_core" crate are needed perf performance related

Comments

@espoal
Copy link

espoal commented Oct 10, 2022

I'm opening this issue to investigate the opportunity of implementing IO_URING in Deno. If you haven't heard about it IO_URING is a new revolutionary API for doing asynchronous operations in Linux with a much higher throughput, much lower tail latencies and CPU usage.

While investigating how to implement IO_URING for node, I came up with an approach which I think can make this very simple: Instead of implementing directly liburing via NAPI, I used a thin layer to expose syscalls to a WASM module, to then do the heavy lifting in WASM. More details here

This not only greatly simplify the approach, but maximize reusability across WASM compatible runtimes. Also I think this kind of taks is a perfect fit for Rust, with its safety guarantees and speed.

Unfortunately I don't know much about Deno internals, but here maybe the Node and Deno community could come together to write a common WASM module?

@Nugine
Copy link
Contributor

Nugine commented Oct 12, 2022

Bun has io_uring support.

Rust ecosystem has both low-level and high-level io_uring support.

It may be easier to start with high-level APIs.

@littledivy littledivy added perf performance related deno_core Changes in "deno_core" crate are needed labels Oct 12, 2022
@espoal
Copy link
Author

espoal commented Oct 12, 2022

Nice. I created an awesome-iouring repo to keep track of all existing io_uring implementations and resources. Please contribute if you know about anything I'm missing.

About tokio-uring: unfortunately it doesn't support some of the nice new things io_uring implemented, like IORING_OP_URING_CMD, registering buffers or IORING_SETUP_SQPOLL (for the last two I can see them in the code but they don't seem to be used anywhere)

I feel my approach could be simpler and easier to update: my implementation is only 400 lines of code, it's easier to port and it supports already all the nice features

littledivy added a commit to littledivy/deno that referenced this issue Oct 12, 2022
littledivy added a commit to littledivy/deno that referenced this issue Oct 13, 2022
littledivy added a commit to littledivy/deno that referenced this issue Oct 13, 2022
littledivy added a commit to littledivy/deno that referenced this issue Oct 13, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
deno_core Changes in "deno_core" crate are needed perf performance related
Projects
None yet
Development

No branches or pull requests

3 participants