IORING_MSG_SEND_FD

[FrankReh]

An issue to track our discussion and eventual PR to support sending a fixed fd (aka direct fd) to another uring.

[FrankReh]

We can decide how to model this functionality for this crate and even whether we only support a locked down version where the crate is responsible for all use cases or there is an open, unsafe, form too to let the user or higher level crate make their own flow.

At its core, the io_uring command allows sending a copy of the file descriptor stored in a fixed table's slot to another fixed table's slot, the fixed table of another uring. Both the source uring and destination uring have to be known, and the source table index has to be known. The destination table index can be chosen by the caller or can set to be auto selected by the destination ring.

If auto selected, the selected index is returned to the op caller as the result. It is not sent to the recipient.
The recipient is sent both a copy of the fixed file descriptor, which changes the destination's fixed file table, and the pair user_data and result, which get placed into the destination's completion queue.

Dest CQE

How to handle the CQE in the destination's completion queue is the most interesting part of this.

As it stands, this crate will simply drop the results of a CQE for a user_data value that it does not recognize. And what value should the sender be using for that dest data value?

One idea, the best I have at the moment, is to have the destination start a kind of fake operation Future. One where it doesn't create an SQE but does register a waker with the slab to get a user data value. And that user data value, the slab index that has been reserved, is reported to the caller in some fashion. Now that the user space knows what dest data value to use, then msg_send_fd can be performed.

That is some handshaking, but it would keep things safe.

One slight complication is the close of the dest ring might find a slab entry for a future that didn't get resolved and no amount of cancelling outstanding operations is going to help, on the sending side or the receiving side.
Perhaps this is a case where when the receiving future (the fake op future) is dropped, it removes the entry from the slab entirely.

[thomasbarrett]

At its core, the io_uring command allows sending a copy of the file descriptor stored in a fixed table's slot to another fixed table's slot, the fixed table of another uring. Both the source uring and destination uring have to be known, and the source table index has to be known. The destination table index can be chosen by the caller or can set to be auto selected by the destination ring.

This is really cool! I didn't know this :)

[FrankReh]

This isn't documented in the liburing man pages yet but it's been promised. There is already the function and an interesting test acting as an example in the tip of io-uring.

[Noah-Kennedy]

Does this work for IPC?

[FrankReh]

Does this work for IPC?

I don't understand the question. The feature allows sending a fixed slot descriptor to any ring that you have a legal descriptor for. If you forked and therefore share a ring descriptor between two processes, then you can send from one process to another. I think they talk about that kind of security level downgrade, where a process running as root can accept connections and then pass the connection to another process running with lesser permissions. But you need a shared ring file descriptor. So general IPC over unix domain sockets - no.

Interestingly, Josh just committed kernel patches to let a uring file descriptor be stored in a fixed slot, so if you have the first level shared, you could send uring descriptors themselves with this mechanism. But that's for an unreleased kernel version.

[Noah-Kennedy]

Ah, that answers my question actually