Means to make actor calls from another thread

[uazu]

The basic idea is to be able to do something like remote_call!([cx, actor], method(args));. However cx would have to be some kind of inter-thread object that forwards calls to the queue on the Stakker thread. Also actor can't be an Actor reference because that's not Send. So it requires something like a RemoteDeferrer and a RemoteActor. A RemoteDeferrer could be something like an Arc<Mutex<(Waker, Vec)>>. To avoid too much contention on a single mutex, it may be preferable to allow several underlying queues to be created, e.g. to create a different one for each thread it is sent to.

An alternative if only a few distinct calls are to be made would be to have a RemoteFwd, which is created from a Fwd and allows data to be forwarded to the Stakker thread. This would be something like an inter-thread stream with the destination fixed at creation time.

There is a question of how to handle failure. If the destination goes away (e.g. destination actor fails), then would the sender in another thread need to know about that? i.e. would they want to be notified to allow them to cleanup and terminate. The PipedThread implementation handles this case, meaning that if the actor holding the reference to the PipedThread dies, then the thread will also be notified to allow it to clean up and terminate. So probably RemoteActor and RemoteFwd should have a means to query whether the destination has gone away. Perhaps even make it hard for the caller NOT to notice, to force it to handle this case.

There is also the question on how this fits into future support for remote calls from further away than other threads, e.g. remote calls across machines. However it may be that remote calls from other machines will be handled with proxy actors, in which case it will be a completely different API and unrelated to this one.

Please add comments here if you have a requirement for something like this and what kind of solution would be preferable. (See also #23)

[joseluis]

Ok I've been thinking a little bit more about this, so I'm gonna share some thoughts in case it helps in moving forward.

What I'd like the most, I think, is to have a simple unified API, with a few interrelated traits, at least one for each type at each side of one bridge, maybe one for the message type that comes through. The core issue may very well be reduced to: Messages comming from one stakker system must be dealt with by another stakker system, back and forth. The rest would be implementation details. What are the guarantees stakker wants to give and what's the minimum stakker needs to ask from the implementor? That's what needs to be clarified in the API I believe.

Also a few basic implementations should be offered, one for inter-thread communication between stakker instances, and maybe one for same-thread communication that could be handy as a base reference for benchmarks. More complex implementations could be added, and custom ones.

I also think theres gonna be a need for an error type. At the very least the api method for referring messages should return a Result with an error that probably should be as generic as possible.

In the case of stakker maybe it's worth considering creating a crate level non_exhaustive enum that includes the inter-stakker communication errors of its concrete implementations?