Async virtual scrolling with Xilem

[raphlinus]

This is a writeup of an exploration in the xilem_tokio branch.

Main goal of exploration: improve performance, in this case scrolling jank. Root cause is loading of expensive resources (image decompression, text layout). Also pulling content from remote source (database). These are similar but not exactly the same. Huge potential gain in former case by exploiting multiple threads. Main tactic in latter case is dealing with async. Both need to be virtual (lazily materialize resources that will be visible on the screen).

Toy test case (in the spirit of 7gui): iterhash(i) is the string representation of i, hashed i times with sha256 hex. iterhash(2000) is about 1ms on fast machine. Task is to display a list of i, iterhash(i). Code is concise, show SwiftUI.

SwiftUI runs the closure on the main UI thread, blocking and single threaded.

Better: run closure in threadpool, update asynchronously. Problem is flashing. Fairly common pattern, especially when data source is remote. TODO(maybe): show SwiftUI code for this.

Ideal is hybrid. Have a timeout of ~4ms. If async closures complete in that time, behave as if synchronous. Otherwise proceed with rendering, displaying pending state (flashing). Note, even with remote data sources this can represent the cache hit case.

Include video showing the comparison. It's dramatic. I'm still doing some fine-tuning on my implementation, but it's nearly flawless up to 9999, with occasional flashes when you scroll quickly.

Implementation in Xilem

This is a serious test of UI architecture, as it crosses several levels, requiring integrating async into the platform runloop. A particular concern is scheduling when things happen - the decision to paint, actually rendering (these may be separated due to the timeout). Similar to frame pacing.

Another challenge is wiring the scroll position to the view hierarchy to decide which children get materialized. This is an edge from the widget tree back to the app logic. Fortunately Xilem handles this, the model is that most deps are data flowing down the tree and down the pipeline, but basically arbitrary edges are allowed by routing the message (using id path) to the event handler.

Implementation details. Move app logic and event dispatch to separate app task. Use channels to move view + view state to UI thread for reconciling (Widget is not Send). Async events now get routed primarily to app task. That wakes the UI thread if necessary. (3 states: start, delayed, and wokeUI).

Observations

This is pasted from the zulip thread, not sure how much will make it into the final blog.

• This experiment was a success. The final result was as good as I was hoping for, and it doesn't feel super-hacky (of course it's prototype quality code). There is essentially zero doubt in my mind at this point that Xilem is the right architecture for UI in Rust. Getting similar results in other UI toolkits will be very difficult I think, though I may be proven wrong as people rise to the challenge.

• This was the first time I used the modern Rust async ecosystem (I did some on Fuchsia 4-5 years ago, when it was much rougher). I'm impressed; it's fairly pleasant to use and looks quite efficient and powerful. The inherent complexity of what I'm trying to do feels pretty tricky, but the mechanism didn't get in my way.

• In particular, it is very possible to mix sync and async Rust code. I wasn't sure about that, and it wasn't 100% smooth, but the learning curve was shallower than I was expecting.

• I think there will be a bunch of async use cases, not really a small number of primitives. This doesn't bother me too much. I think it's very doable to put the async infrastructure in place and allow liberal access through the view trait, Cx, etc. A good amount of that infrastructure is in this experiment: ability to wake the view tree, wake the UI thread, set timeouts, etc. It's also much better than the old ext_event stuff.

• I've done a moderate amount of performance measurement as part of this exploration (looking at a bunch of profiles and system traces) but not hardcore. I suspect that performance will remain good even at very large scale. I look forward to getting into it more.

A challenge

I want to see people try this in other toolkits. I think it will be difficult, but I could be proved wrong. Ideally it will become a standard example like 7gui or TodoMVC.