State, borrowing widgets, etc

[sophacles]

There's a lot of discussion in tickets about changing the render functions to borrow a widget (e.g. #122, #132, #16 ). There's also a bit of discussion about giving more access to internal widget state, and reusing widget structs. I think it makes sense to gather those discussions into one place - if nothing else, they add up to quite a bit of architectural change and a consolidated discussion might help a lot.

[sophacles]

My personal opinion is that state sharing and borrow-to-render share a similar root causes, and are inter-related. I've spent way too much time thinking about the model of this library, and I really like the concept, but there's some warts, and I don't think they are in the obvious places (borrowing, accessors). So I'll start with an explanation of the core concept that's deeper than the (pretty scant) documentation:

Widgets are just a fancy way to pass arguments to a function.

The name "Widget" and a long tradition of Object Oriented UI libraries sort of embeds the notion of widget == object into everyone's heads. This makes a lot of sense in retained mode GUIs. ratatui was designed as an immediate mode GUI though, everything is drawn on each pass. For that functions make a lot of sense - just pass in the data and parameters and get the ui is rendered. This core idea is captured in the docs too:

All widgets are implemented using the builder pattern and are consumable objects. They are not meant to be stored but used as commands to draw common figures in the UI.

Between that and the consuming render and the <'a> lifetime, I can only conclude widgets are a fancy way to pass arguments to a function. I mean this in a very real way, I think:

    let paragraph = Paragraph::new(text.clone())
        .style(Style::default().fg(Color::Gray))
        .block(create_block("Right alignment, with wrap"))
        .alignment(Alignment::Right)
        .wrap(Wrap { trim: true });
    f.render_widget(paragraph, chunks[2]);

Is just a way to write this function call, but with a bunch of benefits:

/// signature
impl Frame {
  fn render_paragraph<'a>(&mut self, area: Rect, text: Text<'a>, block: Option<Block>, style: Option<Style>, wrap: Option<Wrap>, scroll: Option<(u16, u16)>,  alignment: Option<Alignment>) {...}
}
/// call
f.render_paragraph(chunks[2], text.clone(), Some(create_block(...)), Style::default().fg(...), Wrap {trim: true}, None, None);

That's a gross function signature. In python I'd do something like:

def render_paragraph(self, rect,  text, **kw):
     buffer = self.get_buffer()
     style = kw.get("style", Style())
   ...

To allow for only setting non-default options. Rust doesn't have variadic function arguments though, every function must be called with all arguments. Having a builder struct for the arguments instead, reduces the visual clutter of long argument lists, and makes it easier to determine which parameters are set (since rust also doesn't have varname=val keyword arguments). It also means new additions to drawing parameters don't have to be breaking changes, the builder will just set the default instead of requiring users to update all their call-sites.

It's a neat pattern for simplifying complex function calls and allowing changes without a lot of hassle downstream.

**What does this have to do with borrowing and state accessors? **

Well, a lot actually. First I want to acknowledge that borrowing from a parameter struct in render doesn't change any of the benefits I outlined above. It provides the same set of benefits, and allows some ease in compound widget construction (I'll come back to that).

Rather, I think widget reuse doesn't make a lot of sense conceptually in terms of the "widgets are fancy function calls". You set your arguments up every time you call a function, and keeping that notion in the "fancy" approach keeps the pattern familiar.

There's an efficiency argument that I've seen made for the borrowed type, but in reality it's really not much different to do:

let a = 2;
for b in 1..5 {
    do_somthing(a, b);
}

Than it is to do:

for b in 1..5 {
  let args = ArgBuilder::new().a(2).b(b);
  do_something(args);
  // or do_somthing(&args)
 }

They will compile and optimize similarly. This is non-intuitive to me, it "feels" more expensive somehow, but play around with the idea in the playground and check out the assembly - they end up very similar, particularly after optimization (--release build). It's not particularly expensive to recreate simple structs by the time the compiler is done doing its thing.

Almost all of the parameters (Style, Wrap, Color, etc) are very small copy types like usize, so for them, the above is quite true, and the widgets themselves aren't very big from them as a result.

Where the above falls apart is: the buffer and the drawn data (the vec or the Text in pargarph for example) is not a necessarily a small copy type. For the Buffer, it's passed to render as &mut, and the data is owned. I think this is a problem - copying the data into each invocation of widget is unecessary (even in the Cow world of Text this results in a lot of allocs of new vecs and a lot of copying pointers around for each iteration).

This is strike 1 against owned "dawn data" types. I think it would make a lot more sense to allow reference types to the drawn data.

Lets revisit the function call, but with a slightly simplified signature:

fn render_paragraph(&mut self, text: Text<'a>, area: Rect, everything_else: Opts)

Technically it's Into<Text> but that just means a new, owned Text is constructed for you if you pass an &str or whatever. It doesn't make sense that this is needed, nowhere that the Text is used actually needs to consume it, and ultimately it iterates over characters in spans - those are copy types anyway, as is the Style that set for each cell. Changing the conceptual signature to use &Text (etc) keeps the function call concept without any major overhauls needed.

But why require &Text or &[ListItem] in the first place?

I don't know that ListItem or Text need to even be types. It dictates a lot about the rest of the program structure in uncomfortable ways. I want to represent my data in ways that fit the program not the libraries. So to work around it I have to either create a to_text() -> Text method on my struct an pull in ratatui types where they are otherwise not needed, or I have to create a function in my drawing code to slice up my data to the right type. Depending on the data this can be pretty annoying or expensive for every iteration, and even with references to "drawn data", its a little annoying to know what to do with the resulting Text struct.

This is strike 2 against the drawn data types.

I think a better solution would be to implement the passed types as traits... trait Text etc. These allow me to keep my app data in any shape I want, and provide the widget with what it actually needs (in the case of Text its an iter of Spans, but probably this recursively applies... i don't have a full design here). This allows callers to store the data however they want, do their own caching etc. For text, this also means that the app data struct can implement something like fn iter_str() -> impl Iterator<Item = &str> and the drawing function can the call .map(|s| Spans::from(s).style(some_style)), separating the logic on conceptual boundaries.

Aside: A similar rationale can be applied to ListState (et al), they can be traits that provide the relevant info via a function call or two)

This seems orthoganal to borrowed render

It is a bit askew anway. I don't think the above stuff directly contradicts having a borrowed render. I do think it strongly influenced the desire though. The owned "drawn data" types and strict type requirement both contribute to making the widget feel like some sort of object rather than a fancy function call - giving "large" data to a struct usually signals "this struct should live for a while", rather than "this is function args", I think removing that ownership will do a lot for that intuition though.

I applied this idea for the Calendar widget (DateStyler is a trait, and the reference implementation impls the trait on reference types as well as owned), and in a personal project, DateStyler is implemented on a data type that has a bunch of data (incl a date) allowing me to just pass my data to a Calendar without much hassle.

This is strike 3 - owned and dictated "drawn data" types aren't necessary and they suggest use of widgets to store state and be long lived rather than as fancy functions.

** Back to the composite widgets **

Composite widgets (widgets made of widgets) are currently pretty hard to do unless the composition is all one widget type. Heterogeneous collections are kind of a pain in rust no matter how you slice it. The most straight-forward way is Box<dyn Widget> and having the Widget trait use a borrow or mut borrow. It's probably not the only solution - there's an argument to be made for macro magic here, constructing some variants on an enum at compile time, or similar, so the machinery to do consuming render rather than borrowed render, but that's kind of gross too, as is unsafe magic that could make this possible. Maybe someone has a way around it... but really borrowing is best here.

• So about that wall of text...*

Ok so I just spent a lot of words convincing myself that maybe there's a good reason to do borrowed renders after all. I'm certainly not going to delete it - I also got to lay out my traits idea nicely, and is probably a good thing to point to in the future for various feature requests (should we continue down the immediate mode path) because I really still don't like the idea of long-lived widgets even if borrowing for render is good idea. The trait path solves for that.

• What does this have to do with accessor functions?*

I almost forgot to tie that in - accessor functions are mostly useful for getting at owned data. If we go down the trait path, and let people define data how they see fit, they're less likely to need to access that data via the widget, and futher encourage keeping state and widgets separate.

Some other random thoughts on how the using traits for drawn data could be useful:

• We can move word wrapping out of pargaraph. Having that functionality inside the paragraph widget is another big source of wanting accessors of various types - its an opaque operation so setting the cursor becomes a bit of a nightmare. Having a text trait (carefully defined) allows the user to either do their own wrapping, or pass their data object to a generic ratatui provided word wrapper that takes a T: Text and impls Text also. Such a wrapper could store info about how the lines ended up being cut up (when the render function calls self.text.get_lines(width) or whatever), and therefore make it easier to get placement info after the widget is drawn.

• There's a thousand details about the data types themselves and what is useful to someone or another. Some of these conflict, some of these are distinct but very similar, and could lead to lots of bike shedding, churn or giant method lists. Giving the user full control via traits helps avoid this.

Anyway... wall of text over. Thanks for reading my treatise on how I changed my mind on borrowed render, and also I'm curious on what folks think about the rest of the stuff I presented. Also, there's probably a lot of other takes out there, so I'm interested in how you all see this stuff too 😄

[deifactor]

You can already do that:

struct BookChapter(String);

impl<'a> Into<Text<'a>> for &'a BookChapter {
    fn into(self) -> Text<'a> {
        Text::raw(&self.0)
    }
}

fn blah() {
    let chapter = BookChapter("Hello, World".to_owned());
    Paragraph::new(&chapter);
}

and you can even impl Into<Text<'static>> for BookChapter if you want and do Paragraph::new(chapter).

[joshka]

I probably misinterpreted the original post there.

[deifactor]

Actually, looking at the implementors for From, I think From/Into are only used when there's an 'obvious' conversion, such as from a narrower numeric type to a wider one. I don't think the Into impl I mentioned would be a good idea; is the text version of a chapter its title or the full text? what width is it wrapped to? etc

[deifactor]

I think I'm coming around to traits in some cases. For example, DateStyler basically has to be a trait, because you can't style every date ever ahead of time, and making it a trait lets you add more methods and just give them default implementations.

But I don't think traits are the universal answer. Since you can only have one trait implementation for a struct, picking which implementation is effectively having the 'data' site of your application decide how to map itself on to the UI layer. For example, if I have a trait that makes me implement fn as_text(&self) -> impl Iterator<Item = &str> where each item is a line, my state now has to pick the one correct answer for where the linebreaks are when it's displayed. There's really no difference between having as_text as a trait method and adding a method on the data struct itself named as_text in that respect.

I think having a way to draw a paragraph without having to construct a vec every single time is a good idea, but I don't think we need to orient the library's entire design around it.

[joshka]

https://rust-lang.github.io/api-guidelines/naming.html#c-conv, https://rust-lang.github.io/api-guidelines/interoperability.html#c-conv-traits, and and https://rust-lang.github.io/api-guidelines/predictability.html#c-conv-specific have more guidelines for conversion traits.

[joshka]

#833 is an approach that addresses much of this problem by changing our widgets to implement Widget for &SpecficWidget (e.g. Block, Clear, Paragraph etc.). This allows you to pass these by reference to Frame::render() or to call Widget::render() on a reference to those widgets.

Most of the widgets only needed to mutate their internal state because they hold an Option<Block>, which previously had to be either consumed or cloned to be rendered. This approach is actually implemented in a few apps already (it's a solution in plain sight that we just didn't think of). It generally solves the issue of both borrowing (&T) and even mutating as we can implement Widget on &mut SpecificWidget. This allows you should you want to keep state on your own custom widgets (or to keep using StatefulWidget if you want).

The upshot of that PR is that you can now (for the implemented widgets):

• store widgets in fields / variables for reuse
• use the same widget twice in two locations
• render a widget, modify a property and render it again elsewhere
• etc.

There's still a couple of widgets still to do for this, but I think that solves most of this issue.