Gen AI: more robust chat logging

[bencodeorg]

In service of simplifying our chat logging, a couple of related changes.

Break up chat history in UI into three pieces:

• chatMessagesPast: past messages that are from a previous "session" and are only now needed from a UI perspective (ie, to show students what they did before). Added to when chatMessagesCurrent is wiped (see below).
• chatMessagesCurrent: current set of messages that we're sending to the model representing existing conversation history. Append to this when we get a response from the model, and wipe and move existing messages to chatMessagesPast when a new session is created (eg, by a change in system prompt).
• chatMessagePending: the message a user has input and is awaiting response from the server. Wiped once we receive a response from the server.

Standardizes response from Rails in all cases to be a list of messages that should be appended to the chat history in the UI.

Links

• slides with a bit of graphical description: link

[sanchitmalhotra126]

Great refactor! I know this is still WIP so some high-level thoughts on the design

• Love splitting up the chat history into clear categories. Really helps with the mental model of what's going and cleans up the code quite a bit
• Likewise, I like that the chat API has a strict contract of returning a list of messages. Makes it easier and more predictable to parse on the front end
• This will also allow us to have tighter controls on what fields can be updated via redux, vs what fields should only be updated internally in thunks. Clearer division of responsibilities between internal redux code and other UI code.
• Something I've been noodling on relatedly is potentially reorganizing the types of the content in the chat message array(s). Currently, since we started with only messages, we're kind of shoehorning other types of UI (like model updates, and later on, error messages) into the chat message format, when they are distinct UI types. This can (probably should) be it's own separate refactor, but generally I was thinking about having a top level "ChatUpdate"/"ChatItem" interface, from which different sub-types extend (messages, model updates, errors, etc). Ex:

interface ChatItem {
  type: 'message' | 'modelUpdate' | 'error' // (would it be helpful to even separate user and assistant message types here?)
  ...
}

interface ChatMessage extends ChatItem {
  type: 'message' // force all ChatMessages to have type 'message'
  text: string;
  role: Role // this can now just be 'User' & 'Assistant'
  ... // do we even still need an ID?
}

interface ModelUpdate extends ChatItem {
  type: 'modelUpdate';
  updatedField: keyof AiCustomizations;
  updatedValue: AiCustomizations[keyof AiCustomizations];
  timestamp: string;
  ...
}

interface Error extends ChatItem {
  type: 'error';
  errorMessage: string;
  ...
}

With your proposed change, we could use these types to make restrictions on what content can go where. For example, I'd imagine that chatMessagesPast could be an array of ChatItem[] but chatMessagesCurrent can only be a list of actual chat messages, so ChatMessage[]. And chatMessagePending can only be a single ChatMessage.