feat: added new trace types for Response step (PL3-54)

[zhihil]

Fixes or implements PL3-54

Brief description. What is this change?

Added a new family of V3 traces to support the Response handler.

Implementation details. How do you make this change?

Adding new trace types to support new kinds of content in V3:

1. JSONTrace - Stores JSON content
2. VideoTrace - Stores Video content

Added new versions of existing traces to simplify the trace

1. ImageTrace - Stores an image url without the additional properties from a VisualTrace. The v3 Response handler does not have sufficient data to fill out a VisualTrace. This ImageTrace is a simpler trace that avoids a situation where the Response handler returns a VisualTrace with extraneous, useless properties with dummy values.
2. TextTrace - Stores text as Markup output. This replaces the V2 text trace, as the original text trace does not return Markup but instead, a string message along with Slate.js data. We should instead return a Markup data structure and leave it up to the user to translate Markup formatting (e.g. bolding) into a client-specific format (e.g. ** in markdown).
3. DebugTrace - Refactors the debug trace to be more flexible. Highlights include a log-level property, DebugInfoLevel, to support a future feature where users can filter out for certain debug traces from general-runtime similar to different log levels in JS console logging and a new details payload that could store any data. In particular, I've used this in the Prompt variants to output debug traces containing the documents used by the knowledge base to generate its answer.

The new trace types are nested under traces/v3 not node. Ideally, in the future, traces are not coupled 1-to-1 with node. The traces should be a related, but essentially independent data type from the nodes and the general-runtime can be thought of as an "adapter" that adapts input nodes into output traces.

NOTE: This PR was originally submitted for review but has been resubmitted because it has undergone refactors as a result of the response handler changes.

Setup information

N/A

Deployment Notes

N/A

Related PRs

• feat: implement response step handler (PL3-50) general-runtime#551
• feat: added new trace types for Response step (PL3-54) #446

Checklist

• Breaking changes have been communicated, including:
  • New required environment variables
  • Renaming of interfaces (API routes, request/response interface, etc)
• New environment variables have been deployed
• Appropriate tests have been written
  • Bug fixes are accompanied by an updated or new test
  • New features are accompanied by a new test

[trs]

Is the idea to move this out of the text trace's payload?

[zhihil]

That's correct. The new JSON variant seems to also have a sort of delay property, based on the Figma mocks.

[sonarcloud]

Kudos, SonarCloud Quality Gate passed!   

0 Bugs
0 Vulnerabilities
0 Security Hotspots
0 Code Smells

No Coverage information
0.0% Duplication

The version of Java (11.0.3) you have used to run this analysis is deprecated and we will stop accepting it soon. Please update to at least Java 17.
Read more here

[zhihil]

The data payload is subject to change due to a proposal by Tyler H to simply return Markdown rather than Markup. See slack thread here.