Investigate adding streaming to /api/conversation/custom
endpoint
#702
Labels
enhancement
New feature or request
/api/conversation/custom
endpoint
#702
Motivation
At present, there is an env var
AZURE_OPENAI_STREAM
which controls streaming when using the byod endpoint, but it is not used at all when using the custom endpoint.Investigate the feasibility of implementing streamed responses on the custom endpoint. This is probably non-trivial because:
How would you feel if this feature request was implemented?
Requirements
A list of requirements to consider this feature delivered
/custom
endpointTasks
To be filled in by the engineer picking up the issue
The text was updated successfully, but these errors were encountered: