source nodes are added to every chunk of stream response

[shiyuangu]

Hi all, I want to retrieve the source nodes from a stream response of ContextChatEngine.chat. When I look at the code, I observe that each chunk of the stream response has the source nodes attached to it in a duplicate manner. Is it a bug or a feature intentionally designed? What's the recommended way to extract the source codes from a streaming response? Thank you!

LlamaIndexTS/packages/core/src/engines/chat/ContextChatEngine.ts

Line 89 in bcc3d0b

(r: ChatResponseChunk) => new Response(r.delta, requestMessages.nodes),

[johannesstricker]

The way I've done it now is to only extract the source nodes when processing the first chunk of the stream 🤷

[himself65]

Thanks for the feedback. This might be a bug, I'm working on to support getting all context during the Iterator

[marcusschiesser]

@shiyuangu this is how create-llama is currently handling the source nodes inside of a stream: https://github.com/run-llama/create-llama/blob/1ad25451a69e10183200dfaf4c7a8d6c8f800543/templates/types/streaming/nextjs/app/api/chat/llamaindex-stream.ts#L42

@himself65 so your idea is to remove the source nodes out of the stream and instead move them to some response context that can be read while streaming and that can contain more information? Sounds great