Feeding tool result back into the messages

[msjoshlopez]

[elitan]

EDIT: It's supported here: https://sdk.vercel.ai/examples/node/tools/call-tools-with-automatic-roundtrips

OLD ANSWER:

Here's what I'm experimenting with right now:

async function main() {

  let messages: CoreMessage[] = [
    { role: "user", content: "Hi!" },
    { role: "assistant", content: "Hello, how can I help?" },
    {
      role: "user",
      content: "What's the weather in New York?",
    },
  ];

  let textResultIsPresent = false;
  let textResult = "";

  while (!textResultIsPresent) {
    await sleep(2000);
    const { text, toolCalls, toolResults } = await generateText({
      model,
      tools: {
        weather: tool({
          description: "Get the weather in a location",
          parameters: z.object({
            location: z
              .string()
              .describe("The location to get the weather for"),
          }),
          execute: async ({ location }) => ({
            location,
            temperature: 72 + Math.floor(Math.random() * 21) - 10,
          }),
        }),
      },
      messages,
    });

    if (toolResults.length > 0 && toolCalls.length > 0) {
      messages.push({
        role: "assistant" as const,
        content: toolCalls,
      });

      messages.push({
        role: "tool" as const,
        content: toolResults,
      });
    } else {
      textResult = text;
      textResultIsPresent = true;
    }

    console.log(JSON.stringify(messages, null, 2));
  }

  console.log(textResult);
}

Output:

[
  {
    "role": "user",
    "content": "Hi!"
  },
  {
    "role": "assistant",
    "content": "Hello, how can I help?"
  },
  {
    "role": "user",
    "content": "What's the weather in New York?"
  },
  {
    "role": "assistant",
    "content": [
      {
        "type": "tool-call",
        "toolCallId": "call_mlnWc8xQxqlQNOrCwVA2SJTw",
        "toolName": "weather",
        "args": {
          "location": "New York"
        }
      }
    ]
  },
  {
    "role": "tool",
    "content": [
      {
        "toolCallId": "call_mlnWc8xQxqlQNOrCwVA2SJTw",
        "toolName": "weather",
        "args": {
          "location": "New York"
        },
        "result": {
          "location": "New York",
          "temperature": 66
        }
      }
    ]
  }
]
[
  {
    "role": "user",
    "content": "Hi!"
  },
  {
    "role": "assistant",
    "content": "Hello, how can I help?"
  },
  {
    "role": "user",
    "content": "What's the weather in New York?"
  },
  {
    "role": "assistant",
    "content": [
      {
        "type": "tool-call",
        "toolCallId": "call_mlnWc8xQxqlQNOrCwVA2SJTw",
        "toolName": "weather",
        "args": {
          "location": "New York"
        }
      }
    ]
  },
  {
    "role": "tool",
    "content": [
      {
        "toolCallId": "call_mlnWc8xQxqlQNOrCwVA2SJTw",
        "toolName": "weather",
        "args": {
          "location": "New York"
        },
        "result": {
          "location": "New York",
          "temperature": 66
        }
      }
    ]
  }
]

The current temperature in New York is 66°F.

[zineanteoh]

Is there anyway to stream the final output?

The current temperature in New York is 66°F.

[benjaminalgreen]

Have been exploring how to do exactly this, thanks!

Just curious, how good/bad is the AI at being able to find the right info from previous tool responses when appropriate? E.g. Let's say you ask it to find the weather for New York. Then you ask it later how the weather in NY compares to SF. Does it know to search for SF and use the previous data from NY if prompted well?

[lgrammel]

With useChat, you can now do automatic roundtrips: https://sdk.vercel.ai/docs/ai-sdk-ui/chatbot-with-tool-calling

Server-side roundtrip support in streamText and generateText is planned

[benjaminalgreen]

@lgrammel Lars, just curious, how soon could server-side roundtrip support be coming? <3 mo?

[lgrammel]

yes less than 3 months is def realistic. it's available for generateText already: https://sdk.vercel.ai/examples/node/tools/call-tools-with-automatic-roundtrips

[benjaminalgreen]

Oh that's perfect, thank you!