Tool calling

/chat/completions supports OpenAI-style function tools (and so does /responses, which returns calls as function_call output items). When tools are present, the router only picks models that support tool calling, and the answer is never served from the response cache.

const tools = [
  {
    type: 'function' as const,
    function: {
      name: 'get_order',
      description: 'Look up an order by id',
      parameters: { type: 'object', properties: { id: { type: 'string' } }, required: ['id'] },
    },
  },
];

const messages: any[] = [{ role: 'user', content: 'Where is order A-1042?' }];
const first = await client.chat.completions.create({ model: 'oryvox-balanced', messages, tools });

const call = first.choices[0].message.tool_calls?.[0];
if (call) {
  const result = await getOrder(JSON.parse(call.function.arguments).id);
  messages.push(first.choices[0].message);
  messages.push({ role: 'tool', tool_call_id: call.id, content: JSON.stringify(result) });
  const second = await client.chat.completions.create({ model: 'oryvox-balanced', messages, tools });
  console.log(second.choices[0].message.content);
}

first = client.chat.completions.create(model="oryvox-balanced", messages=messages, tools=tools)
call = (first.choices[0]["message"].get("tool_calls") or [None])[0]

tool_choice

"auto" (default), "none", "required", or {"type": "function", "function": {"name": "get_order"}}.

Keeping tool results small

Large tool results are the most common source of wasted tokens. Use optimization.fields to keep only the fields the model needs:

{"optimization": {"fields": {"get_order": ["id", "status", "eta"]}}}

With the project's tool pruning enabled, tool definitions that are irrelevant to the current turn may also be left out of the prompt sent to the model. See Token optimization.

Streaming

Streamed tool calls arrive as delta.tool_calls fragments with an index. The Node SDK's finalChatCompletion() and the Python SDK's get_final_completion() assemble them for you.

Caching

Requests that include tools are never served from the cache.