Streaming

/responses

Each event has an event: and a data: line:

Event	Data
`response.created`	`{"type", "response": {"id", "object", "created_at", "model", "status": "in_progress"}}`
`response.output_text.delta`	`{"type", "delta": "text", "item_id"}`
`response.completed`	`{"type", "response": {...full response, with output_text, output (incl. function calls), usage and oryvox...}}`
`error`	`{"type": "error", "error": {"message", "type", "code", "request_id"}}`

Node.js:

const stream = client.responses.stream({ model: 'oryvox-fast', input: 'Write a limerick.' });
for await (const event of stream) {
  if (event.type === 'response.output_text.delta') process.stdout.write(event.delta);
}
const final = await stream.finalResponse();
console.log(final.oryvox.cost);
// or just: const text = await stream.finalText();

Python:

with client.responses.stream(model="oryvox-fast", input="Write a limerick.") as stream:
    for event in stream:
        if event.type == "response.output_text.delta":
            print(event.delta, end="", flush=True)
    final = stream.get_final_response()

Async Python: async with client.responses.stream(...) as stream: async for event in stream: ....

/chat/completions

OpenAI-compatible chunks (the last one has finish_reason and usage), then an ORYVOX summary, then [DONE]:

data: {"object":"chat.completion.chunk","choices":[{"index":0,"delta":{"content":"Hel"}}], ...}
data: {"object":"chat.completion.chunk","choices":[{"index":0,"delta":{"content":"lo"},"finish_reason":"stop"}], ...}
data: {"object":"oryvox.summary","oryvox":{"request_id":"req_...","selected_model":"...","provider":"...","cost":0.0001,"saved":0.0009,"cache":null,"fallbacks":0}}
data: [DONE]

OpenAI SDKs ignore the summary line safely. The ORYVOX SDKs expose it as stream.summary:

const stream = await client.chat.completions.create({ model: 'oryvox-auto', messages, stream: true });
for await (const chunk of stream) process.stdout.write(chunk.choices[0]?.delta?.content ?? '');
console.log(stream.summary?.cost);
const completion = await stream.finalChatCompletion(); // assembled content and tool calls

stream = client.chat.completions.create(model="oryvox-auto", messages=messages, stream=True)
for chunk in stream:
    print(chunk.choices[0]["delta"].get("content") or "", end="")
print(stream.summary.cost)

Errors during a stream

Errors before the stream starts are normal HTTP errors. Errors after it started arrive as data: {"error": {...}} (chat) or an error event (responses); the SDKs raise them as the matching typed error (Errors) with status unset.

Notes

Fallbacks can happen until the first token is sent (see Fallbacks).
The SDKs retry a stream only if it fails before the response starts.
Cache hits stream too: the cached answer is replayed quickly.
Streams carry X-Oryvox-Request-Id; model, provider and cost arrive in the summary / response.completed.
A streamed result is not stored for idempotent replay: repeating its Idempotency-Key answers 409 conflict (Billing).
Test keys stream simulated answers too.
Disable proxy buffering for SSE (X-Accel-Buffering: no is set by the API).