Fallbacks

How it works

1. The router ranks the candidate models for the request.
2. ORYVOX calls the first candidate on its healthiest provider.
3. On a provider error (timeout, 5xx, rate limit, overload), it tries the same model on another provider, then the next candidate model, up to 3 fallbacks.
4. The first success is returned. oryvox.fallbacks says how many fallbacks were used, and trace.attempts in the request log lists every attempt with its outcome and latency.

Failing providers are taken out of rotation for a short time (circuit breaker), so later requests skip them.

Errors that a different model cannot fix (invalid request, policy block, budget exceeded) are returned immediately and are not retried.

Limits

• With a fixed model id, fallbacks stay on that model (other providers only).
• With a routing profile, fallbacks may use other models that meet the same requirements (quality floor, tools, vision, JSON, region and provider policy, max_cost).
• In a stream, fallback is possible until the first token is sent. After that, a failure is reported as a mid-stream error event.

If every candidate fails, the API returns the last attempt's error: usually 503 provider_unavailable, 504 timeout, 502 provider_error or 429 rate_limited (ORYX_4291); 503 no_model_available when no model could be tried at all. trace.error in the request log names the primary error, its likely root cause and the recovery tried. The SDKs retry these statuses with backoff (see SDKs).

Billing

You pay only for the attempt that answered. Failed attempts are not billed.