Cache

A cache hit returns oryvox.cache: "exact" or "semantic" (semantic hits also carry oryvox.similarity), oryvox.provider: "cache", oryvox.cost: 0, oryvox.saved = the full baseline cost, and the X-Oryvox-Cache: exact | semantic | miss header. Cached answers are never charged. Streaming requests can be answered from the cache too.

What is never answered from the cache

Before any lookup ORYVOX classifies the request. The response cache is skipped when:

• the request asks for live data ("today", "now", "latest", weather, prices, exchange rates, news, order status, stock or availability...) or contains tool messages,
• the request has tools, or n greater than 1,
• the request is personal ("my account", "my order"..., or personal data was found) and has no user to scope it to,
• the project policy turns the cache off, or its retention is Zero,
• the request says X-Oryvox-Cache: off or "cache": {"mode": "bypass"}.

Semantic matching additionally needs: semantic cache on in the policy, one short user question (at most ~500 tokens, at most one system message), no response_format, nothing personal. A semantic hit must also name the same things: numbers, ids, dates, e-mails, URLs, quoted strings and names must match, so "order #123" never answers "order #124".

GET /requests/{id} shows the decision in trace.cache (classification, exact / semantic result, similarity, miss_reason). A cache outage is always a miss, never a failed request.

Semantic modes

A false hit is worse than a miss, so start conservative. The project can also set an exact semantic_threshold.

How long answers are kept (TTL)

Set by content unless the project policy sets a fixed cache_ttl (which wins):

What a cached answer depends on

Every key includes the company, project, user scope (user), client scope (X-Oryvox-Cache-Scope) and the versions the answer depends on: system prompt, tools, routing and security policy, and optionally the versions you send:

{"oryvox": {"versions": {"prompt": "support-v12", "tools": "t3", "knowledge": "kb-2026-10-01", "memory": "m7"}}}

Changing any of them is a miss, so deploying a new knowledge base never serves old answers. Answers are never shared across companies or projects.

Per-request controls

await client.responses.create(
  { model: 'oryvox-auto', input, cache: { tags: ['document:456'] } },
  { cacheScope: `tenant_${tenantId}` },   // or { cacheOff: true }
);

client.responses.create(model="oryvox-auto", input=text, cache={"tags": ["document:456"]}, cache_scope=f"tenant_{tenant_id}")

Use a scope (or user) whenever answers depend on who is asking. Without one, cached answers are shared within the project.

Invalidation

POST /v1/cache/invalidate (key scope cache) removes the project's cached answers:

curl https://api.oryvox.ai/v1/cache/invalidate \
  -H "Authorization: Bearer $ORYVOX_API_KEY" -H "Content-Type: application/json" \
  -d '{"tags": ["document:456"]}'          # or {"all": true}
# {"object": "cache.invalidation", "deleted": 12}

const { deleted } = await client.cache.invalidate({ tags: ['document:456'] });

client.cache.invalidate(tags=["document:456"])   # or all=True

An answer matches when it carries any of the tags (up to 100 per call). Besides your own tags, answers are tagged automatically with project:<id>, subtype:<task subtype>, knowledge:<version> (when you send oryvox.versions.knowledge) and agent:<name> (when you send oryvox.agent). The dashboard (Core > Cache, /app/core/cache) shows hit rates and savings and can empty a project's cache.