Token optimization

Before a request reaches a model, ORYVOX can shrink the prompt: removing noise, trimming long tool results, ranking documents by relevance and summarising old turns. Fewer input tokens means lower cost and often better answers. Every stage is recorded in the request's trace (trace.optimization).

Modes

Mode	What it does
`conservative`	Lossless clean-up only: whitespace, duplicate content, boilerplate.
`balanced` (default)	Plus trimming of long tool results and low-relevance document chunks.
`aggressive`	Plus summarising older conversation turns and stronger document pruning.

The project sets the default; override it per request:

{
  "model": "oryvox-auto",
  "messages": [...],
  "optimization": {
    "mode": "aggressive",
    "preserve": ["INV-2026-0042", "ACME Corp"],
    "documents": [
      {"id": "policy", "title": "Refund policy", "text": "...", "preserve": true},
      {"id": "faq", "title": "FAQ", "text": "..."}
    ],
    "fields": {"search_orders": ["id", "status", "total"]}
  }
}

Options

Field	Type	Description
`mode`	string	`conservative`, `balanced` or `aggressive`.
`preserve`	string[]	Strings that must survive verbatim (ids, names, amounts). Passages containing them are never removed.
`do_not_compress`	boolean	Turn optimisation off for this request.
`documents`	array	Reference documents `{id?, title?, text, preserve?}`. ORYVOX ranks them against the question and keeps the relevant parts. `preserve: true` keeps a document whole.
`fields`	object	Per tool name, the JSON fields of its results to keep. Other fields are dropped from tool results before they are sent to the model.

When to turn it off

Use "do_not_compress": true for prompts that must be sent byte for byte: legal text, prompts you evaluate, or content that is already minimal.

Reading the results

detail = client.requests.retrieve(response.oryvox.request_id)
opt = detail.trace.optimization
print(opt["mode"], opt["original"], "->", opt["final"], "tokens")
for stage in opt["stages"]:
    print(stage)

The savings from optimisation are part of oryvox.saved.