# Routes

> A stable model name your application calls, backed by your own provider connections: ordered or weighted targets, if-rules and failure rules.

Source: https://infro.io/docs/routes

A route is a name your application sends as `model` — `support-agent`, `batch-summary`, `nightly-summary` — that resolves to an ordered list of your own [provider connections](https://infro.io/docs/providers) and the model each one should serve. It means the choice of vendor and model lives in configuration rather than compiled into your code.

```bash
curl https://api.infro.io/v1/chat/completions \
  -H "Authorization: Bearer $INFRO_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "support-agent",
    "messages": [{"role": "user", "content": "Where is my order?"}]
  }'
```

Nothing else about the request changes. The response shape, the streaming behaviour, the error model and the usage record are identical to naming a model directly; the [trace](https://infro.io/docs/observability/traces) additionally names the route and the connection that served it.

## Strategies

| Strategy | Behaviour | Use it for |
| --- | --- | --- |
| `priority` | Walks the enabled targets in order and uses the first that answers. | A primary with fallbacks. The commonest case, and the default. |
| `weighted` | Picks the first target at random in proportion to its weight, then falls through the rest in descending weight if it fails. A target at weight 0 is a standby that never takes a first attempt. | A canary, a gradual migration, or splitting load across two accounts. |

> **Note:** `weighted` still fails over. The weight decides which target is *tried first*, not which is the only one tried — a route that dropped the request when its weighted pick failed would be a worse reliability story than no route at all.

## Rules

A route can carry up to 20 rules in front of its targets. Each rule is a set of conditions on the request and its own list of targets: when the conditions hold, those targets serve the request instead of the route's default list. Rules are checked in order and the first match decides; a request that matches none uses the default targets.

```json
{
  "name": "Gold tier",
  "match": "all",
  "conditions": [
    { "field": "metadata", "key": "tier", "op": "in", "value": ["gold", "platinum"] },
    { "field": "input_tokens", "op": "gt", "value": 50000 }
  ],
  "targets": [{ "connection_id": "conn_…", "model": "openai/gpt-6-sol" }],
  "fallthrough": true
}
```

| Field | What it reads |
| --- | --- |
| `metadata` (with `key`) | A key in the request's `metadata` object. A numeric value compares as a number. |
| `feature`, `workflow`, `step`, `end_user_id`, `session_id` | The [attribution](https://infro.io/docs/features/attribution) values, from `metadata` or the `X-INFRO-*` headers. |
| `environment`, `project_id`, `api_key_id` | The API key that sent the request. |
| `stream`, `has_images`, `has_tools` | Whether the request streams, carries an image, or declares tools. |
| `input_tokens`, `max_tokens` | INFRO's upper-bound estimate of the prompt size, and the request's own `max_tokens`. |
| `hour_utc`, `weekday_utc` | When the request arrived: 0–23, and 1 (Monday) to 7 (Sunday). |
| `class`, `class_confidence` | The class the route's [prompt classifier](#prompt-classifier) picked, and how sure it was (0–1). Only on a route with a classifier. |

- Operators: `eq`, `neq`, `in`, `not_in`, `contains`, `starts_with`, `ends_with`, `gt`, `gte`, `lt`, `lte`, `exists`, `not_exists`. String comparisons are case-sensitive. `match: "all"` needs every condition, `"any"` one; a rule holds up to 10.
- A value the request did not send is *not set*: `exists` is false, `neq` and `not_in` are true, and every other operator is false. A request that says nothing never lands on a rule written for requests that say something.
- A matched rule uses its own targets and, optionally, its own `strategy`. With `fallthrough`, the default targets follow as further failover; without it, a matched rule whose targets cannot serve answers `503`, because the rule may exist to keep that traffic off the default targets.
- Conditions read the request's metadata, headers and size, never its prompt. The one fact taken from the prompt is `class`, which a classifier you configured produces on your own connection to a decision model's provider; the rule still sees only the class.
- The dry run in the console takes a sample request and shows which rule matches and which of its conditions held.

## Prompt classifier

A route can classify each prompt before its rules run: a decision model — Jev by default, or any other [decision model](https://infro.io/docs/features/decision-models) — picks one of the route's classes on your own connection to its provider, and rules send each class to its own targets with `class is …` conditions — small requests to a small model, hard ones to a large one. The classifier runs only when an enabled rule reads `class` or `class_confidence`. The response says what it picked in `X-INFRO-Route-Class` (`large; confidence=0.91; source=model`), and the request's row records the class; the classifier call is its own row, linked to the request.

```json
{
  "classifier": {
    "model": "typesafe/jev-1.13",
    "instructions": "How much reasoning does answering this request need?",
    "classes": [
      { "key": "small", "description": "A lookup, a greeting or a short factual answer." },
      { "key": "large", "description": "Multi-step reasoning, hard code or careful analysis." }
    ],
    "input": "last_user",
    "timeout_ms": 1500,
    "min_confidence": 0.5,
    "fallback_class": "large"
  },
  "rules": [
    {
      "name": "Hard prompts",
      "conditions": [{ "field": "class", "op": "eq", "value": "large" }],
      "targets": [{ "connection_id": "conn_…", "model": "anthropic/claude-opus-5-5" }]
    }
  ]
}
```

- Routing never waits on a broken classifier. A timeout, an error, or a confidence under `min_confidence` gives `fallback_class`; with none, the request has no class, matches no `class` rule, and the default targets serve it.
- `input`: `last_user` reads the latest user message; `conversation` reads user and assistant messages, newest kept. Images are not sent. The classifier reads the prompt after any guardrail redaction.
- A rule that tests `class` on a route without a classifier, or names a class the classifier cannot pick, is refused when the route is saved.
- Each classified prompt is one call to the classifier's decision model, on your own connection to its provider, which bills it. See [Decisions with Jev](https://infro.io/docs/features/decisions) and [Decision models](https://infro.io/docs/features/decision-models).

## Failover

- A target that errors or times out is abandoned and the next one runs — before any bytes have reached your client. A target is skipped when its connection is disabled, ejected or outside the key's environment; when its provider is one the key or the request's `routing.providers` excludes; when its key does not [cover its model](https://infro.io/docs/providers#models-each-key-covers); or when the vendor has shut its model down on that provider. A route left with nothing but shut-down targets is refused with `404 model_shut_down`, naming the vendor's replacement. A route holds up to 10 targets.
- A route can set a per-attempt timeout (1 second to 10 minutes) and up to 3 retries on the same target before moving on. Those retries run immediately, without backoff.
- Before any of that, a connection with [alternate hosts](https://infro.io/docs/providers) tries them in order when its API host cannot be reached. That is the same key and the same account, so it spends no retry and moves to no other target.
- Every attempt appears on the request's trace, with the connection that made it, the host when the connection has several, and the error that ended it. Recovery is evidenced rather than asserted.
- Once a stream has started, a failure ends that stream. INFRO will not silently restart a response you have already begun receiving; the trace shows exactly where it stopped.

### Failure rules

A route's `failover` object decides what each kind of upstream failure does. Anything you leave out keeps the default, `retry`.

| Failure | When | Actions |
| --- | --- | --- |
| `rate_limit` | The provider answered 429. | `retry` (default): the route's retries on the same target, then the next target. |
| `server_error` | The provider answered 5xx, other than 504. A `503` saying the key has no channel for the model is not one: that key cannot serve the model (below). | `failover`: the next target at once, no retries. |
| `timeout` | No answer within the timeout, or the provider answered 504 or 408. | `stop`: return this error now; no other target is tried. |
| `connection` | DNS, TLS or connection failure. |  |
| `credential` | The provider rejected the key (401/403). |  |
| `moderation` | The provider's safety system refused the request. | Each provider applies its own policy; `stop` ends the request on the first refusal. |

A provider rejection (400, 422) and a key that cannot serve the model (a 404, or a `503` saying the key has no channel for it) are not in this table: `failover` cannot change them, and the route's retries never repeat them on the same target, since the same request gets the same answer there.

- **After a 400 or 422**, the other keys on the same provider for the same model are skipped too: validation belongs to the model and the API, not to a key. The chain goes on to targets on another provider or model, because providers differ on which optional parameters they accept.
- **After a 404 or a no-channel 503**, the next target runs at once, and that key goes behind the provider's other keys for that model for 15 minutes, so the next requests start on a key that can serve it.

When every target has failed, the request ends with the first of these that applies: a moderation refusal, as that refusal; a `400` in the last provider's own words when every attempt rejected the request as invalid; a `503` with code `model_not_served`, naming each key and what its provider said, when every attempt said its key cannot serve the model (a `400` naming what each target takes, when every target refused one of the request's settings before any call); `429` with code `provider_rate_limited` and your provider's `Retry-After` when any attempt was rate-limited; `408` when every attempt timed out; otherwise `502 upstream_error`, naming each attempt.

Decision calls (decision policies, rerank, `/v1/systemone` and the checks INFRO runs for guardrails, route classifiers and evals) are the exception: a 400 or 422 from a decision model ends the request on the first attempt, since every key validates the question the same way. `POST /v1/embeddings` likewise ends on a provider's 400, 404, 413 or 422.

## Editing a route

Routes are created and edited in the console, under Routing. Targets are saved as a whole list, in one write, rather than moved one at a time. A route's meaning is the *order* of its targets, and a sequence of individual moves has intermediate states that are live — a moment where production has two primaries, or none. One write means the chain a request sees is always a chain somebody intended.

## Rules a route cannot bend

- **A route may not reach a model your [allowlist](https://infro.io/docs/org/model-allowlists) forbids.** Each target's model is checked against the allowlists that apply to the key, and a target that fails the check is skipped. A policy layer a route could step around is not a policy layer.
- **Every target on a route must serve the same modality.** A route is a name for one kind of work; a chat completion falling back to an image model is not a fallback.
- **A route slug cannot be a model name.** Slugs are lowercase letters, digits, dots, dashes and underscores with no `/`, so they never collide with a `vendor/model` id, and a slug equal to a catalog model's bare name is refused with `route_shadows_model`.
- **A route is scoped to some environments or to every one**, and the same fail-closed matching as connections applies: a route whose environments do not include the key's is invisible to that key.
- **Per-request `fallbacks` do not apply to a route.** A route's failover is its target list.

## Pinning a key to routes

An API key can be restricted to named routes. A key restricted that way cannot call a model directly either — otherwise the restriction would be one line of code away from being bypassed, which is no restriction. Both refusals are `403` with `code: "route_not_allowed"`.

## When you do not need one

A request naming a model directly already resolves over your connections, with failover between them: providers in the order you connected them, and the requests spread across the active keys of each provider, with standby keys, keys that just answered `429` and keys that just said they cannot serve the model tried last. If you have one provider and no plans to move a workload, a route adds a name and nothing else. It earns its place the first time you want to change what serves something without shipping code.