DocsFeatures
Routes
A stable model name your application calls, backed by an ordered list of your own provider connections. Priority and weighted strategies, and failover.
A route is a name your application sends as model — support-agent, batch-summary, nightly-embed — that resolves to an ordered list of your own provider connections and the model each one should serve. It means the choice of vendor and model lives in configuration rather than compiled into your code.
curl https://api.infro.io/v1/chat/completions \
-H "Authorization: Bearer $INFRO_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"model": "support-agent",
"messages": [{"role": "user", "content": "Where is my order?"}]
}'Nothing else about the request changes. The response shape, the streaming behaviour, the error model and the usage record are identical to naming a model directly; the trace additionally names the route and the connection that served it.
Strategies
| Strategy | Behaviour | Use it for |
|---|---|---|
priority | Walks the enabled targets in order and uses the first that answers. | A primary with fallbacks. The commonest case, and the default. |
weighted | Picks a target by weight, then falls through the rest if it fails. | A canary, a gradual migration, or splitting load across two accounts. |
weighted still fails over. The weight decides which target is tried first, not which is the only one tried — a route that dropped the request when its weighted pick failed would be a worse reliability story than no route at all.
Failover
- A target that errors, times out, or whose connection is ejected is skipped and the next one runs — before any bytes have reached your client.
- Every attempt appears on the request's trace, with the connection that made it and the error that ended it. Recovery is evidenced rather than asserted.
- Once a stream has started, a failure ends that stream. INFRO will not silently restart a response you have already begun receiving; the trace shows exactly where it stopped.
Editing a route
Targets are saved as a whole list, in one write, rather than moved one at a time. A route's meaning is the order of its targets, and a sequence of individual moves has intermediate states that are live — a moment where production has two primaries, or none. One write means the chain a request sees is always a chain somebody intended.
Rules a route cannot bend
- A route may not reach a model your allowlist forbids. Each target's model is checked exactly as a direct call would be. A policy layer a route could step around is not a policy layer.
- Every target on a route must serve the same modality. A route is a name for one kind of work; a chat completion falling back to an image model is not a fallback.
- A route always wins over a model of the same name. A slug that shadows a catalog id will shadow it, so avoid naming one after a model.
- A route is scoped to an environment or to none, and the same fail-closed matching as connections applies.
Pinning a key to routes
An API key can be restricted to named routes. A key restricted that way cannot call a model directly either — otherwise the restriction would be one line of code away from being bypassed, which is no restriction. Both refusals are 403 with code: "route_not_allowed".
When you do not need one
A request naming a model directly already resolves over your connections, in creation order, with failover between them. If you have one provider and no plans to move a workload, a route adds a name and nothing else. It earns its place the first time you want to change what serves something without shipping code.