DocsFeatures
Smart routing
Route AI requests over your own provider connections using priority or weighted targets, environment boundaries, retries, and automatic failover.
INFRO is in private early access. Running these examples requires an early-access account, an active trial or subscription entitlement, and your own provider connection. Self-serve purchasing is not generally open. See Early access.
INFRO routes requests over the provider accounts connected to your organization. Naming a model directly resolves to eligible connections: providers in the order you connected them, with requests spread across the keys of each provider (a key marked standby takes over only when the others fail). For explicit selection, ordering and per-request conditions, configure a named route and pass its slug as model.
Priority and weighted routes
| Strategy | Behavior |
|---|---|
priority | Try enabled targets in configured order. |
weighted | Choose the first target by weight, then try the remaining eligible targets if it fails. |
Targets name your own connections and the models they serve. Disabled, ejected, or environment-incompatible connections are excluded. Model allowlists and key route restrictions still apply.
{
"model": "support-agent",
"messages": [
{
"role": "user",
"content": "Where is my order?"
}
]
}Retries and failover
A route can retry the same target 0–3 times (its max_retries) before moving on; retries run immediately, with no backoff. A route may also set a per-attempt timeout. Any upstream failure before the response starts — including a provider 4xx, 429, 5xx or timeout — moves to the next configured target. INFRO's own refusals (authentication, budget, allowlist) are decided before any provider is called and never fail over. A route's failure rules can change that per kind of failure (rate limit, server error, timeout, connection, rejected key): retry, move to the next target at once, or stop. A connection with alternate hosts tries them first when its API host cannot be reached. Once a stream has started, INFRO does not restart it on a different target. Use request traces to inspect attempts and handle a partial-stream failure in your application. See Routes and Fallbacks.
Request routing fields
The API accepts legacy routing.policy values (cheapest, fastest, balanced) and routing.regions in its request schema. routing.policy has no effect. routing.regions is recorded on the request and refused with a 400 if it conflicts with a region policy stored on your organization, but it does not rank or region-constrain customer-owned connections. Use named routes to choose connection order or weights. Neither field is a lowest-price guarantee or a data-residency control.
routing.providers chooses which of your providers may serve this request, by provider id: allow keeps only those, deny removes those. It narrows what your API key allows and never widens it. Order and failover stay with your connections and routes.
Cost and location
Your providers bill your accounts at your own rates. INFRO estimates and attributes that spend; it does not buy or resell the inference. Configure provider deployments according to your own location requirements. INFRO does not currently guarantee regional processing; see EU compliance.