Use casesReliability
When a provider goes down, your AI features keep answering.
Put a named route in front of the provider accounts you already have. A request that times out or fails before its response starts is retried on the next target you chose, and your client sees one answer, not an outage.
Example route. Your client sends one request; the trace keeps every attempt.
The problem
Without a control plane
- A provider incident is an incident in your product, for as long as theirs lasts.
- Fallback logic lives in every service, written differently by every team.
- A retired model is discovered when its requests start failing.
With INFRO
- Routes list priority or weighted targets across your own connections.
- Failover happens in the gateway, before your client sees an error.
- Retirements are announced ahead, and refused by name from the shutdown date.
How it works
Connect the accounts you already pay for
Add your own keys for two or more providers, or two keys and hosts for one. Nothing is resold; each provider bills you as it does today.
Name a route
List targets in priority order or by weight. Your code calls the route's name instead of a model id, so the order changes without a deploy.
Decide what a failure means
Failure rules choose per failure kind whether to retry, move to the next target or stop, and if-rules send matching requests to their own targets.
What you get
Each one is documented, and each says which plan includes it. Where no plan is named, it is on every plan.
Named routes
Priority or weighted targets over your own connections, with health checks, retries and failover.
DocsFailover before the first byte
A target that times out or returns 5xx before the response starts is retried elsewhere without your client seeing it.
DocsIf-rules and failure rules
Send requests to their own targets by metadata, feature, end user, key, project, prompt size, streaming, tools or time of day.
DocsAlternate hosts and key spreading
A connection can list more hosts for the same key; direct-model requests spread across a provider's keys, standby keys last.
DocsModel retirements
Organizations still sending a retiring model are warned; from the shutdown date it is refused before any call, naming the replacement.
DocsAvailability from your keys
The model list your own key returns is read every three hours, and a model it stops listing stops being offered.
Docs
Who it is for
- Platform teamsOne place to define how every service reaches every provider, instead of a fallback library per repository.
- On-call engineersEach failed attempt on the trace with the provider's own message, so the first question is answered before the page.
- Product ownersA provider's bad afternoon stops being your users' bad afternoon.
FAQ
Does failover switch to a model I did not choose?
No. A route moves only to the targets you listed on it, in the order or weights you set. INFRO never answers with a different model on your behalf.
What happens if a stream fails halfway?
Failover applies before the response starts. Once tokens have reached your client, the request is not replayed elsewhere; the error is returned and recorded on the trace.
Do I need accounts with several providers?
It helps, but it is not required: one provider with two keys, or one key with an alternate host, already gives a route somewhere to go. Every account stays yours and bills you directly.
The full contract is in the Routes docs, and every other question is on the FAQ.
Field guides
- Field guideAutomatic LLM failover: a production implementation guideFailover is more than retrying a 500. This guide covers which failures to move, how many times, where streaming makes recovery impossible, when a different model is acceptable, and what operators need to see.
- Field guideFailover drills: test LLM provider failure before it happensThe failover path is the least-exercised code in most AI applications, and it runs for the first time during an outage. A method for drilling it on purpose: which faults to inject, a mock upstream that injects them, and the assertions that prove recovery happened.
- Field guideSeptember 3, 2026: provider incidents and failoverOn 3 September 2026 three AI providers posted incidents within hours of each other. Their own status pages tell a more precise story than the headlines, and it is the story failover has to be designed for.
Get started
Try it on your own provider accounts.
Connect a provider, create a gateway key and change one base URL. Your providers keep billing you directly; 0% markup on model spend.