# What uptime can a fallback chain promise? A composite SLO method

URL: https://infro.io/blog/what-uptime-can-a-fallback-chain-promise-a-composite-slo-method
Author: INFRO Editorial (https://infro.io/blog/editorial)
Publisher: INFRO
Published: 2026-11-15
Last modified: 2026-11-15
Last reviewed: 2026-11-15 (next review due 2026-12-15)
Topic: Routing and reliability
Format: Methodology

> Compute the chain's availability from measured per-provider failure rates, add a common-mode term for failures the providers share, multiply by the gateway and client terms in series, and commit below the result. Provider SLAs define availability narrowly and pay only a bill reduction, so they are inputs, not the target. [^aws-bedrock-sla][^vertex-sla]

Key takeaways:
- A provider SLA is a narrow, region- or project-scoped definition with a bill-reduction remedy; it is an input to your SLO, never the SLO itself.
- Chain availability is 1 minus the product of failure rates only if failures are independent; add a common-mode term estimated from your own history, and it will usually dominate.
- Multiply the chain by the terms in series: your gateway, client timeouts, failures after the first token, and refusals such as budgets and quotas.
- Count 429s, timeouts and cut-off streams as bad in your SLI even when provider SLAs exclude them.
- Commit to a number below the computed one, express it as an error budget in minutes and requests, and re-estimate every quarter.

## Why provider SLAs do not add up to your SLO

The tempting calculation is to take two providers at 99.9% each and write down 99.9999% for the chain. It fails for three reasons: the SLAs measure something narrower than your users do, their remedy is not an availability guarantee, and the failures are not independent.

Read what the definitions actually count. Amazon's Bedrock SLA computes availability per 5-minute interval as the share of requests that do not fail with an Error, and defines an Error as a request that returns a 500 error code; an interval with no requests is assumed 100% available.[1] A 429, a 503, a timeout or a stream that stops halfway is not in that definition. It is also scoped per AWS region and per account, and it excludes, among other things, problems caused by your own configuration and underlying software that causes repeated model crashes.[1] Google's Vertex AI SLA defines Downtime as more than a 5% error rate over a minute, counting only HTTP 500 and 503 responses, measured server side, and it excludes errors that result from quotas.[2] The covered-service table on that page lists training, deployment, batch prediction, AutoML and custom model online prediction; check whether the endpoint you call is a covered service before you use any figure from it.[2]

The remedy is also not uptime. Bedrock's tiers are 10%, 25% and 100% of the affected region's charges for the month, applied against future payments and claimed through a support case with your own request logs.[1] Vertex's tiers are 10%, 25% and 50%, capped at 50% of the month's amount for the covered service, and you must request them within 30 days.[2] Neither pays for your outage. So treat the SLA as a statement about what the provider will measure, and build your target from what *you* measure.

## Define the SLI the user would define

An SLO needs a service-level indicator first. For an AI feature, write it from the caller's side:

- **Good event:** the request returned a usable response within your latency threshold. Include a time-to-first-token or total-time bound, because a 90-second answer to a chat turn is down for the user.
- **Bad events:** any 5xx, any timeout, a 429 that survived the whole chain, a stream cut off after the first token, and a refusal that stops the request (budget, allowlist, guardrail).
- **Not counted:** requests your own validation rejects as malformed, which are a bug in the caller, not an outage.

Counting 429s and cut-off streams is the important departure from the provider SLAs above. A capacity shortage that returns 429 is invisible to a 500-only definition and fully visible to your user.

It also matters where failover can act. INFRO's documentation states that both connection failover and model fallbacks operate strictly before the first token, and that every upstream failure before the first token (5xx, timeout, 429, rejected credential, even a 400 or 422) moves on to the next connection.[3] Failures after the first token are outside what a chain can rescue, so they form their own term in the calculation. See [streaming failure modes](https://infro.io/blog/streaming-llm-failure-modes) for how often that term matters for your traffic.

Gateway refusals are a separate class. INFRO's own refusals, such as an invalid body, a budget, an allowlist or an unknown model, are decided before any provider is called and are not failed over.[3] Decide up front which of them you count against the SLO. A budget refusal is working as designed, but a user who is refused still did not get an answer; many teams exclude it from the SLI and track it as a separate rate.

## The chain term: independent first, then correlated

Take a feature with two targets, A then B. Measure each one's per-attempt failure rate over a long window, using your own request records: `u_A` and `u_B`. If failures were independent, a request fails only when both fail:

`U_indep = u_A × u_B`

That number is a ceiling, not an estimate. To make it honest, add a common-mode term using a beta factor `β`: the fraction of the more reliable provider's failures that happen together with the other's.

`U_chain = (1 − β) × u_A × u_B + β × min(u_A, u_B)`

Estimate `β` from history, not from hope. Using the Vertex definition as a convenient yardstick, mark a minute as "down" for a provider when its error rate in your traffic exceeds 5%. Then `β ≈ (minutes where both are down) / (minutes where the more reliable one is down)`. It is a modeling choice, not a standard, and with few incidents it is noisy, so keep the raw counts beside the estimate.

> **Example: Worked numbers (illustrative, not measured)**
> Provider A fails 0.5% of attempts over the window and provider B fails 0.7%.
> 
> - Independent: 0.005 × 0.007 = 0.0035% → chain availability 99.9965%.
> - β = 0.1: 0.9 × 0.0035% + 0.1 × 0.5% = 0.0532% → 99.947%.
> - β = 0.3: 0.7 × 0.0035% + 0.3 × 0.5% = 0.1525% → 99.848%.
> 
> The product of the two failure rates is almost irrelevant. The common-mode term decides the answer, which is why the independent figure should never be quoted.

```python
def chain_unavailability(u_a: float, u_b: float, beta: float) -> float:
    """Two-target chain; beta = share of the smaller failure rate that is common-mode."""
    return (1 - beta) * u_a * u_b + beta * min(u_a, u_b)

def composite(terms: list[float]) -> float:
    """Series terms (availabilities): gateway, chain, post-first-token, ..."""
    out = 1.0
    for a in terms:
        out *= a
    return out

chain = 1 - chain_unavailability(0.005, 0.007, beta=0.1)   # illustrative inputs
gateway = 0.999        # placeholder: use your own gateway's measured availability
mid_stream = 0.999     # placeholder: measured share of streams cut after first token
print(f"{composite([gateway, chain, mid_stream]):.5f}")    # ~0.99747
```

For three or more targets, extend the same idea: the chain fails when all targets fail, and the common-mode term is the fraction of time that *all* of them are down together, which you can count directly from minute-level history instead of modeling.

## Series terms: everything the chain sits inside

The chain is one block in a series. Multiply its availability by every component that every request must pass through:

1. **The gateway or proxy** in front of the providers. If you run INFRO, its published position is a 99.9% target per calendar month on Business and enterprise agreements, measured by its own minute checks, where a failed check or more than two minutes with no check counts as downtime; on a published plan it is a target and brings no refund.[4] The status page is explicit that this covers the gateway, not your provider accounts, and that the record is kept inside the vendor's own infrastructure.[4] Use the gateway's measured figure, not the target, once you have some history, and note the no-refund caveat: it is a number to plan with, not insurance.
2. **Your own client and network path**, including DNS, egress and timeouts.
3. **Post-first-token failures**, from the previous section.
4. **Anything that refuses before a provider is called**: spend caps, key controls, allowlists, guardrails. [INFRO's error reference](https://infro.io/docs/api/errors) lists a `gateway_unavailable` 503 among the codes that surface only when no connection can serve the request.[5]

In the worked example, a 99.9% gateway, the 99.947% chain and a 99.9% post-first-token term multiply to about 99.75%. The gateway term is larger than the whole chain's failure budget. Adding a third provider does almost nothing to that number; hardening the layers in front of the chain does far more. Whether the application should fail open to a direct provider call when the gateway is down is a separate design choice, covered in [gateway down: fail open or fail closed](https://infro.io/blog/gateway-down-should-ai-traffic-fail-open-or-fail-closed).

## Correlated failures: where β comes from

The common-mode term is not abstract. These are the places failures line up:

- **A shared cloud or region.** The same model family may be offered by its maker and by a hyperscaler, and two hyperscalers may both be fronted by the same CDN, identity service or DNS. SLAs are scoped per region or per project,[1][2] so a regional event is one outage in two contracts. Check where each of your connections is actually hosted before calling them independent.
- **Same-model failover.** Routing the same model through two hosts removes host failures but keeps model-level faults: a bad snapshot, a safety-filter regression or a capacity shortage on that model.
- **Shared quota or account.** Two connections that are the same organization's keys share its rate limit. INFRO's connection failover spreads requests across a provider's active keys and tries standbys last,[3] but if the limit is per organization, more keys add no capacity.
- **Shared demand spikes.** When a popular model launches or a competitor has an incident, traffic moves and several providers throttle at once.
- **Your own dependencies.** The same egress NAT, secret store or deploy pipeline sits under every target.

A cross-model fallback has a different weakness. It decorrelates model-level faults, but it changes answers, so "available" now depends on whether the fallback is good enough for that feature; test it with [fallback equivalence](https://infro.io/blog/is-your-fallback-model-good-enough-test-fallback-equivalence) before counting its successes as good events.

## Measure the inputs from your own records

You need per-attempt outcomes, not just final outcomes: which target was tried, in what order, and how it failed. INFRO's request records name the model, the connection that served each request and the route,[6] and `GET /v1/requests` and a CSV export of up to 50,000 rows per file are available, bounded by your plan's queryable history of 7 to 365 days.[7] OpenTelemetry spans can also be pushed to your own collector, but delivery is best effort with one retry and dropped spans on persistent failure, so the documentation says not to use that channel as the sole accounting record.[7] For an SLO, export the request records on a schedule and keep them yourself, because a 7-day window will not cover a 90-day estimate.

A minimal estimator over per-minute data (column names are illustrative; map them from your export):

```python
import pandas as pd

# minute, provider, attempts, failures  (aggregated from your exported request records)
df = pd.read_csv("attempts_by_minute.csv", parse_dates=["minute"])
df["down"] = (df.failures / df.attempts) > 0.05      # Vertex-style 5% minute rule, as a yardstick

wide = df.pivot(index="minute", columns="provider", values="down").fillna(False)
a_down, b_down = wide["provider_a"], wide["provider_b"]

both = (a_down & b_down).sum()
rarer = min(a_down.sum(), b_down.sum())
print("minutes both down:", both, "of", rarer, "-> beta ~", both / rarer if rarer else None)
```

Report the counts with the ratio. A β of 0.1 from 3 incident minutes is a guess; from 300 it is a measurement. Re-estimate it quarterly and after any change in where targets are hosted.

## From composite to commitment and error budget

Do not commit to the computed number. Subtract a margin for what the model cannot see: unmeasured correlation, a future provider with a worse year, and the variance of a short window. A common practice is to commit to a target whose error budget is at least about twice what the model predicts you will spend. In the example, the model predicts about 99.75%, so a 99.5% commitment leaves roughly half of its budget unspent.

Then state the budget in units people can act on. Over a 30-day month (43,200 minutes):

| Target | Budget (fraction) | Budget in minutes of full outage |
| --- | --- | --- |
| 99.5% | 0.5% | 216 |
| 99.9% | 0.1% | 43.2 |
| 99.95% | 0.05% | 21.6 |

Table: Error budget over a 30-day month; request-based SLOs spend the same fraction of requests instead of minutes.

Because AI traffic is bursty and uneven, express the budget as a fraction of *requests*, not minutes, so a quiet night does not hide a bad hour. Alert on burn rate: page when a short window and a longer window are both consuming budget much faster than the month allows, and open a ticket on a slow burn. When the budget is spent, the policy is the usual one: freeze risky changes to routes, models and prompts until it recovers.

Finally, write the assumptions on the SLO page itself: the SLI definition, the window used to estimate each term, the β and how it was obtained, and which refusals are excluded. When any of those change, the number changes.

## Building the chain and the record in INFRO

INFRO can supply the mechanics this method needs: [routes](https://infro.io/docs/routes) with priority or weighted targets over your own connections, which still fall through to the rest when the first pick fails,[6] [failover and fallbacks](https://infro.io/docs/features/fallbacks) before the first token, and per-request records with the connection that served each attempt for the estimates above. It does not make the chain's failures independent, and its own availability is one of the series terms to include.

## Questions

### Should the SLO be per model, per provider or per feature?

Per feature. The user experiences the feature, not the provider. Compute a composite for each feature from the chain it actually uses, since a feature with a cross-model fallback and one with same-model failover have different correlation terms.

### How much history do I need before committing?

Ninety days is a workable minimum, because a 30-day window may hold only one or two provider incidents and the common-mode estimate rests on those events. Until you have it, commit to a lower number and say it is provisional.

## Sources

1. Amazon Bedrock Service Level Agreement — Amazon Web Services. https://aws.amazon.com/bedrock/sla/ (accessed 2026-11-15)
2. Vertex AI Platform Service Level Agreement (SLA) — Google Cloud. https://cloud.google.com/vertex-ai/sla (accessed 2026-11-15)
3. Failover & fallbacks — INFRO Docs. https://infro.io/docs/features/fallbacks (accessed 2026-11-15)
4. Gateway status and uptime — INFRO. https://infro.io/status (accessed 2026-11-15)
5. Errors — API reference — INFRO Docs. https://infro.io/docs/api/errors (accessed 2026-11-15)
6. Routes — INFRO Docs. https://infro.io/docs/routes (accessed 2026-11-15)
7. Exports & OpenTelemetry — INFRO Docs. https://infro.io/docs/observability/exports (accessed 2026-11-15)
