Provisioned throughput vs pay-as-you-go: when reserved capacity pays

Why the question is really about utilization

Provisioned capacity inverts the pay-as-you-go deal. On Azure, a provisioned deployment holds a fixed amount of processing capacity exclusively for you, whether or not requests are being made.¹ You are billed hourly on the PTUs you deploy rather than on tokens consumed.² Bedrock's Reserved tier works the same way in principle: you reserve capacity for 1 month or 3 months, pay a fixed price per 1K tokens-per-minute, and are billed monthly.³

So the only variable is how much of the bought capacity you fill. If you fill all of it, reserved capacity costs the same as the tokens it carried, priced at the reserved rate. If you fill a third of it, you pay three times the per-token rate. Before the arithmetic, check that cost is actually your problem. Reserved capacity also buys latency and rate-limit headroom, and there are cheaper-to-try levers for that: Azure's priority processing bills per token at a priority rate with a defined latency target per model and no long-term commitment,¹ and Bedrock's Priority tier charges the standard rate plus a premium, with up to 25% better output tokens per second on most supported models.³ Bedrock's pricing page lists Standard, Flex, Priority and Reserved side by side.⁴ Try those first if the pain is latency spikes on moderate volume. Our tiers comparison covers them.

The break-even formula

Define the capacity of one unit in tokens per hour, and the on-demand price of those same tokens. The value of one fully used unit-hour is:

full_use_value = tokens_per_unit_hour × on_demand_price_per_token
break_even_utilization = fixed_cost_per_unit_hour / full_use_value

If your average utilization of the capacity you bought is above that ratio, reserved wins. Below it, on-demand wins, even before counting the risk of stranded capacity.

On Azure the unit is the PTU, and you can derive tokens_per_unit_hour from Microsoft's own sizing method. Normalized TPM is input TPM times one minus the cache rate, plus the output-to-input ratio times output TPM, and PTUs required is normalized TPM divided by the model's input TPM per PTU.⁵ For GPT-6 Sol, the sizing page lists 3,000 input TPM per PTU, and for the newer normalized-token models one short-context input token counts as one normalized token, with output weighted at 5.0 because it is priced at five times input.⁵ Global Standard input is listed at $2.00 per million tokens.⁵ That gives a full-use value you can compute from published numbers: 3,000 × 60 × $2.00 / 1,000,000 = $0.36 per PTU-hour.

Three adjustments make the real number differ from this clean one:

• Cache rate. Models that do not use normalized accounting deduct cached tokens fully from utilization; GPT-6-class models consume capacity for cached input at weights such as 0.1 of input.⁵ Compute full_use_value with your actual token mix, not a pure-input mix.
• On-demand rate you actually pay. Use the negotiated rate if you have one. Break-even moves when the denominator moves.
• Rounding. Capacity comes in minimum sizes and scale increments, for example 15 PTU minimum and 5 PTU increments on Global and Data Zone for most listed text models, but 50 or 100 on Regional.⁵ Round up before computing utilization.

Size the reservation to the baseline, then spill the rest

Each additional unit of capacity is only worth buying if it would be filled more than break-even of the time. That makes the right size a percentile of your load, not its maximum. The script below finds it from a series of average normalized tokens per minute, one value per hour, which you can build from request-level usage.

def best_units(load_tpm, tpm_per_unit, unit_hr, od_per_mtok, step=5):
    """load_tpm: avg normalized tokens/min for each hour of the window."""
    hours = len(load_tpm)
    best = (0, sum(l * 60 * od_per_mtok / 1e6 for l in load_tpm))
    for units in range(step, 1000, step):
        cap = units * tpm_per_unit
        fixed = units * unit_hr * hours
        spill = sum(max(l - cap, 0) * 60 * od_per_mtok / 1e6 for l in load_tpm)
        total = fixed + spill
        if total < best[1]:
            best = (units, total)
    return best  # (units to reserve, total cost incl. on-demand overflow)

Two limits apply. Hourly averages hide intra-hour bursts, so use per-minute data if you have it. And the script prices overflow at the on-demand rate, which is only true if overflow actually has somewhere to go, which is the next section.

Azure adds operational constraints that change how you buy. Reservations are a billing construct, loosely coupled to deployments, and do not guarantee capacity, so create the deployment first and buy the reservation after.² Quota is not capacity either: with quota but insufficient regional capacity, the deployment fails.¹ Scaling down releases capacity permanently, with no guarantee you can scale back up.² A reservation is not resized when the deployment is: PTUs below the reserved quantity are unused coverage, and PTUs above it bill at the hourly rate.² Treat the reserved size as a long-lived number.

Where the overflow goes

A reserved slice that is sized to baseline will saturate during bursts. What happens then is a design decision, in three layers.

Layer 1: same-provider on-demand. Azure's spillover redirects requests to a standard deployment in the same Foundry resource when the provisioned deployment returns non-200 responses such as a 429 when PTUs are exhausted.⁶ It also triggers on a 400 for long-context requests the provisioned deployment does not support, and on 500 or 503 errors.⁶ You can enable it on the deployment with spilloverDeploymentName or per request with the x-ms-spillover-deployment header; if both are set, the deployment property wins.⁶ It needs an active pay-as-you-go deployment matching the model and version.⁶ Spilled requests are billed at the standard input, cached and output token rates, while requests the provisioned deployment serves carry only the hourly cost.⁶ Two caveats: the service sends requests to provisioned first, which may add latency to the overflow,⁶ and spillover covers Azure OpenAI models but not Azure DeepSeek or Meta Llama.¹ Responses show x-ms-spillover-from-deployment and x-ms-spillover-error, so log them to measure how often you overflow.⁶ That rate is your real utilization signal.

Layer 2: a second provider. Layer 1 shares a provider path with the primary. If the problem is regional or provider-wide, or the standard deployment has its own limits, you want a target elsewhere, such as Bedrock on-demand or another vendor's API, with a model you have tested as a substitute. Spillover's cost is also worth comparing: overflow goes at list on-demand rates, and a second provider may price that traffic differently.

Layer 3: stop. For non-urgent work, return a retry-after rather than paying for overflow. Not every request deserves a fallback.

Doing layers 1 to 3 at the gateway

If the second provider lives behind a gateway, the pattern is a route. INFRO routes resolve a stable name to an ordered list of your own provider connections and the model each serves; the priority strategy walks enabled targets in order and uses the first that answers.⁷ A failure before the first token, including a provider 429, 5xx or timeout, moves the request to the next connection, and the client sees only a higher time to first token.⁸ Both layers operate strictly before the first token, so a stream that breaks midway is not retried.⁸ A provider 429 does not count against a connection's health.⁸ Your providers keep invoicing you directly, and INFRO takes 0% of model spend.⁹

A reasonable route has the reserved connection first, then the same vendor's on-demand path if it is a separate connection, then the second provider. Rules can pre-empt the order. This one, modeled on the docs' rule shape, sends prompts too long for the provisioned deployment straight to on-demand:

{
  "name": "Long prompts",
  "match": "all",
  "conditions": [
    { "field": "input_tokens", "op": "gt", "value": 120000 }
  ],
  "targets": [{ "connection_id": "conn_…", "model": "openai/gpt-6-sol" }],
  "fallthrough": true
}

Rules can match on input_tokens, stream, has_tools, metadata and time of day, among others.⁷ That lets you keep interactive traffic on reserved capacity and send batch-like work to on-demand from the start, which lowers the baseline you need to reserve. Note that on-demand failover moves traffic to a model you must have validated; see our notes on circuit breakers for why a throttled primary is not a failed one.

What to record so finance can verify the decision

The break-even holds only while the inputs do. Keep three records:

1. Per-request token counts and estimated cost, grouped by project, so the baseline can be re-derived when traffic changes. INFRO records model, latency, token counts and estimated provider cost for each request, and you can enter negotiated rates so estimates use them before list rates.
2. The reservation or commitment itself, tracked against reported cost. INFRO can hold a committed-spend agreement and forecast it to the end of the term, which is useful for the 1-month, 3-month and 1-year terms these products use.
3. The spillover rate. If more than a small share of traffic overflows, the reservation is undersized; if the reserved units sit idle, you are below break-even and should not renew at the same size.

Re-run the sizing script at every renewal, and when the model changes, since tokens per PTU differ by model.⁵

When on-demand is still the right answer

Stay on pay-as-you-go when traffic is spiky enough that a baseline percentile is near zero, when volume is low, or when you cannot size the deployment in advance; Microsoft itself says standard deployments remain the better fit for development, testing, low volume and highly variable traffic.¹ Also stay if you cannot confirm capacity in your region, or if your failover target is untested. A reservation without a tested overflow path trades a rate-limit problem for a cost problem.