# Do providers bill cancelled or failed streams? What to forecast

URL: https://infro.io/blog/do-providers-bill-cancelled-or-failed-streams-what-to-forecast
Author: INFRO Editorial (https://infro.io/blog/editorial)
Publisher: INFRO
Published: 2026-11-05
Last modified: 2026-11-05
Last reviewed: 2026-11-05 (next review due 2026-12-05)
Topic: Cost and FinOps
Format: Analysis

> In the pages read for this analysis, neither OpenAI nor Anthropic states whether a cancelled, timed-out or errored stream is billed, and Google's billing page could not be read. Forecast retry waste as a range between "failed attempts cost nothing" and "failed attempts bill in full", then narrow it with your own provider cost reports.

Key takeaways:
- The streaming and error pages from OpenAI and Anthropic read for this piece describe stream mechanics, not billing of interrupted calls, so any billed fraction you put in a forecast is an assumption until your own cost reports confirm it.
- A dropped connection and a cancelled request are different events: OpenAI's background mode documents that a response keeps running after the client drops, and cancel is a separate call.
- Retry waste scales with prompt size times the rate of failed attempts, so long-context routes dominate the line even when failure rates are low.
- Forecast two cases, failed attempts free and failed attempts billed in full, then compare the high case with provider cost reports to see which one your traffic behaves like.
- Record every attempt with its own request id, not only the final success, or the waste can't be separated from normal spend.

## What the documentation actually says

The question is simple: if a stream is aborted, times out or errors halfway, what does the provider bill? The pages read for this analysis don't answer it. OpenAI's streaming guide covers how to enable streaming and which events to listen for: `response.created`, `response.output_text.delta`, `response.completed` and `error`.[1] It says nothing about billing for a stream that never reaches `response.completed`. Anthropic's streaming page describes the event flow and notes that the token counts in the `usage` field of `message_delta` are cumulative.[2] Its errors page describes what happens when a call fails, but it doesn't state a charge either.[3]

Google is a gap in this review. Its Gemini billing page could not be read in this session, so there is no Google row below with a claim in it. If you run Gemini, read that page yourself and fill the row in before using the numbers in this article.

That leaves a forecaster with a choice: guess a billed fraction, or model it as a range and measure. The rest of this piece takes the second route. What the pages do document is useful for deciding which events *can* produce spend that never reaches a user.

| Event | OpenAI | Anthropic | Google |
| --- | --- | --- | --- |
| Error before the stream starts | Not stated in the pages read | HTTP error codes, including 500 `timeout_error` when the server time limit is hit before the response starts[3] | Not read |
| Error after the stream starts | `error` event in the stream[1] | `error` event after a 200, such as `overloaded_error`[2] | Not read |
| Client disconnects | Background streams keep running; you can resume from a cursor[4] | Not stated in the pages read | Not read |
| Explicit cancel | Cancel endpoint for background responses, idempotent[4] | Not stated in the pages read | Not read |
| Billing rule for any of the above | Not stated in the pages read | Not stated in the pages read | Not read |

Table: What each provider's pages state about interrupted calls. "Not stated" refers only to the pages read on 2026-11-05.

## Four events that can bill without delivering

Absent a stated rule, classify events by what the provider's system has probably done by the time your client acts. The documents support some of this and leave the rest to inference, and the inference is labelled below.

1. **Failure before the first token.** Anthropic's errors page lists 500, 504 and 529 as possible outcomes, and says the official SDK retries transient failures twice by default, honoring `retry-after`.[3] Those retries are attempts you did not write. If any attempt were billable, the SDK default could multiply your cost without a line in your code.
2. **Error after a 200.** Both providers can fail inside an established stream. Anthropic says errors can occur after the API returns a 200, and that standard error handling does not apply then.[3] Your retry logic has to detect the in-stream `error` event, not just the HTTP status. The partial output was generated; whether it is billed is the open question.
3. **Client timeout while generation continues.** OpenAI's background mode documents that if the connection drops, "the response continues running" and you can reconnect.[4] That is the documented behaviour for background responses. For ordinary foreground streams the pages don't say, so treat this as a risk to test, not a fact.
4. **Cancel that arrives late.** OpenAI's cancel call is a separate request, and cancelling twice returns the final response object.[4] Between the user leaving and the cancel landing, tokens are being produced. The gap length is your own latency, which you can measure.

A fifth case belongs to anyone running a gateway. INFRO's failover moves a request to the next connection only before the first token, and its docs say every upstream failure before the first token moves on, including 5xx, timeouts and 429s.[5] After a mid-stream error event the connection closes without the `[DONE]` sentinel.[6] So a single user request can create several upstream attempts, each with its own possible charge on your own provider account.

## The retry-waste line

Put waste in the forecast as its own line with an explicit billed-fraction parameter per failure class. For each class *c*:

```text
waste_c = requests × failure_rate_c × billed_fraction_c × cost_of_one_failed_attempt_c
```

Where `cost_of_one_failed_attempt` is the prompt tokens times the input rate plus the output tokens produced before the failure times the output rate. The billed fraction has three useful values: 0 (the provider bills nothing for a failed attempt), 1 (it bills what was generated up to the failure) and a measured value in between. Run the forecast at 0 and at 1 and report both.

Two inputs deserve care. First, the prompt term repeats on every retry, so long prompts make the line grow with context size even when failure rates are flat. Second, output before failure depends on where the failure lands. Mid-stream failures after the model has produced hidden reasoning (Anthropic's page documents `thinking_delta` events streaming before text[2]) can carry more output than the visible text suggests.

> **Example: Illustrative numbers, not measured**
> Assume 1,000,000 streamed requests a month, 3,000 prompt tokens and 600 output tokens each, at an illustrative $2 per million input tokens and $10 per million output tokens. Base spend is $12,000.
> 
> - Failure before first token, retried: 1.5% of requests, 3,000 input tokens per failed attempt. High case: 15,000 × $0.006 = $90. Low case: $0.
> - Error mid-stream at about half the output, then retried: 0.5%, 3,000 input plus 300 output tokens, $0.009 per attempt. High case: 5,000 × $0.009 = $45. Low case: $0.
> - Client timeout where the provider finishes anyway, then the client resubmits: 0.3%, a full $0.012 attempt. High case: 3,000 × $0.012 = $36. Low case: $0.
> 
> The retry-waste line is $0 to $171 a month, about 1.4% of base spend. With a 100,000-token prompt the input term is roughly 33 times larger, and the same failure rates produce a line that matters.

The example is deliberately modest. The point is the shape: the line is small on short prompts and large on long-context routes, and the base forecast can't show that because average cost per request blends both.

## Measure the billed fraction instead of assuming it

The low and high cases are a bracket. To tighten it, you need attempt-level records and a provider cost report to compare against.

- **Log every attempt.** Give each upstream attempt its own record with the provider's request id, the attempt number, how it ended (success, error before first token, error mid-stream, client abort, timeout) and the tokens seen so far. Anthropic's responses include a `request-id` header for this purpose.[3]
- **Capture usage on streams that finish.** With `stream_options.include_usage`, INFRO sends a final chunk carrying `prompt_tokens`, `completion_tokens` and an estimated `cost` before `[DONE]`.[6] That chunk never arrives on a stream that dies, which is why a stream's absence of a usage chunk is itself a data point: count those.
- **Run a controlled probe.** Send a known set of requests, abort a known fraction at a fixed token count, wait for the provider's usage report to settle, and compare totals with and without the aborts. Do it per provider and per model, because one provider's behaviour tells you nothing about another's.
- **Reconcile daily.** The gap between your estimate and the provider's reported cost, tracked over weeks, shows whether failed attempts are billed. A gap that rises with your failure rate points at the high case.

```python
from dataclasses import dataclass

@dataclass
class Attempt:
    request_id: str       # provider's id, not yours
    attempt: int
    outcome: str          # ok | pre_token_error | mid_stream_error | client_abort | timeout
    prompt_tokens: int
    output_tokens_seen: int

def waste_range(attempts, in_rate, out_rate):
    """Return (low, high) spend on attempts that did not succeed.
    Rates are per token. Low assumes failed attempts bill nothing."""
    high = sum(
        a.prompt_tokens * in_rate + a.output_tokens_seen * out_rate
        for a in attempts if a.outcome != "ok"
    )
    return 0.0, high
```

If you run requests through a gateway, it can supply some of this. INFRO records per-request model, latency, token counts and estimated provider cost, exports them through `GET /v1/requests`, and compares estimated spend daily against the cost OpenAI's and Anthropic's own cost APIs report. A persistent gap is the signal that something is billed that the estimate did not count. The estimate is not an invoice, so keep the provider's report as the source of truth.

## Reduce the line before you measure it

Some controls are free of any billing assumption.

- **Cap SDK retries deliberately.** Anthropic's SDK retries twice by default and exposes `max_retries`.[3] Decide that number; don't inherit it. Stacking an SDK retry under a gateway retry under an application retry multiplies attempts.
- **Retry once, at one layer.** Put the retry where you can see it, and make the other layers pass failures through.
- **Never resubmit on a timeout without checking.** If the provider may still be generating, resubmitting creates a second full attempt. For long jobs, OpenAI documents background mode with polling and resume.[4]
- **Send cancel promptly.** A cancel issued when the user leaves bounds the output the provider produces after that point, where the API supports it.
- **Match timeouts to the first-token time.** A timeout shorter than your slow-but-healthy tail turns working requests into retries; see [measuring time to first token](https://infro.io/blog/measure-time-to-first-token).

For the surrounding costs of failover itself, such as price gaps between models and cold caches, see [what failover does to your bill](https://infro.io/blog/what-does-failover-do-to-your-bill-price-gaps-and-cold-caches). For hedging and retry budgets, see [should you hedge LLM requests](https://infro.io/blog/should-you-hedge-llm-requests-retry-budgets-and-tail-latency).

## What to put in the forecast review

A short checklist for the next forecast cycle:

- [ ] A retry-waste line exists, separate from base spend, with a low and a high case.
- [ ] Each failure class has a rate you measured from attempt logs, not a guess.
- [ ] The billed fraction for each provider is marked "assumed" or "measured on (date)".
- [ ] Long-context routes are forecast separately, because their prompt term dominates.
- [ ] Retry layers are inventoried: SDK, gateway, application, queue worker.
- [ ] Google's billing page, or any provider page not read here, has been checked by someone before its row is filled in.

Provider docs change, and a rule that's unstated today may be stated tomorrow. Re-read the pages when you re-run the probe, and date the result.

## Questions

### Is a request I cancel from the client billed?

The pages read for this piece don't say. OpenAI documents a cancel call for background responses and says cancelling twice is idempotent[4], but neither it nor the streaming page states a billing rule. Treat the billed fraction as unknown and test it against your provider's cost report.

### Should retry waste be part of the base forecast or a separate line?

A separate line. Folded into the base, it hides behind average cost per request. As its own line it has an owner, a failure-rate input and a low and high case that you can tighten as invoices arrive.

## Sources

1. Streaming API responses — OpenAI. https://developers.openai.com/api/docs/guides/streaming-responses.md (accessed 2026-11-05)
2. Streaming messages — Anthropic. https://platform.claude.com/docs/en/build-with-claude/streaming.md (accessed 2026-11-05)
3. Claude API errors — Anthropic. https://platform.claude.com/docs/en/api/errors (accessed 2026-11-05)
4. Background mode — OpenAI. https://developers.openai.com/api/docs/guides/background.md (accessed 2026-11-05)
5. Failover & fallbacks — INFRO. https://infro.io/docs/features/fallbacks (accessed 2026-11-05)
6. Streaming — API reference — INFRO. https://infro.io/docs/api/streaming (accessed 2026-11-05)
