Reasoning tokens: the hidden cost of thinking models

A thinking model answers "what is 2 + 2" with one visible token and may have produced hundreds before it. Those tokens do not appear in the response text, but they appear on the invoice, at the output rate, and they take generation time before the answer arrives. For teams moving production traffic onto reasoning models, this is the single largest difference between the price table and the bill.

How each provider bills reasoning

OpenAI. Reasoning tokens are not visible through the API, but they occupy space in the context window and are billed as output tokens. The response reports them separately as reasoning_tokens in the output token details. Reasoning effort takes values from none through minimal, low, medium, high, xhigh and max, though GPT-6 Astra does not support none¹.

Anthropic. On Claude 4.7 and later models thinking is adaptive: the model decides per request whether and how much to think, steered by output_config.effort, whose levels are low, medium, high, xhigh and max. high is the default on most models and medium on Claude Opus 5.5². The manual budget_tokens setting is rejected with a 400 on these models³. Thinking tokens are billed as output; thinking blocks from earlier turns that stay in context are billed as input. Crucially, you are billed for the full thinking the model generated, not the summarized thinking returned in the body, so the billed output count does not match the visible one. usage.output_tokens_details.thinking_tokens reports how many billed output tokens were reasoning².

Google Gemini. Gemini 3 models think dynamically by default and are steered with thinking_level; Gemini 3.8 Flash accepts low, medium and high, and no Gemini 3 model lists an option to turn thinking off. Response pricing is the sum of output and thinking tokens⁴, and the pricing page states that output rates include thinking tokens⁵.

The rate that applies is the output rate

Because reasoning is billed as output, its cost scales with the model's output rate, which is typically four to six times its input rate. The same thousand reasoning tokens cost $50 per million on GPT-6 Astra and Claude Fable 5.1, $20 on Claude Opus 5.5, $10 on GPT-6 Sol and Claude Sonnet 5.5, and $3.75 on Gemini 3.8 Flash through 2026-12-31⁶⁷⁵. DeepSeek Flash lists $1.20 at peak hours⁸.

Reasoning takes the output limit too

Thinking tokens count against the request's output ceiling. Anthropic states that max_tokens is a hard cap on thinking and response text combined, and that a max_tokens sized for a response with no thinking is often too small once the model starts thinking on hard requests; a response ending with stop_reason: "max_tokens" means either raising the cap or lowering effort². OpenAI returns a response with status: "incomplete" and incomplete_details.reason of max_output_tokens when the limit is reached, and recommends reserving at least 25,000 tokens for reasoning and output when you start experimenting¹.

The failure is easy to misread. A request that exhausted its limit while thinking may return little or no visible text, and still bill every reasoning token it produced. Treat a length-limited finish as a sizing problem and track its share per model.

Latency follows the tokens

Reasoning tokens are generated like any other output, so they take generation time. When a model thinks before it answers, that time is spent before the first visible token of the answer, which is why a thinking model can show a long time to first token on a stream even when its throughput is normal. Effort therefore trades three things at once: quality on hard requests, cost, and time to first visible token. Measure all three when you change it.

Reasoning and prompt caching

On Anthropic, the resolved effort value and the thinking configuration are rendered into the prompt, so changing either between requests invalidates cache breakpoints. Anthropic recommends picking one thinking configuration and effort level per conversation, and steering individual turns with guidance in the newest user message instead, which leaves earlier breakpoints intact². A per-request effort switch that looks like a cost control can raise input cost by turning cache reads into cache writes. See prompt caching break-even.

Reasoning in multi-turn and agent loops

A single request is the easy case. Conversations and tool loops change the arithmetic in two ways.

Earlier reasoning can come back as input. Anthropic says Claude Opus 4.5 and models numbered 4.6 and higher keep earlier turns' thinking blocks in context and bill them as input, where older models stripped them³. OpenAI says GPT-5.6 models default to rendering available reasoning from earlier turns into the next one, where earlier models did not¹. In a long conversation, then, reasoning is billed once as output when generated and again as input on every later turn that carries it, which is where prompt caching earns its keep.

The output limit bounds a request, not a task. In a tool-use loop each request has its own max_tokens, so the cap does not bound the whole turn's spend². Anthropic also notes that the first request after new user input typically carries most of the reasoning, and follow-up requests that only process tool results may skip thinking even at high effort². An agent's reasoning cost is therefore uneven across its steps: large on planning steps, small on tool-result steps. Budget it per run, not per call.

Controls that work

1. Set effort per workload, not globally. Classification, extraction and routing rarely need high effort; multi-step planning and debugging often do.
2. Size the output limit for reasoning plus answer. Start generous, measure actual output tokens per request, then tighten.
3. Record reasoning tokens. Store the provider's reasoning or thinking token count beside total output tokens, so you can see how much of the output bill is reasoning.
4. Hold the configuration stable inside a conversation where you rely on prompt caching.
5. Compare on cost per successful task. A higher effort that raises success rate can lower the cost of finished work. See cost per successful task.

def reasoning_share(usage: dict) -> float | None:
    """Share of billed output that was reasoning, from an OpenAI Responses or Anthropic usage object."""
    output = usage.get("output_tokens") or 0
    details = usage.get("output_tokens_details") or {}
    reasoning = details.get("reasoning_tokens", details.get("thinking_tokens"))
    if not output or reasoning is None:
        return None
    return reasoning / output

Reasoning models through INFRO

INFRO reports the token counts your provider returned and does not re-tokenize, so the completion token count on a response is the provider's output count, which already includes reasoning because providers bill it as output. The estimate in usage.cost applies the output rate to it, an estimate of what your own provider bills; it does not break reasoning out as a separate field today, so keep the provider's own detail field if you need that split. Top-level parameters INFRO does not model, such as reasoning_effort, are forwarded unchanged on OpenAI and OpenAI-compatible connections; see chat completions.

Three INFRO features bear directly on thinking models. Traffic changes raises a finding when output tokens per request on a model rise by at least 15% and 20 tokens against its usual day, or when the share of answers cut off by the length limit rises by 3 percentage points, each with its estimated weekly change in provider cost. Under a blocking budget, a chat request with no max_tokens reserves the model's maximum output and INFRO sends that ceiling to the provider; set max_tokens yourself, with room for reasoning, to reserve less. And attribution by workflow lets you compare cost per run for the same workflow before and after an effort change.