How to see the cost of every LLM request

Every LLM provider will tell you what you spent yesterday. None will tell you what this request cost. Their cost APIs report by day and by account, project or line item; the cost of an individual request is something you compute from the response. This guide covers the four ways to see it, and the three mistakes that make the number wrong.

What the provider gives you

In each response: token counts. OpenAI reports input and output tokens in the response's usage object, with cache reads and writes, and reasoning tokens, broken out in the token details¹². Anthropic reports input_tokens, output_tokens, and cache reads and writes in cache_read_input_tokens and cache_creation_input_tokens³.

Afterwards, by API: totals. OpenAI's Costs API returns organization spend in daily buckets, groupable by line item or project, and its Usage API returns token counts in buckets of a minute, an hour or a day⁴. Anthropic's Usage and Cost API reports usage in buckets down to a minute and cost by day, grouped by workspace and line item⁵.

Neither carries your own dimensions: which customer, which feature, which experiment. So per-request cost is always token counts from the response, multiplied by a rate you hold, tagged with dimensions you add.

Four ways to see it

1. Compute it in your own code

Read the usage object from each response, multiply by your rates, and write a row with your own tags. It is free and fully under your control. It is also the one most likely to drift: every service that calls a model has to do it the same way, every new model needs its rate added, and requests from a script someone wrote on a Friday never appear.

from decimal import Decimal

# Your contract's rates per 1M tokens: example values, not any provider's prices.
RATES = {"example-model": {"input": Decimal("1.00"), "cached": Decimal("0.10"), "output": Decimal("4.00")}}

def openai_style_cost(model: str, usage: dict) -> Decimal | None:
    rate = RATES.get(model)
    if rate is None:
        return None  # unpriced: never record as 0
    details = usage.get("prompt_tokens_details") or {}
    cached = details.get("cached_tokens", 0)
    fresh = usage["prompt_tokens"] - cached  # cached tokens sit INSIDE the input total
    per_m = Decimal(1_000_000)
    return (fresh * rate["input"] + cached * rate["cached"] + usage["completion_tokens"] * rate["output"]) / per_m

The rates above are placeholders. Cache-write premiums and reasoning tokens need their own lines on models that bill them.

2. Provider dashboards

Good for the monthly total and for spotting a bad day. Not per request, not across providers, and not by your customer or feature. They are still worth opening weekly: they are the provider’s own view, so a day that looks very different there from your own per-request totals is the earliest sign that a rate is wrong or that some traffic is reaching the provider without passing through whatever records your costs.

3. An observability tool

Tools that receive your traces can price each one. Langfuse, for example, infers cost from model definitions you can edit, or stores the cost you send it verbatim, which it recommends when you have private pricing⁶. This gives a per-request view with your own tags, alongside latency and quality. It sits beside the request rather than in it, so it shows cost after the call and cannot stop a request.

4. A gateway

A gateway records every request it serves with its token counts and an estimated cost, so nothing that passes through it is missed and no application has to do the arithmetic. If its budgets are checked before the call, the same figure can also refuse a request that would breach a limit. Its coverage is exactly the traffic sent through it; requests made directly to a provider are invisible to it.

The three mistakes that make the number wrong

The rate. Most tools price at the provider's public list rate by default. If your company has a negotiated rate, every per-request figure is wrong by your discount from list, in the same direction, and every per-customer margin calculation inherits the error. Enter your contract's rates wherever the tool allows.

The cache fields. Anthropic's input_tokens counts only tokens after the last cache breakpoint, with reads and writes reported beside it; the total is the sum of all three³. OpenAI's cached and cache-write counts sit inside the input total¹. Code that treats both the same way either double-counts cached tokens or misses them. Tokens are not the billing unit works through an example.

The unknown price. When a tool has no rate for a model, some record the request at $0. That turns your newest, often most expensive model into what looks like free traffic. The correct value is "unpriced", counted and visible as its own line, until someone enters a rate.

Requests that are not text

Images, video and speech are billed in units that are not tokens: per image at a resolution and quality, per second of video, per character or minute of audio. A per-request cost view that only multiplies tokens shows these requests as zero or leaves them out, and for a product that generates media they are often most of the spend. Record the billing unit and quantity for each media request (images: 2, size: 1024x1024, video_seconds: 8) beside the token fields, and price it with the provider's rate for that unit. Estimate LLM cost before the request lists the dimensions that change the price for each kind.

Streaming, retries and failures

Three kinds of request are easy to price wrongly:

• Streamed responses. Usage arrives with the end of the stream, if the client asked for it; a client that disconnects early may never receive it. Price what the provider reports, and treat a stream with no usage as unpriced rather than free.
• Retries. A request retried after a timeout can be billed twice if the first attempt reached the model. Record each attempt as its own row, linked to the request it served.
• Failed requests. A request that fails before the model runs usually costs nothing; one that fails after it has started generating can still be billed. Record the failure with whatever usage the provider returned.

Getting these right matters more as a product grows, because they cluster in exactly the incidents you will want to explain.

Check the total against the bill

Per-request figures are estimates. Once a month, sum them per provider per day and compare with the provider's own cost API. A small, stable gap is usually the rate; a gap that grows is usually traffic that went around whatever records the cost. Reconcile LLM spend against provider invoices has the code for both OpenAI and Anthropic.

How INFRO records it

INFRO is a gateway, so this is the vendor's description of the fourth option. Every request it serves is recorded with provider-reported token counts and an estimated provider cost, returned in the response as usage.cost and queryable by project, key, model and end customer through GET /v1/requests and the console. The estimate uses the vendor's list rate, or your negotiated rate from the Team plan; a model with no known rate is marked unpriced. On the Business plan, the month's estimates are checked against OpenAI's and Anthropic's cost APIs and the variance is stated. Requests that do not pass through INFRO are not in it.

Which to pick

If you call one provider from one service, compute it in your own code and check it monthly. If you already use an observability tool, turn on its cost tracking and give it your rates. If you call several providers from several services, or need the figure to enforce a limit, put it in the request path.