Reconciling estimated LLM spend with provider invoices

Almost every cost number you see for LLM traffic is computed, not measured: token counts from the provider's response, multiplied by a rate table. It is a good estimate when the rate table matches your contract and every request went through the system doing the multiplying. Reconciliation is how you find out whether both are true.

This guide is the mechanical version: which endpoints to call, what shape the answers come in, how to compare them without chasing noise, and how to turn a persistent gap into a single named cause. For why the gap exists at all, see Reconcile your AI bill: estimate vs. provider invoice.

What you are comparing

Two figures, for the same provider and the same UTC day:

• Estimated cost. The sum of your per-request estimates for traffic sent to that provider's accounts. It comes from your gateway, your logs, or your own metering.
• Reported cost. What the provider's own cost API says your organization spent. It is the closest thing to the invoice you can read by API, and it is organization-wide.

Reconcile at that grain and no finer. A provider reports totals per day, per line item or workspace; it does not report the cost of your request req_123. Dividing a day's reported total back over individual requests produces numbers that look precise and mean nothing. Keep each request at the cost it was estimated at, and keep the comparison at the level both sides can see.

Reading OpenAI's Costs API

OpenAI's organization Costs API lives at GET /v1/organization/costs and needs an Admin key, created separately from the keys that serve traffic. It takes start_time in Unix seconds, returns daily buckets (bucket_width of 1d), and can group results by line_item or project_id. Each bucket carries start_time, end_time and results; each result has an amount object with a numeric value and a currency such as usd. Pages continue while the response has a next_page cursor, passed back as page¹.

The companion Completions Usage API, GET /v1/organization/usage/completions, returns token counts — input_tokens, input_cached_tokens, output_tokens and num_model_requests — in buckets of a minute, an hour or a day, groupable by model, project or API key¹. Cost tells you how much; usage tells you how many tokens, and which key sent them.

Reading Anthropic's Cost Report

Anthropic's Usage and Cost Admin API needs an Admin API key (sk-ant-admin01-…) or another admin credential; workspace API keys do not work². The cost endpoint is GET /v1/organizations/cost_report, daily only, groupable by workspace_id and description. Grouping by description adds parsed fields per line: model, token_type (uncached input, cache read, five-minute and one-hour cache creation, output), service_tier (standard or batch), context_window and cost_type³.

Two details matter for reconciliation:

• Units. amount is a decimal string in the lowest currency unit: "123.45" in USD is $1.23³. Read it as dollars and every Anthropic figure is a hundred times too large.
• Coverage. Priority Tier costs are not in the cost endpoint, and the programmatic endpoints are not available for Claude Platform on AWS². An account using either will never reconcile from this endpoint alone.

Usage data typically appears within five minutes of a request completing; the usage endpoint supports buckets of 1m, 1h and 1d, groupable by API key, workspace, model and service tier².

Pulling both into one shape

The code below reads each provider's reported cost into { "YYYY-MM-DD": Decimal dollars }. Use Decimal rather than floats; you will be summing many small figures and comparing ratios.

import datetime as dt
import os
from decimal import Decimal

import requests

UTC = dt.timezone.utc

def _unix(day: dt.date) -> int:
    return int(dt.datetime.combine(day, dt.time(), UTC).timestamp())

def openai_daily_cost(start: dt.date, end: dt.date) -> dict[str, Decimal]:
    """OpenAI: amount.value is a number of dollars."""
    url = "https://api.openai.com/v1/organization/costs"
    headers = {"Authorization": f"Bearer {os.environ['OPENAI_ADMIN_KEY']}"}
    params = {"start_time": _unix(start), "end_time": _unix(end), "bucket_width": "1d", "limit": 30}
    totals: dict[str, Decimal] = {}
    while True:
        resp = requests.get(url, headers=headers, params=params, timeout=30)
        resp.raise_for_status()
        body = resp.json()
        for bucket in body.get("data", []):
            day = dt.datetime.fromtimestamp(bucket["start_time"], UTC).date().isoformat()
            for result in bucket.get("results", []):
                value = Decimal(str(result["amount"]["value"]))
                totals[day] = totals.get(day, Decimal(0)) + value
        if not body.get("next_page"):
            return totals
        params["page"] = body["next_page"]

def anthropic_daily_cost(start: dt.date, end: dt.date) -> dict[str, Decimal]:
    """Anthropic: amount is a decimal string of cents."""
    url = "https://api.anthropic.com/v1/organizations/cost_report"
    headers = {
        "x-api-key": os.environ["ANTHROPIC_ADMIN_KEY"],
        "anthropic-version": "2023-06-01",
    }
    params = {
        "starting_at": f"{start.isoformat()}T00:00:00Z",
        "ending_at": f"{end.isoformat()}T00:00:00Z",
        "bucket_width": "1d",
        "limit": 31,
    }
    totals: dict[str, Decimal] = {}
    while True:
        resp = requests.get(url, headers=headers, params=params, timeout=30)
        resp.raise_for_status()
        body = resp.json()
        for bucket in body.get("data", []):
            day = bucket["starting_at"][:10]
            for result in bucket.get("results", []):
                dollars = Decimal(result["amount"]) / 100
                totals[day] = totals.get(day, Decimal(0)) + dollars
        if not body.get("has_more"):
            return totals
        params["page"] = body["next_page"]

Your estimate side must be summed on the same boundary: the UTC day in which the provider processed the request. If your logs are in local time, convert before grouping.

Comparing without chasing noise

Compute a daily ratio, reported over estimated, and treat it with suspicion until it has a history.

from decimal import Decimal
from statistics import mean, pstdev

def ratios(reported: dict[str, Decimal], estimated: dict[str, Decimal], today: str):
    days = sorted(d for d in reported if d in estimated and d < today and estimated[d] > 0)
    return [(d, float(reported[d] / estimated[d])) for d in days]

def summarize(series: list[tuple[str, float]], min_days: int = 10):
    values = [r for _, r in series]
    if len(values) < min_days:
        return {"status": "insufficient_data", "days": len(values)}
    m = mean(values)
    cv = pstdev(values) / m if m else None
    steady = cv is not None and cv <= 0.05
    return {"days": len(values), "ratio_mean": round(m, 3), "ratio_cv": round(cv, 3), "steady": steady}

Three rules keep the comparison honest:

1. Exclude today. A part-day compared with a whole one shows a large gap every morning.
2. Ignore single days. A request at 23:59 UTC can land in one system's day and the other's next day. Over a week these cancel.
3. Watch direction, then size. Five days leaning the same way is information; one large day is not. This is the same discipline the FinOps Foundation describes for anomalies generally: detect, then investigate and document the cause before acting⁴.

Naming the cause

A gap has a direction and a shape, and together they usually point at one cause.

• A negotiated rate. Reported cost is a steady fraction of the estimate. Anthropic, for example, says volume pricing is negotiated case by case⁵. Infer the effective rate per model — reported cost for that model divided by its tokens, times one million — and enter it. See negotiated rates vs list price.
• A rate change. The ratio steps to a new level on one day and stays there. Find the contract term or the vendor's price change with that date.
• An unmodelled premium. Reported cost runs above the estimate in proportion to something the estimate ignores: cache writes, a long-context tier, a service tier. Anthropic's cost report itemises each of these by token_type, context_window and service_tier³, so group by description and compare line by line.
• Traffic outside your system. The provider reports more tokens than you sent. Group the usage endpoint by API key: a key your gateway never used is somebody's script, a notebook, or a batch job.
• Unpriced requests. Your estimate has requests with no cost because a model has no rate. The estimate is a lower bound, and the gap is partly a coverage problem, not a pricing one. Count those requests separately.

A reconciliation checklist

• Admin credentials for each provider, separate from serving keys, stored as secrets and read-only in use.
• Estimates summed per provider per UTC day, from provider-reported token counts.
• Anthropic amounts divided by 100; OpenAI amounts read as dollars.
• Today excluded; no conclusion from fewer than ten days.
• Unpriced requests counted beside the estimate, never as zero.
• Usage grouped by API key, so traffic you did not route is visible.
• Every correction applied forward only, with the dates and evidence recorded.

Reconciliation in INFRO

INFRO estimates the cost of every request when it serves it, from your own rate for the model and provider if you entered one, otherwise the vendor's list rate, and records the source of that rate on the request. On the Business plan, cost reconciliation runs the comparison above once a day: it reads OpenAI's GET /v1/organization/costs or Anthropic's GET /v1/organizations/cost_report with a separate billing credential (an OpenAI Admin key or an Anthropic sk-ant-admin key), stores reported and estimated cost side by side, and excludes today.

A persistent gap becomes a finding with one cause, a confidence and its evidence. A negotiated-rate cause needs the ratio to hold within 5% coefficient of variation over at least 10 days, and a suggested rate needs at least 7 days and $20 of reported spend for that model. Applying a suggestion writes a rate for that model and provider kind, prospectively; nothing already recorded is re-priced. With usage access switched on, INFRO also reads token counts by provider key and can say which traffic did not go through it; without it, bypass detection reports "not checked", never "no bypass".

curl https://api.infro.io/v1/rate-findings \
  -H "Authorization: Bearer $INFRO_API_KEY"

When the estimate for a provider leans the same way by more than 5% on at least five of the last seven days, INFRO also raises a cost.drift alert. Your provider's invoice remains the authority; both INFRO figures are for planning, attribution and checking, and INFRO platform fees are never part of either.