Cost per successful task: the LLM metric that matters

Model comparisons usually start with a price table: input rate, output rate, maybe a blended rate per million tokens. Those numbers are real and they matter, but they measure the wrong thing. Nobody's product sells tokens. It sells a resolved ticket, a correctly extracted invoice, a passing test, an accepted summary. What that unit costs depends as much on how often the model has to try again as on what each try costs.

Three levels of unit cost

The FinOps Foundation distinguishes resource-efficiency unit metrics, such as cost per token, from business unit metrics, such as cost per customer or cost per transaction, and describes unit economics as relating costs to the benefits received¹. For LLM workloads there are three useful levels:

• Cost per token — the provider's rate. Useful for forecasting, useless for comparing models that use different numbers of tokens for the same work.
• Cost per call — tokens times rates for one request. Better, but it treats a call that produced garbage the same as one that produced the answer.
• Cost per successful task — everything spent trying to complete a unit of work, divided by the units completed.

The last one is the only one that goes down when a model gets better at the job.

The formula

cost_per_successful_task = Σ cost of every call in every run  /  number of runs that succeeded

where a run is one attempt to complete one unit of work, and includes:
  first attempt + same-target retries + fallbacks to other targets
  + calls whose output failed validation + every tool-loop step

Two properties matter. Failed runs are in the numerator and not in the denominator, so a model that fails more often pays for it. And every call a run makes counts, including the ones your application threw away, because your provider billed them.

Not every failure is billed. Anthropic does not bill batch requests that errored, were canceled or expired², and OpenAI does not charge for a flex request refused with 429 Resource Unavailable³. But an attempt that timed out on your side after the provider finished it, or an answer your schema validator rejected, was billed in full. The only reliable numerator is the sum of what each call was actually estimated to cost.

Why a lower rate can lose

The general form is cost_per_call × calls_per_run / success_rate. It makes the break-even between two options easy to see: a model three times the price per call wins if it cuts calls per run and raises the success rate enough that the product of those two ratios exceeds three.

Defining success

The metric is only as honest as the success signal. Prefer signals your system already produces for reasons that have nothing to do with the model:

• Deterministic checks: the output parsed, matched the schema, passed the unit tests, compiled.
• Downstream acceptance: the user sent the drafted reply without rewriting it, the ticket closed without escalation, the extracted invoice posted without a correction.
• Sampled human review, when the above do not exist, on a fixed random sample per week.

Avoid a model grading its own output as the only signal. Keep the definition fixed while comparing options; changing what counts as success halfway through a comparison changes the answer.

Instrumenting it

The instrumentation is two identifiers and one join.

1. A workflow name on every call: the stable name of the work, such as invoice-extract.
2. A run id on every call made for one unit of work, including retries and fallbacks.
3. A success record keyed by the same run id, written by the part of your system that knows whether the work succeeded.

The join itself is short. Here it runs against per-run costs from a gateway API and a success table from your own database:

import os
from decimal import Decimal

import requests

def run_costs(workflow: str, period: str) -> dict:
    resp = requests.get(
        "https://api.infro.io/v1/unit-economics",
        params={"by": "workflow", "period": period},
        headers={"Authorization": f"Bearer {os.environ['INFRO_API_KEY']}"},
        timeout=30,
    )
    resp.raise_for_status()
    for row in resp.json()["data"]:
        if row["group"] == workflow:
            return row
    raise LookupError(f"no runs for {workflow} in {period}")

def cost_per_successful_task(workflow: str, period: str, successful_runs: int) -> dict:
    row = run_costs(workflow, period)
    total = Decimal(row["estimated_cost"])
    return {
        "workflow": workflow,
        "runs": row["runs"],
        "successful_runs": successful_runs,
        "success_rate": successful_runs / row["runs"] if row["runs"] else None,
        "cost_per_successful_task": total / successful_runs if successful_runs else None,
        "unpriced_share": row["unpriced_share"],  # a partial figure must say so
    }

The total here covers every request tagged with the workflow, including requests without a run id; count only runs that carry one if your success table is keyed by run.

Reading the number

• Look at the distribution, not only the mean. Median and p90 cost per run show whether a few runaway runs dominate. An agent with a low median and an expensive tail needs a step or spend limit, not a different model.
• Report the unpriced share beside the figure. If some calls ran on a model with no known rate, the cost is a lower bound; say so rather than publishing it as whole.
• Compare like with like. Same task mix, same period, same success definition. Seasonal changes in input length move cost per task without anything being better or worse.
• Revisit when rates change. A provider price change, a new snapshot or a new negotiated rate changes the numerator. The success rate does not change with price; recompute the product.

Using it to choose

Cost per successful task turns model selection, prompt changes and routing into one comparable number, but only if every option is measured the same way.

• Run options on the same sample. A fixed, labelled set of real inputs, large enough that a few percentage points of success rate are not noise. See evaluating LLMs on real workloads.
• Include the fallback path. If a route falls back to a second model when the first fails, the route's cost per successful task includes both. Measure the route, not the models in isolation.
• Price with the rates you pay. A negotiated rate on one provider can reverse a comparison made at list prices. Use your own rates where you have them.
• Account for the failures you do not see. If failed runs go to a person, add that handling cost per failure to the comparison, separately from provider cost, so the two are not blended into one figure.
• Recheck after changes. A new snapshot, prompt or effort setting changes both the numerator and the success rate. Re-run the sample before and after.

A useful habit is to publish the three inputs next to the result: cost per call, calls per run, success rate. When cost per successful task moves, the inputs say which lever moved it.

Cost per run in INFRO

INFRO records the cost dimensions; your application records success. Send metadata.workflow and metadata.trace_id on every call (or the X-INFRO-Workflow and X-INFRO-Trace-Id headers), and INFRO treats one (workflow, trace_id) pair as a run. Unit economics, on every plan, returns per group the requests, runs, total estimated cost, cost per request, and the median and p90 cost per run, with unpriced_requests and unpriced_share beside every row:

curl "https://api.infro.io/v1/unit-economics?by=workflow&period=2026-08" \
  -H "Authorization: Bearer $INFRO_API_KEY"

Requests without a trace id count toward cost and cost per request but not toward runs, median or p90. Retries your application sends are new requests with their own cost, and count toward the run when they carry the same trace id. Failover inside the gateway is recorded on the request that finally served, with each failed attempt listed; a charge your provider makes for an attempt that failed on INFRO's side, such as a timed-out call it still completed, is outside the estimate, as spend controls states. Group by=feature, end_user or cost_center for the same view along other dimensions. All figures are estimates of what your own providers bill you; INFRO platform fees are not included. INFRO sees request metadata, not answers, so it does not judge whether a run succeeded — that join stays yours.