Where AI spend sits in the Framework
The FinOps Framework is built from principles, personas, phases, domains, capabilities and scopes, and it describes itself as flexible and non-prescriptive: you start where the need is greatest and grow in scale and maturity.1 AI appears there as a Technology Category beside Public Cloud, SaaS, Data Platform, Private Cloud, Licenses and Data Center.1 A Scope is a segment of spend aligned to a business construct such as a product or cost center, and it can draw from several categories.1
That matters for model spend because it rarely lives in one category. The FinOps for AI paper lists direct provider APIs (per token, under AI as token-based SaaS), cloud provider model services (on the cloud bill, under Public Cloud), embedded AI in SaaS products, GPU hours, and owned infrastructure as channels that can all feed one AI Scope.2 The same paper says most of the practice carries over. Cost is still price times quantity, cloud-billed AI shows up in FOCUS-format data, and tagging, allocation and commitments still apply. What changes is the unit (tokens, calls, provisioned throughput), volatile pricing, new stakeholders, and thin allocation data: direct model provider bills often lack native tagging, so allocation needs engineering instrumentation.2
This guide covers one slice: model spend on provider accounts and keys your company holds, where the provider invoices you directly and your gateway sits in the request path. It walks the capabilities that most often break first.
The three ledgers you reconcile
On customer-held keys, no single system holds the whole truth. You work with three records:
- The provider's invoice and cost API. This is what you owe. It is accurate and late, and it is usually grouped by provider project or key, not by your product features.
- The cloud bill, for anything bought through a cloud provider's model service. This is already in your cloud FinOps pipeline, often in FOCUS format.2
- The gateway request log. This is timely and carries your own dimensions, but its cost is a rate-table calculation made at request time, not an invoice.
An INFRO-style gateway makes the third ledger possible without changing the first: you connect provider accounts you already hold, your providers invoice you directly, and the gateway records what each request used and estimates what your provider will bill.3 The job of the capability map is to decide which ledger answers which question.
| Question | Best ledger | Why |
|---|---|---|
| What do we owe this month? | Provider invoice or cost API | It is the invoiced amount |
| Which feature or customer drove it? | Gateway request log | Only the application knows the feature and customer |
| Did something spike today? | Gateway request log | Invoices arrive after the fact |
| Is our estimate right? | Both, compared daily | The gap is a rate or coverage problem to fix |
The capability map
The Framework groups capabilities under four domains: Understand Usage & Cost, Quantify Business Value, Optimize Usage & Cost, and Manage the FinOps Practice.1 The map below covers the capabilities a team extending cloud FinOps to model spend touches first.
| Capability (domain) | AI data source | Gateway field or endpoint |
|---|---|---|
| Data Ingestion (Understand) | Provider cost APIs, cloud bill, request log | FOCUS export of request-level usage |
| Allocation (Understand) | Request metadata, key and project structure | workflow, feature, user_id, promoted tags, project, key, environment |
| Anomaly Management (Understand) | Daily spend by scope | Spend anomaly, forecast and drift events |
| Planning & Estimating (Quantify) | Token counts, rate tables | Cost estimate before the request |
| Forecasting (Quantify) | Units of work, calls per unit, tokens per call | GET /v1/unit-economics drivers |
| Budgeting (Quantify) | Forecast, hard caps | Budgets per organization, project, member and key |
| Unit Economics (Quantify) | Cost divided by outcome | cost_per_run, per feature, per end user |
| Rate Optimization (Optimize) | Contract rates, committed spend | Negotiated rates and commitment tracking |
| Invoicing & Chargeback (Manage) | Allocated cost by cost center | Cost centers with rules and period history |
Everything below follows from the first rows. If the request log cannot carry your dimensions, none of the later rows work.
Allocation: put the dimensions on the request
The Allocation capability asks for an allocation strategy, a tagging and hierarchy strategy, and a shared cost strategy, plus validation that the allocation is complete.4 It also describes shifting left: ownership and metadata enforced where a resource is created, not reconstructed in a later tagging cleanup.4 For AI, the "resource" is a request, so shifting left means the calling code sends the fields.
A gateway can carry them as reserved metadata. In INFRO's contract the reserved keys are workflow, trace_id, step, feature and user_id, and free-form tags; every request already carries project, gateway key, environment, model, provider and route.5 Values are validated: a malformed workflow, step or feature is refused with 400 invalid_attribution rather than dropped.5
from openai import OpenAI
client = OpenAI(base_url="https://api.infro.io/v1", api_key="INFRO_API_KEY")
resp = client.chat.completions.create(
model="openai/gpt-5.6-sol",
messages=[{"role": "user", "content": "Summarize this ticket"}],
extra_body={
"metadata": {
"workflow": "support-agent",
"trace_id": "run_8f2c",
"feature": "summarize",
"user_id": "cust_123",
"tags": {"team": "support"},
}
},
)Two behaviours shape your tagging policy. First, attribution applies from the request that first carries it; earlier requests stay unattributed and nothing is backfilled or guessed.5 Second, to group or match on a free-form tag you promote its key (up to five per organization), and only requests metered after promotion carry it; grouping by an unpromoted key returns 400 tag_not_promoted.5 Decide your five keys (team, cost code, product line) before launch, not after the first quarterly review.
For chargeback, cost center rules book estimated spend by first-match order on project, key, member, environment, workflow, feature, promoted tag or end-user prefix. What matches nothing is shown as Unallocated and never spread across other centers.6 Shared costs, such as a retrieval index used by three teams, can be split by fixed weights or in proportion to direct spend, and split lines are labelled as allocated.6 That is the Framework's shared cost strategy made explicit: you decide per shared item whether to allocate it or fund it centrally, the "informed ignore" choice the capability page describes.4 Rules are versioned by period, so a reorganization in October does not rewrite what September said.6 Showback mechanics are covered in Showback and chargeback for AI spend.
Anomaly management: scope the baseline
The Framework's Anomaly Management capability says detection should look at subcategories, not only aggregate usage, and that allocation metadata is critical both to detect anomalies and to know who can resolve them.7 Its Walk level includes forecast-breach alerts and automatic routing to responsible teams; Run adds integration with ticketing and event systems.7 It also warns that anticipated spikes, like a new environment launching, will trigger alerts, so owners should announce them and document the outcome.7
For model spend, the useful signals are the ones a daily invoice cannot give. INFRO's alert events illustrate the set:8
spend.anomaly: a scope is on course to spend far more than its usual day.spend.forecast_over_budget: at the current rate a budget or key limit will be exceeded by at least 5% before it resets, with at least three days of a monthly period left.cost.drift: on at least five of the last seven days the estimate leaned the same way, by more than 5%, from the cost the provider's own API reported. This needs cost reconciliation.unpriced.traffic: for more than six hours, at least 5% of successful requests (and at least 20) had no price, so every spend figure is a lower bound.traffic.model_changedandtraffic.regression: output tokens per request, cache share, fallback share or latency moved with no error.
Map these to the Framework's KPIs rather than inventing new ones: count of anomalies, mean time to detect, mean time to notify the owner, and unresolved duration.7 Allocation fields decide the owner, so the alert for feature=summarize goes to the team that owns it. See five LLM spend anomalies for the shapes these take.
Forecasting: drivers, not trend lines
The Framework describes forecasting as a model of future spend for a scope, built from historical spending, planned changes and lifecycle effects, feeding Budgeting. Its Run level combines rolling, trend-based and driver-based models and aligns forecasts to the same allocation constructs used for reporting.9 A pure trend line fails on AI spend because a model swap, a longer prompt or a lost cache changes the rate without changing the volume.
A driver-based model splits spend into units of work, calls per unit, and tokens per call, each owned by someone. The request log gives you the last two; the product plan gives you the first.
Two inputs make this more accurate. Rates should be your contract rates, not list rates; INFRO lets you enter negotiated rates so estimates use them before the vendor's list rate. Committed-spend agreements can be recorded and tracked against reported cost with an end-of-term forecast, which feeds the rate side of the model. For planning before launch (Planning & Estimating), a cost estimate for a request on your own provider account gives the per-call figure without sending traffic. Budgets then turn the agreed forecast into a ceiling at organization, project, member or key level, which is where the Framework hands Forecasting off to Budgeting.9 A longer method is in AI budget planning for 2027.
Unit economics: from cost per token to cost per outcome
The Framework separates resource efficiency metrics (cost per token) from business unit metrics (cost per tenant, per case resolved), and says early generative AI unit economics usually starts at cost per token and moves toward cost per assist, per agent action or per case deflected.10 The FinOps for AI paper adds two rules worth adopting: measure the use case rather than the token, and report cost per successful outcome as a range.2
The denominator is what your application records. The numerator is allocated estimated cost. INFRO's unit economics endpoint groups by workflow, feature, end_user or cost_center and returns requests, runs, estimated cost, cost per request, and median and p90 cost per run, where a run is one (workflow, trace_id) pair.11
curl "https://api.infro.io/v1/unit-economics?by=feature&period=2026-10" \
-H "Authorization: Bearer $INFRO_API_KEY"Read the unpriced_requests and unpriced_share fields beside every figure. Cost per request divides by priced requests only, so a partial number never reads as a whole one, and requests without the dimension appear in a separate unattributed row.11 Percentiles matter because agent runs have fat tails; a median alone hides the runs that blow the budget. Quality still needs its own measure: a cheap run that fails is not a cheap outcome, as covered in cost per successful task.
Landing gateway data beside cloud cost
The FinOps for AI paper notes that cloud-billed AI already arrives in FOCUS format.2 To put gateway-observed spend in the same dataset, INFRO exports a FOCUS v1.4 Cost and Usage dataset from GET /v1/usage/focus, broken down by project, key, environment, model, provider, connection, route and your metadata tags.12
The important design decision is where the estimate goes. FOCUS requires BilledCost to reflect amounts as invoiced by the invoice issuer, not estimated values. Because the provider, not INFRO, invoices you, the per-request estimate is written to ListCost and ContractedCost, and BilledCost and EffectiveCost are populated only from figures your provider's own cost API reported.12 Two modes follow:
cost_basis=estimated(the default) covers every provider-day, omitsBilledCostandEffectiveCost, and reports conformance as partial.12cost_basis=reconciledrequirescharge_period=day, includes only provider-days with a reported figure, and needs a billing credential, available for OpenAI and Anthropic only.12
curl "https://api.infro.io/v1/usage/focus?cost_basis=reconciled&charge_period=day" \
-H "Authorization: Bearer $INFRO_API_KEY"If your FinOps platform sums BilledCost, expect gaps for providers without a cost API, and do not backfill them with estimates. Keep estimated and reconciled rows distinguishable in your warehouse. Reconciliation mechanics are in Reconciling estimated LLM spend with provider invoices.
A rollout order by maturity
The Framework's Crawl, Walk, Run model suits this extension.1 A workable order:
- Define the AI Scope and its owners, including who fills the Engineering persona for AI products.
- Fix the allocation taxonomy: workflow names, feature names, and the five promoted tag keys.
- Make the metadata a code-review requirement for any new call site.
- Book Unallocated to a named owner and track its share weekly.
- Turn on daily reconciliation where the provider offers a cost API, and review drift findings.
- Route spend anomaly and forecast alerts to the owning team, not only to the central FinOps channel.
- Build the driver-based forecast for the top three workflows and report it as a range.
- Publish cost per run or per outcome with the unpriced share beside it.
Start with the metadata. Every later capability, from anomaly routing to chargeback to unit cost, reads it.
