# Five LLM spend anomalies and the signal each leaves

URL: https://infro.io/blog/llm-spend-anomalies
Author: INFRO Editorial (https://infro.io/blog/editorial)
Publisher: INFRO
Published: 2026-09-29
Last modified: 2026-09-29
Last reviewed: 2026-09-29 (next review due 2026-10-29)
Topic: Cost and FinOps
Format: Analysis

> Most unexpected LLM spend comes from five patterns: an agent loop or retry storm, longer answers after a provider moves a model name to a new snapshot, a prompt change that stops the provider's cache from matching, requests silently falling back to a dearer model, and traffic reaching your provider account without passing through your gateway. Each shows first in a specific metric — calls per run, output tokens per request, cached share, fallback share, reported versus estimated cost — so watch those per scope against each scope's own usual day, not a fixed dollar threshold.

Key takeaways:
- Compare each scope with its own usual day — the median of the same weekday over recent weeks — not with a fixed dollar threshold.
- Measure distance in robust units such as the median absolute deviation, so one bad day does not widen the band that should have caught it.
- A runaway agent shows as calls per run, not as total requests; group by workflow and run id.
- Longer answers, a lost cache and more fallbacks raise cost with no error at all. Watch output tokens, cached share and fallback share per model.
- Spend your gateway never saw only appears when you compare its estimate with the provider's own reported cost and usage by key.

The FinOps Foundation defines a cost anomaly as spending that is different, usually higher, than normal historical or expected spend, and anomaly management as detecting, alerting on and managing those events quickly enough to limit the damage[1]. For LLM traffic the definition holds, but the causes are specific. Spend rises because something about the requests changed, and most of the time it is one of five things.

This analysis describes each pattern, the signal it leaves first, and a detection method that does not drown you in false alarms.

## Why a fixed threshold fails

"Alert when daily spend exceeds $500" breaks in both directions. A project that normally spends $40 a day can triple and never cross it. A project that normally spends $480 crosses it every busy Monday. Weekends are quieter than weekdays, so a Saturday that looks like a Tuesday is itself unusual.

A better baseline is each scope's own **usual day**: the median of the same weekday over the last few weeks. A better yardstick is that scope's typical day-to-day variation, measured robustly. NIST's handbook defines the median absolute deviation as the median of each value's distance from the median, and notes it is even less affected by extremes in the tail than other measures of spread[2]. That matters here: with a standard deviation, last week's incident widens the band that should catch next week's.

```python
from statistics import median

def usual_day(history: list[float]) -> float:
    """history: the same weekday's spend over the last four weeks."""
    return median(history)

def spread(history: list[float], floor: float) -> float:
    m = median(history)
    mad = median(abs(x - m) for x in history) * 1.4826  # scaled to match a std. dev. on normal data
    return max(mad, 0.10 * m, floor)

def score(projected_today: float, history: list[float], floor: float = 10.0) -> float:
    return (projected_today - usual_day(history)) / spread(history, floor)

# A Tuesday projected at $412 against Tuesdays of $90, $96, $101 and $94:
print(round(score(412.0, [90.0, 96.0, 101.0, 94.0]), 1))
```

Three guards keep the score useful: project today's spend to the end of the day instead of comparing a partial day with a whole one; do not score until enough of the day has passed for the projection to mean something; and ignore changes too small to matter in dollars, however large they are in percent.

[Diagram: Illustrative: a week of estimated spend for one project. The weekend dip is normal; Tuesday's projected spend is several times its usual Tuesday.]

[Diagram: Illustrative: a day's projected spend expressed in units of the scope's usual spread, against warning and critical thresholds. A spend figure only means something relative to that scope's own history.]

## 1. A runaway loop or retry storm

An agent that never reaches its stop condition, a tool that keeps returning an error the model retries, or a client that retries on every timeout all produce the same shape: many more model calls per unit of work. Total request volume may not look extreme if only one workflow is affected, which is why this is easy to miss at the organization level.

**First signal:** calls per run for one workflow, or requests per minute on one key. **Confirmation:** many calls sharing one trace or run id. **Fix:** cap steps, tokens and spend per run, and give each key its own spend limit so one caller cannot consume a whole budget. See [bounding agent loops](https://infro.io/blog/bound-ai-agent-loops).

## 2. Longer answers after a snapshot change

Providers regularly point a model alias at a new snapshot. Your code did not change, your prompts did not change, but answers get longer, and output tokens are the most expensive tokens on the request.

**First signal:** output tokens per request on one model rise. **Confirmation:** the model id the provider names in its response changed on the same day. **Fix:** pin a dated snapshot where the provider offers one, set `max_tokens` deliberately, and re-run your evaluation on the new snapshot before accepting it.

## 3. A lost prompt cache

A cache hit bills the prefix at a fraction of the input rate. A one-line edit near the top of a system prompt, a reordered tool list, or a timestamp added to the first message makes every prefix new, and the input bill jumps with no change in volume. On Anthropic even changing the thinking configuration between requests invalidates cache breakpoints[3].

**First signal:** the cached share of prompt tokens on one model or route drops. **Confirmation:** a prompt or tool-definition change deployed that day. **Fix:** keep stable content first and volatile content last. The economics are in [prompt caching break-even](https://infro.io/blog/prompt-caching-break-even).

## 4. Silent fallback to a dearer model

Failover protects availability. When a primary starts timing out for a slice of requests, those requests quietly move to the next target in the chain. If that target is a larger model, cost per request rises while error rates stay flat.

**First signal:** the share of requests served by a fallback on one route rises. **Confirmation:** failed attempts on the primary recorded against the recovered requests. **Fix:** treat a rising fallback share as an incident on the primary, and put a same-model standby on another account ahead of a different model in the chain.

## 5. Spend your gateway never saw

The four patterns above are visible in your own request records. The fifth is not: traffic that reaches your provider account through another path. A key copied into a notebook, a service calling the provider directly, or a batch job submitted outside the gateway all bill the same account.

**First signal:** the provider's reported cost for a day exceeds your estimate for it, in the same direction day after day. **Confirmation:** usage grouped by API key shows a key your gateway never used. Both OpenAI's usage API and Anthropic's usage report can be grouped by API key[4][5]. **Fix:** rotate the key, move the caller behind the gateway, and give the provider account only the keys you route. See [reconciling estimated spend with provider invoices](https://infro.io/blog/reconcile-llm-spend-provider-invoices).

[Diagram: Five anomaly patterns, the metric that moves first, the scope in which it is visible and the evidence that confirms it.]

## Where to look first

When an alert fires on an organization or project, the useful next question is which model, key, workflow or end customer contributed most of the difference. An alert that names its top contributors turns a dashboard hunt into a single click. Document each resolved anomaly and its cause; the FinOps Foundation's framework makes documenting anomalies and their resolutions part of the process, not an afterthought[1].

## Triage in the first hour

When a spend alert fires, the order of questions matters more than the tooling. A short runbook keeps the first hour focused:

1. **Is it real?** Check whether the projection rests on enough of the day and enough priced requests. A day with many unpriced requests is a lower bound, not an anomaly.
2. **Where is it?** Narrow from organization to project, then to model, key, workflow and end customer. One contributor usually explains most of the difference.
3. **Which of the five?** Compare that contributor's calls per run, output tokens per request, cached share and fallback share with its usual day. The metric that moved names the pattern.
4. **Is it still growing?** A loop or a leaked key keeps spending while you investigate; a snapshot change or a lost cache costs at a steady higher rate. Contain the first kind immediately with a key limit or a budget; the second can wait for a proper fix.
5. **What changed?** Deploys, prompt edits, provider announcements and routing changes on the same day. Most anomalies have a change behind them within a few hours of the first signal.
6. **Write it down.** The cause, the signal that found it, the time to detection and what would have caught it sooner.

The last step is the one that improves detection. A pattern you have documented twice is a pattern worth a dedicated alert.

## Detecting these in INFRO

INFRO's [spend alerts](https://infro.io/docs/observability/alerts) score each scope — the organization, and each project, key, provider connection and model, plus workflows and end customers from the Team plan — against its usual day: the median of the same weekday over the last four weeks, with distance measured in the scope's median absolute deviation, never less than 10% of the usual day. A `spend.anomaly` is a warning at 4 spreads above the usual day and critical at 8. Changes smaller than the larger of $10 and 3% of the organization's average day are never reported, a scope is learning until it has 14 days of history, nothing is scored before a quarter of the UTC day has passed, and a day with more than 20% unpriced requests is not scored. When an organization or project fires, the finding names its three largest contributors.

Patterns 2, 3 and 4 are [traffic changes](https://infro.io/docs/observability/traffic-changes): `traffic.model_changed` when a model on one connection is answered by a model id the provider had not reported in the previous 14 days, and `traffic.regression` when output tokens per request, the cached share of prompt tokens, the fallback share, length-limited answers, p95 latency or an error code's share move far from the usual day. Each finding states the estimated weekly change in provider cost. Pattern 5 needs [reconciliation](https://infro.io/docs/cost/reconciliation) on the Business plan, which raises `cost.drift` when the estimate leans the same way by more than 5% on five of seven days and, with usage access, names traffic that bypassed INFRO.

Alerts describe spend; [budgets](https://infro.io/docs/org/spend-controls) enforce it. Anomaly detection for latency and error rates beyond these signals is on INFRO's roadmap, not available today.

## Sources

1. Anomaly Management capability — FinOps Foundation. https://www.finops.org/framework/capabilities/anomaly-management/ (accessed 2026-09-29)
2. Measures of Scale — Engineering Statistics Handbook — NIST. https://www.itl.nist.gov/div898/handbook/eda/section3/eda356.htm (accessed 2026-09-29)
3. Prompt caching — Anthropic. https://platform.claude.com/docs/en/build-with-claude/prompt-caching (accessed 2026-09-29)
4. How to use the Usage API and Cost API to monitor your OpenAI usage — OpenAI. https://developers.openai.com/cookbook/examples/completions_usage_api (accessed 2026-09-29)
5. Usage and Cost API — Anthropic. https://platform.claude.com/docs/en/manage-claude/usage-cost-api (accessed 2026-09-29)