DocsObservability
Traffic changes
Catch the change before the bill does: new model snapshots, longer answers, lost prompt caching and more fallbacks, with likely cause and weekly cost.
INFRO is in private early access. Running these examples requires an early-access account, an active trial or subscription entitlement, and your own provider connection. Self-serve purchasing is not generally open. See Early access.
Some changes in AI traffic arrive with no error at all. A provider moves a model name to a new snapshot and answers get longer. A small prompt edit stops the provider's prompt cache from matching. A primary starts timing out for a few requests and they quietly move to a dearer fallback. INFRO watches the requests it routes for these silent regressions and raises each one as a finding. The finding names the likely cause, the numbers it fired on, and the estimated change in what your own provider will bill you.
What INFRO cannot see. Traffic changes read request metadata (models, token counts, finish reasons, latency, errors), never prompts or answers, even when payload capture is on. So INFRO cannot tell that answers got worse. That is output quality, and it needs an evaluation tool. INFRO tells you that something changed, and what it costs.
Signals
| Signal | Fires when | What it usually means |
|---|---|---|
| Model answered by a new snapshot | On one provider connection, a model is answered by a model id the provider had not reported in the previous 14 days. The new id must carry at least 20% of today's successful requests, and at least 50 of them. | The provider moved the model name to a new snapshot. |
| Output tokens per request | Up by at least 15% and at least 20 tokens | A prompt or parameter change, or a new snapshot. Cost rises with it. |
| Cached share of prompt tokens | Down by at least 10 percentage points | The start of the prompt changed, so the provider's cache no longer matches. |
| Requests served by a fallback | Up by at least 5 percentage points | The primary is failing or timing out for some requests. |
| Answers cut off by the length limit | Share of finish_reason: length up by at least 3 percentage points | Answers are reaching max_tokens. |
| p95 latency, p95 time to first token | Up by at least 25% and at least 300 ms, two days running | A slowdown on the provider's side, or longer answers. |
| One error code's share | Up by at least 3 percentage points of the requests that reached a provider | For example, more content-filter refusals. |
Refusals INFRO makes itself (a budget, an allowlist, a guardrail) never reached a provider, so they are not counted as a provider's errors. Spend alerts cover those.
Requests answered from the response cache are left out of every signal: a hit says nothing about how your provider is behaving.
How a change is detected
- A new snapshot is an event, not a trend. It is detected per model on each provider connection, because the same model reports different ids on different providers. A request that falls back from one provider to another is therefore not mistaken for a snapshot change. A connection or model needs 7 days of reported model ids before any id can count as new.
- Every other signal is compared with the scope's usual day. INFRO uses the same statistic as spend anomalies: the median of the same weekday over the last four weeks. Distance is measured in the scope's typical day-to-day spread. A finding needs 4 spreads and the minimum change in the table above.
- A slow creep is caught too. A rise of a percent or two a day moves the usual day along with it. So INFRO also compares the median of the last 7 days with the same 7 days four weeks earlier, using the same thresholds.
- Latency must stay high for two days. A single bad day at a provider is an incident for your monitoring. A slowdown that persists is a traffic change. When p95 latency fires, time to first token on the same scope is folded into it.
- A scope is learning until it has 14 days of history, with at least 7 days of 200 or more requests behind the metric. A quiet scope stays silent. Today is scored only once it has 200 requests, and never before a quarter of the UTC day has elapsed.
- Scopes: the organization and each model on each provider connection, on every plan. From the Team plan, also each route and each workflow (your 100 busiest workflows).
What a finding says
For example: Since 2026-09-22, gpt-5 on Production OpenAI has been answered by gpt-5-2026-09-18: 64% of today's requests. Output tokens per request 812 → 1,120 (+38%). Estimated provider cost per request $0.00610 → $0.00740 (+21%), about $410 a week at today's volume. Likely cause: the provider moved this model name to a new snapshot.
- The cause is always stated as likely, never as certain, and the finding lists every number it fired on.
- A snapshot finding compares output tokens per request, cache share, truncation, fallbacks and p95 latency from the 7 days before the change with the traffic since. A comparison with fewer than 200 requests on either side says so. The finding fills in over the next week without sending another notification.
- Estimated weekly impact is the estimated provider cost per priced request, before and after, multiplied by today's volume projected to a full day, times seven. Unpriced requests are left out and counted beside the figure. If more than 20% of today's requests are unpriced, no figure is given.
- The finding links to the matching requests on the Requests screen. For example,
served_modelshows the requests served by the new snapshot, andfinish_reason=lengthshows the truncated ones.
Severity and notifications
| Finding | Severity |
|---|---|
| New snapshot | warning; critical from an estimated $500 a week |
| Metric change, estimated $500 a week or more | critical |
| Metric change, estimated $50 to $500 a week | warning |
| Metric change, under $50 a week | info |
| Metric change, cost impact unknown | info; warning at 8 spreads or more |
With no notification rule, warning and critical traffic changes are emailed to the billing contact. info findings stay in the console. Rules route traffic.model_changed and traffic.regression to email, Slack or a webhook like any other alert event.
- A warning that becomes critical later is sent once more. Nothing is sent a third time.
- A signal that fired on a scope stays quiet on that scope for 21 days, the time the usual day takes to absorb a shift, unless it gets worse again by at least its minimum change. A sustained shift is one finding.
- A new-snapshot finding absorbs metric changes on the same model and connection for 21 days, because it already reports them. A route or workflow carried mostly by a model and connection that already fired is not raised again. An organization-wide change is raised only when no single model, route or workflow explains it.
- At most 5 traffic findings are raised per organization per UTC day, largest estimated impact first. Scope mutes apply as they do to every alert.
In the API
Traffic changes come through GET /v1/alerts with type traffic.model_changed or traffic.regression. figures.metric names the signal. figures.weekly_impact_micros is the estimated weekly impact in micro-USD as a decimal string, or null when unknown. figures.unpriced_requests counts the requests left out. Each request row carries served_model (the model id your provider's reply named) and finish_reason, and GET /v1/requests filters by both.