# LLM spend limits compared: who caps spend on your own keys

URL: https://infro.io/blog/llm-spend-limits-compared
Author: INFRO Editorial (https://infro.io/blog/editorial)
Publisher: INFRO
Published: 2026-10-13
Last modified: 2026-10-13
Last reviewed: 2026-10-13 (next review due 2026-11-12)
Topic: Cost and FinOps
Format: Comparison

> Spend limits differ on what they count, not just where you set them. Vercel AI Gateway's budgets do not count spend on your own provider keys; Cloudflare AI Gateway's do, for models with known pricing; OpenRouter counts your own-key spend only when you turn it on; LiteLLM and Kong count whatever you route through them. Provider-native limits at OpenAI and Anthropic block on your account itself, while AWS and Azure budgets notify after the fact. Most gateway limits are recorded after a request completes, so a burst can pass them.

Key takeaways:
- Ask first whether a limit counts spend on your own provider keys: Vercel's budgets do not, Cloudflare's do for priced models, and OpenRouter's do only when you turn it on.
- Most gateway limits record cost after a request completes and enforce on the next one; the vendors that say so also say a burst can briefly pass the limit.
- Check what happens to a model with no known price: several products leave it uncounted, which turns a hard cap into a cap on priced models.
- Provider-native limits at OpenAI and Anthropic stop traffic on the account itself; AWS and Azure budgets notify hours after the usage.
- Layer limits: end-customer and key caps for blast radius, an organization cap in the gateway, and the provider's own limit as the backstop.

Every AI gateway and every provider console now has something called a budget or a spend limit. The label hides large differences. One limit counts only the spend you route through the vendor's own account and ignores your own provider keys. Another counts your keys but only for models it has a price for. A third refuses the next request; a fourth sends an email a day later. If you are setting a limit to stop a runaway agent at 3 a.m., those differences decide whether it works.

This comparison reads each product's own documentation, opened on 2026-10-13, against six questions. It compares documented behaviour only. Nothing here was load-tested, and no claim is made about how any product behaves under concurrency beyond what its own documentation says.

## Six questions to ask of any spend limit

[Diagram: The questions this comparison asks of each product.]

1. **Does it count spend on my own provider keys?** If you bring your own key (BYOK) and the limit counts only what you buy through the gateway vendor, it bounds the wrong number.
2. **Does it refuse, or only notify?** An alert is for a person. A refusal is for a loop that will not stop on its own.
3. **When is spend counted?** Either before the call, by reserving a worst case, or after it completes, enforced on the next request. The second is exact about cost and can be passed by whatever is in flight.
4. **At which scopes?** Organization, project, key, member, end customer, model, provider.
5. **What happens to a model with no known price?** If it is simply not counted, a hard cap becomes a hard cap on priced models.
6. **When does it reset?** Calendar month, a billing day, a rolling window, or never.

## Gateways

### Vercel AI Gateway

Vercel's budgets cap spend at four scopes: team, project, API key and user (a team member, across the keys attributed to them), and they stack, so a request must pass every budget in scope. Periods are daily, weekly, monthly (resetting on the first of the month, midnight UTC) or none. A request over a budget is refused with `402` and type `quota_for_entity_exceeded`[1].

Two sentences on the same page matter most. First: "A budget is a soft cap, not a hard limit. The check runs at the start of each request, so the request that crosses the limit still completes and total spend can end up slightly over the budget." Second: "BYOK spend isn't counted in budgets. Spend that draws on your own provider keys is metered separately and doesn't count toward any limit"[1]. Vercel's pricing page adds that BYOK needs purchased AI Gateway credits, and that when a request with your credentials fails, Vercel retries it with its own system credentials and charges that fallback to your credits balance[2]. The page does not say how models without a price are treated.

### Cloudflare AI Gateway

Cloudflare's spend limits do count your own keys: they apply to requests through Unified Billing and through BYOK keys, for "models with known pricing". Each rule is a dollar budget over a rolling or fixed window, optionally split or filtered by provider, model or custom metadata such as a user, team or application id, up to 20 rules per gateway. Over the limit, requests get `429 Too Many Requests`, or go to a fallback model through a dynamic route[3].

Cloudflare is explicit about timing: limits are "eventually consistent", a request's cost is recorded after it completes, and "a burst of concurrent requests can briefly exceed the limit before enforcement catches up". It calls cost tracking "a best-effort estimation" and points to the provider's dashboard for exact billing. The page does not describe what happens to a model whose price it does not know[3].

### OpenRouter

OpenRouter has three layers: a pooled workspace budget, guardrail budgets per member or per API key that stack with it, and per-key credit limits; "the first limit reached blocks the request". Guardrails reset daily, weekly or monthly[4].

By default these count only OpenRouter credit spend. Each guardrail and each workspace has its own "Include BYOK spend" toggle; when it is on, your own-key traffic is counted at "OpenRouter's recorded inference-cost value for each request", which may differ from what your provider invoices[4]. The BYOK guide adds that, by default, when all your keys fail or hit a rate limit, OpenRouter falls back to its shared endpoints, which spend OpenRouter credits; a per-provider setting can turn that off[5]. Treatment of unpriced models and the timing of enforcement are not documented on these pages. Stripe agreed to acquire OpenRouter (announced 2026-08-19); as of 2026-10-13 neither company had published a completion notice[6].

### LiteLLM proxy

LiteLLM is software you run, so every key it uses is yours and every request through it can count. Its proxy offers budgets for the whole proxy, teams, team members, internal users, virtual keys and customers (the `user` field your application sends, global per deployment); per-model budgets on keys are an enterprise feature. `budget_duration` sets the reset interval, and with none set the budget never resets. Over budget, requests are rejected with an error[7].

It is the most explicit of the gateways about the two harder questions. Budget reservation, which estimates cost from `max_tokens` or the model's limits before the call, is on by default, and the docs warn that disabling it "can allow concurrent requests to exceed a configured budget". A model is treated as free only when both its cost fields are explicitly zero; unset prices still go through budget checks. Without a database, the global budget check is skipped, and for routes without token pricing, enforcement falls back to recorded spend[7].

### Portkey, now Prisma AIRS AI Gateway

Portkey's budget limits set a cost or token limit on a provider integration, with no reset, a weekly reset (Sunday, midnight UTC) or a monthly one (the first, midnight UTC). Reaching the limit expires the key to stop further use; email alerts can fire before. The page says limits apply to providers and models with Portkey pricing support, and requests to models without it do not count. It describes the feature as available on the Enterprise plan and to select Pro customers, and carries the notice that Portkey is now Prisma AIRS AI Gateway[8].

### Kong AI Gateway

Kong's AI Rate Limiting Advanced plugin has a `cost` strategy: you configure `input_cost` and `output_cost` per million tokens, and the plugin limits on that arithmetic, per consumer, consumer group, IP, header, path, model or provider, returning `429` at the limit. Because you supply the rates, the counted figure is whatever you decide it should be, including your negotiated rates. Two documented details shape how it behaves: cost "is only reflected during the next request", and the cluster and Redis strategies are accurate only with `sync_rate` set to `0`. The plugin is part of Kong's AI Gateway Enterprise offering[9].

## Provider-native and cloud limits

### OpenAI

OpenAI separates spend alerts, which "do not enforce a cap", from hard limits, which return `429` with `organization_spend_limit_exceeded` or `project_spend_limit_exceeded`. An organization hard limit covers every project; a project hard limit covers traffic billed to that project. Limits reset with the monthly cycle, and "enforcement is not instantaneous, so recorded spend can slightly exceed the configured amount". Per-key or per-user limits are not documented[10].

### Anthropic

Every Anthropic usage tier below Custom carries a monthly spend cap; over it, requests return `429` with `enforced_spend_limit_reached` until 00:00 UTC on the first of the next month. You can set your own organization limit below the tier's cap, and when usage reaches it requests return `400 invalid_request_error`[11]. Workspaces take their own spend limit, which must be lower than the organization's, plus email notifications at amounts you choose[12].

### AWS and Azure budgets

Neither cloud's budget is in the request path. AWS Budgets notifies on actual or forecast cost and can apply actions such as an IAM policy that denies provisioning; its information "is updated up to three times a day", and AWS warns that costs can exceed a threshold before you are notified[13]. Azure Cost Management budgets are explicit: "Resources aren't affected, and your consumption isn't stopped", with costs evaluated every 24 hours[14]. Azure API Management's `llm-token-limit` policy is in the request path, but it limits tokens, not dollars, returns `403` when a quota period is used up, counts per gateway rather than across an instance, and notes that concurrent requests can temporarily exceed it[15].

## Side by side

|  | Counts spend on your own keys | At the limit | Scopes | Models with no known price | Resets |
| --- | --- | --- | --- | --- | --- |
| Vercel AI Gateway | No | 402; a soft cap | Team, project, API key, user | Not documented | Daily, weekly, monthly, none |
| Cloudflare AI Gateway | Yes, for models with known pricing | 429, or fallback route | Per gateway; split by provider, model, metadata | Not documented | Rolling or fixed window |
| OpenRouter | Only with "Include BYOK spend" on | Blocked | Workspace, member, API key | Not documented | Daily, weekly, monthly |
| LiteLLM proxy | Yes | Rejected with an error | Proxy, team, member, user, key, customer | Counted unless priced at zero | Any interval, or never |
| Portkey / Prisma AIRS | On provider integrations | Key expired | Integration | Not counted | None, weekly, monthly |
| Kong AI Rate Limiting Advanced | Yes, at costs you configure | 429 | Consumer, group, IP, header, path, model, provider | You set every cost | Window-based |
| OpenAI | Your account | 429 (hard limit) or alert | Organization, project | Not applicable | Monthly |
| Anthropic | Your account | 400 at your limit; 429 at tier cap | Organization, workspace | Not applicable | Monthly |
| AWS Budgets | Your account | Notify; optional actions | Account, filters | Not applicable | Budget period |
| Azure budgets | Your account | Notify; consumption not stopped | Management group to resource group | Not applicable | Monthly, quarterly, annual |

Table: From each vendor's documentation on 2026-10-13. "Not documented" means the cited page is silent, not that the behaviour is absent.

[Diagram: The three questions that decide whether a limit stops a runaway workload on your own keys.]

## When the count happens

The products split into two designs, and the documentation says which is which.

[Diagram: Recording after the response is exact about cost; reserving before the call is exact about the ceiling.]

Recording after completion is what Cloudflare, OpenAI and Azure's token policy describe, and each says concurrent or in-flight traffic can pass the limit; Kong applies a request's cost on the next request. Vercel checks at the start of each request, and the request that crosses the limit still completes. Reserving a worst case before the call is what LiteLLM describes as its default. The cost of reserving is refusing a request that would have fitted, and needing a price for every model; the cost of recording afterwards is an overshoot that grows with concurrency and with how long requests run. For a monthly team budget, the overshoot rarely matters. For a $5 daily cap on one of your customers, or an agent that can open fifty streams at once, it can be most of the cap.

## Spend limits in INFRO

INFRO is a gateway over provider accounts you connect, so every request it serves runs on your own keys and every request is counted. What it counts is an estimate of what your provider will bill you: your negotiated rate where you have entered one (custom rates are included from the Team plan), else the vendor's list rate. It is not a balance INFRO holds, and it is not the provider's invoice.

- **Plans.** Budgets are on every plan: Developer, Startup, Team and Business.
- **Scopes.** Organization, project and member budgets run monthly from the organization's budget cycle day. A key takes a daily, weekly or monthly limit, or a lifetime ceiling that never resets. Every applicable blocking limit must allow the request. Owners, admins and finance members set budgets; only the owner sets the organization-wide limit.
- **Alert or block.** Each budget has `alert` or `block` behaviour with threshold fractions. Alert-only budgets notify and never refuse. Key limits always block.
- **Before the call.** A blocking budget admits a request only if its worst-case estimated cost fits the room left, counting recorded spend plus the worst case of every request still in flight across all of INFRO's infrastructure. A refusal is `402` with type `budget_exceeded` and a code naming the ceiling: `organization_budget`, `project_budget`, `member_budget`, `key_budget`, `key_lifetime_limit` or `end_user_budget`.
- **Unpriced models.** Under a blocking budget, a request that could reach a model with no price is refused with `402 unpriced_model_under_budget` by default. Enter your rate under Prices, or the owner can allow unpriced traffic, which blocking budgets then do not govern.
- **End customers.** From the Team plan, one cap per end customer of your product (the id your app sends in `metadata.user_id`), as an organization rule and optionally per project, daily, weekly or monthly, with overrides your backend sets through the management API. The refusal carries `end_user_id`, `period` and `resets_at` so your app can tell its user when they can continue.

On concurrency: in a test on INFRO's sandbox on 2026-09-25, a $1 organization cap held in 12 of 12 runs at 50, 200 and 500 concurrent requests, plain, streaming with client disconnects, multi-level and with injected failures. The most recorded was $0.9720, every request was served and recorded or refused with `402 budget_exceeded`, and no reservation was left behind. That was a sandbox with a test connection named `echo`, not production traffic; the cap counts INFRO's estimate, so a provider that bills above the reserved worst case passes it by that excess; a model with no known price is refused while the cap applies unless unpriced traffic is allowed; and admission serializes under load (p95 1.6 to 4.5 seconds, 42 seconds for streams at 500). The [full method, results and limits](https://infro.io/blog/ai-spend-cap-concurrency-test) are published. No other product was tested.

## Layer the limits

No single limit does every job. A sensible stack, from the narrowest to the last word:

[Diagram: Each layer contains a different failure; the provider's own limit stays as the backstop.]

- [ ] An end-customer cap for each user of your product, so one customer's loop cannot spend another's share.
- [ ] Key and project limits for workloads: agents, batch jobs, evaluations, a contractor's key.
- [ ] An organization cap in the gateway, blocking, across every provider you route to.
- [ ] Each provider's own hard limit on its account, set above the gateway's cap, for traffic that never passed through the gateway and for anything the gateway's estimate misses.
- [ ] Cloud budgets and the invoice, reviewed monthly, as the record of what was actually billed.

Whichever products you use, check the six questions against their documentation on the day you decide, and test the one that matters most: fire a few hundred requests at a small cap and see what is recorded.

## Sources

1. AI Gateway Budgets and Spend Limits — Vercel. https://vercel.com/docs/ai-gateway/observability-and-spend/budgets (accessed 2026-10-13)
2. AI Gateway Pricing — Vercel. https://vercel.com/docs/ai-gateway/pricing (accessed 2026-10-13)
3. Spend limits — Cloudflare. https://developers.cloudflare.com/ai-gateway/features/spend-limits/ (accessed 2026-10-13)
4. Spend controls — OpenRouter. https://openrouter.ai/docs/guides/best-practices/spend-controls (accessed 2026-10-13)
5. BYOK — OpenRouter. https://openrouter.ai/docs/guides/overview/auth/byok (accessed 2026-10-13)
6. Stripe agrees to acquire OpenRouter to help businesses optimize token routing and usage — Stripe. https://stripe.com/newsroom/news/stripe-agrees-to-acquire-openrouter (accessed 2026-10-13)
7. Budgets, rate limits — BerriAI. https://docs.litellm.ai/docs/proxy/users (accessed 2026-10-13)
8. Budget limits — Portkey. https://portkey.ai/docs/product/ai-gateway/virtual-keys/budget-limits (accessed 2026-10-13)
9. AI Rate Limiting Advanced — Kong. https://developer.konghq.com/plugins/ai-rate-limiting-advanced/ (accessed 2026-10-13)
10. Spend limits — OpenAI. https://developers.openai.com/api/docs/guides/spend-limits (accessed 2026-10-13)
11. Rate limits — Anthropic. https://platform.claude.com/docs/en/api/rate-limits (accessed 2026-10-13)
12. Creating and managing workspaces in the Claude Console — Anthropic. https://support.claude.com/en/articles/9796807-creating-and-managing-workspaces-in-the-claude-console (accessed 2026-10-13)
13. Managing your costs with AWS Budgets — Amazon Web Services. https://docs.aws.amazon.com/cost-management/latest/userguide/budgets-managing-costs.html (accessed 2026-10-13)
14. Tutorial: Create and manage budgets — Microsoft. https://learn.microsoft.com/en-us/azure/cost-management-billing/costs/tutorial-acm-create-budgets (accessed 2026-10-13)
15. llm-token-limit policy — Microsoft. https://learn.microsoft.com/en-us/azure/api-management/llm-token-limit-policy (accessed 2026-10-13)