How do you limit AI usage per customer? Give every customer three caps: a request rate that stops bursts, a quota that bounds volume, and a spend cap in dollars per period that bounds cost across every model they can reach. Check them before the provider is called, using an opaque customer id sent with each request. Two details decide whether the caps work in production: they must hold when many requests arrive at once, and the refusal must tell the user what happened.
Provider rate limits do not help here. OpenAI, Anthropic and Google limit your organization, project or key; none of them knows which of your customers sent a request. Per-customer limits are yours to build or to buy.
Why per-customer limits matter
- Margin. A small group of heavy customers usually carries most of an AI feature's cost. Without a cap, a plan's worst customer can cost more than the plan earns. AI feature unit economics works through an example.
- Abuse. Free trials and free tiers attract scripts that use your product as a free proxy to a model. A per-customer cap bounds the damage of each account; a sign-up limit bounds the number of accounts.
- Runaway automation. A customer's integration in a retry loop, or an agent stuck re-running a step, spends at machine speed. A spend cap stops it in minutes rather than at the end of the month.
- Fairness. Your provider limits are shared. One customer's burst can starve everyone else unless each customer has a share.
The three kinds of cap
| Cap | Unit | Stops | Weakness on its own |
|---|---|---|---|
| Request rate | Requests per second or minute | Bursts, scripts, retry storms | A few very large requests pass |
| Quota | Requests, tokens or units per period | Sustained volume | Cost varies widely by model and request size |
| Spend cap | Estimated dollars per day, week or month | Cost, across every model | Needs a price for every model the customer can reach |
Use all three. The rate limit is cheap and catches the fastest failures; the spend cap is what actually bounds your provider bill. A token quota is useful only where one model serves the feature and is unlikely to change: switch the feature from a small model to a large one and the same quota costs ten times as much.
Identify the customer on every request
Every request needs the id of the customer it serves, decided by your backend, never by the client. Use an opaque id — a database key or a hash — and never an email address or a name, because the id will be stored with every request record.
Providers ask for the same kind of id for abuse detection. OpenAI recommends sending a safety_identifier and suggests hashing the username or email "to avoid sending us any identifying information"1. Anthropic's metadata.user_id should be "a uuid, hash value, or other opaque identifier", up to 512 characters, and must not include a name, email address or phone number2. Sending the same opaque id to the provider and to your own limiter keeps one identity across both.
Decide what happens to requests with no customer id: internal jobs, evaluations, a background summariser. Either route them through a separate key or project with its own budget, or refuse them at the boundary. A cap that silently skips anonymous requests is a cap with a hole in it.
Check before the call, and survive concurrency
A cap checked after the provider call is a report, not a limit. The check belongs in front of the call, and it has to account for requests already in flight.
The naive version reads the customer's spend, compares it with the cap, and sends the request. Under load that fails: twenty requests read the same "$4.90 of $5.00 used", all pass, and the customer ends at $9. The fix is a reservation: atomically add the request's worst-case cost to the customer's in-flight total if, and only if, spend plus in-flight plus this request fits the cap; release the reservation and record the real cost when the request finishes.
-- KEYS[1] = spent total for this customer and period
-- KEYS[2] = in-flight reservations for this customer and period
-- ARGV[1] = cap, ARGV[2] = worst-case cost of this request (same unit, e.g. micro-dollars)
local spent = tonumber(redis.call("GET", KEYS[1]) or "0")
local inflight = tonumber(redis.call("GET", KEYS[2]) or "0")
local cap, cost = tonumber(ARGV[1]), tonumber(ARGV[2])
if spent + inflight + cost > cap then
return 0 -- refuse: no room
end
redis.call("INCRBY", KEYS[2], cost) -- hold the worst case
return 1Redis runs a script like this atomically, so no other command interleaves between the read and the increment3. After the response, decrement the reservation by the worst case and add the real cost to the spent total. Set an expiry on both keys at the end of the period so they reset on their own, and on Redis Cluster give both keys the same hash tag ({cust_42}:spent, {cust_42}:inflight) so the script can touch them together. The worst case is the input token count times the input rate plus the output limit times the output rate; count tokens before you call the API covers getting that number. The 500-request concurrency test shows what this protects against.
Periods, plans and overrides
- Period. Daily caps catch runaway automation fastest; monthly caps match how customers think about a plan. Many products use both: a monthly allowance and a daily ceiling at a few times the average day. State the reset time in UTC.
- Plan defaults. Each plan gets a default cap sized from its price and your target margin.
- Overrides. Enterprise contracts, trials and goodwill gestures need per-customer exceptions. Store them as overrides on a rule, not as a row per customer, so changing a plan's default changes everyone who has no override.
- A zero cap should mean "refuse every priced request", which is useful for suspended or unpaid accounts.
The refusal your users see
When a customer hits a cap, your app needs enough information to say something useful. The refusal from your limiter should carry:
- which limit was hit (rate, quota or spend; daily or monthly),
- when it resets, as a timestamp,
- the customer id it applied to.
Turn that into a sentence in your product: "You've used this month's AI allowance on the Team plan. It resets on 1 November, or you can upgrade now." Return it as its own error type so the client never retries it; a retry loop against a spend cap is one more way to annoy the user.
Alert before the wall
A cap that is hit without warning feels like an outage to the customer. Send an in-product notice at 80% of the allowance, and an internal alert when a customer reaches a cap for the first time, or several days in a row. The second is a sales signal as often as an abuse signal.
Per-customer caps with INFRO
INFRO calls these end-customer caps. Your app sends its own opaque customer id in metadata.user_id or the X-INFRO-User-Id header. A rule sets a default spend cap per day, week or month for the organization or for one project, overrides change it for specific customers, and either the console or your backend through the management API sets them. Before a request reaches your provider, INFRO checks the customer's spend for the period plus the worst-case cost of their requests still in flight, and admits the request only if its own worst case fits; otherwise it answers 402 with code: "end_user_budget", the period and resets_at. Requests with no customer id are served or refused by your choice. Caps are created on the Team plan and above; your organization, project, member and key budgets keep applying on top, and the spend counted is INFRO's estimate of what your provider will bill.
Summary
Put a rate limit, a quota and a dollar cap on every customer, identified by an opaque id your backend sets. Enforce them before the provider call with reservations that hold under concurrency, give plans defaults and contracts overrides, and make the refusal say what was hit and when it resets.