How do you prevent abuse of an AI feature? Make every request belong to an identified end user, cap what each user can spend or generate per period before the provider is called, add friction to sign-up and give new accounts small limits, fix the model and output length on the server, refuse prompts that are off the feature's purpose, and watch per-user behavior. Without those, an AI feature is a general-purpose model API that anyone can script, and you pay for every token.
How a feature becomes a free LLM API
The pattern is simple. A product adds an assistant, a summarizer or a "write it for me" button. The browser sends text to an endpoint; the server adds a system prompt, calls a model with the company's key, and returns the answer. If that endpoint accepts any text, from any signed-in user, with no limit beyond the provider's own rate limit, it is functionally a model API. Three kinds of abuse follow:
| Abuse | What it looks like | Why it happens |
|---|---|---|
| Account farming | Many free or trial accounts, often from the same devices or networks, each used to its limit | Free capacity multiplied by accounts is a lot of free inference |
| Scripted use | Requests at machine timing, around the clock, from a client that is not your front end | Someone wrapped your endpoint as an API for their own product |
| Output harvesting | Templated prompts covering a topic systematically, long answers, requests for step-by-step reasoning | Your feature's outputs become training data for someone else's model |
The third is the product-scale version of what model providers describe at their own scale. Anthropic said distillation campaigns against Claude used networks of fraudulent accounts, one proxy network managing more than 20,000 at once, and that its response included stronger verification for educational, research and startup accounts1. Providers have classifiers and fingerprinting for this1; a product team usually has neither, which makes a lightly protected AI feature an easier target than the provider itself. What is an AI distillation attack explains the technique.
OWASP files all three under LLM10 Unbounded Consumption, which covers denial of wallet, functional model replication through API outputs, and resource exhaustion; OWASP LLM10 explained maps each to controls2.
Abuse by your users is abuse on your account
Your users' prompts reach the provider under your key, so the provider sees your organization doing whatever they do. Anthropic says warnings for API customers are linked to thresholds of violative behavior across the entire API account3, and its Usage Policy forbids providing access to Claude through unauthorized means and coordinating activity across many accounts to avoid detection4. OpenAI recommends sending a safety identifier, a stable per-user string such as a hash of the user's ID, so it can monitor and detect abuse and help your team respond faster5. Do it; it lets the provider see one bad user rather than one bad company.
Controls, in the order they pay off
1. Identify every request's end user
Every call to a model should carry an end-user id from your own authentication, server-side, never from a client header the user controls. Without it, every other control applies to everyone at once. OpenAI's guidance is that users should generally register and log in to use a service, and that a session ID can stand in for a preview offered to signed-out visitors5. Use an opaque id, not an email address.
2. Cap spend or tokens per end user per period
A rate limit bounds speed; a cap bounds cost. Set a default per end user per day or month, raise it by plan, and enforce it before the provider is called, counting requests already in flight so parallel calls cannot slip past together. OWASP lists rate limiting and user quotas, and limiting the resources any single user can consume, among its mitigations2. Per-customer AI usage limits covers the design, including the refusal message users see.
3. Put friction where accounts are created
Sign-up is where farming is cheapest to stop. Verify email, block disposable domains, rate-limit sign-ups per network and device, and ask for a payment method before granting meaningful free usage. OpenAI notes that requiring a payment card or an ID card reduces risk further5. Then give new accounts small limits that grow with account age, payment history and normal use. The CISA, NSA and FBI advisory on distillation lists new accounts that go straight to maximum usage, and shared accounts used from many IPs and user agents, among the indicators AI companies should watch6; the same patterns apply to your own sign-ups, although that reading is ours rather than the advisory's.
4. Limit input and output
OpenAI's guidance is that limiting input length helps against prompt injection and limiting output tokens reduces the chance of misuse5; OWASP lists input size limits2. A summarizer does not need 100,000-token inputs or 8,000-token answers. Set max_tokens (or the provider's equivalent) to what the feature needs, and reject inputs above a size the feature would never see.
5. Fix the model and parameters on the server
Never let the client choose the model, the system prompt, the temperature or the tools. An endpoint that passes a model field through is an invitation to call your most expensive model. Keep one or two models per feature in server configuration, and refuse anything else.
6. Refuse prompts that are off the feature's purpose
A meeting summarizer has no reason to answer questions about chemistry or to write code. Narrow the system prompt to the task, shape the input (a document plus an instruction from a fixed set, rather than free text), and check that the request is on purpose before or after the model call, with a small classifier or a cheap model. Refuse requests to reveal the system prompt or full reasoning. These checks are imperfect, but they make your feature a poor general-purpose API, which is what removes the incentive.
7. Watch per-user behavior
Signals worth an alert, per end user or per cluster of users:
- Usage at a steady rate for many hours with no idle period.
- Request timing too regular for a person.
- Prompts that look templated: the same structure with a varying topic, across one or many accounts.
- Output tokens per request far above the feature's norm.
- Many accounts sharing devices, networks, payment fingerprints or prompt templates.
- A user at their cap every day from the first day.
Respond in steps: lower the user's cap, require verification, then suspend. Tell legitimate users what happened; a clear refusal costs less than a support thread.
8. Keep provider keys off the client
If a provider key is in a browser bundle or mobile app, the abuser skips your endpoint and every control above. Why AI API keys do not belong in front-end and mobile apps covers the architecture. Separately, set a provider-side spend limit on the project or workspace that serves the feature, as the backstop if your own limits fail.
Choosing limits without hurting real users
Start from data: the distribution of daily spend per user over a normal month. Set the free cap above what nearly all genuine users reach, and paid caps well above that. Run caps in alert-only mode for a week first and look at who would have been refused; if they are your best customers, the cap is wrong, and if they are accounts created yesterday, it is right. Reset caps on a fixed schedule in UTC and say so in the refusal. AI feature unit economics helps set the numbers against what each plan earns.
How INFRO handles per-user limits
INFRO is a gateway in front of provider accounts you own. Your server sends each request with the end user's id in metadata.user_id (or the X-INFRO-User-Id header). End-customer caps, from the Team plan, set a default spend per end user per day, week or month for the organization or a project, with overrides per customer that your backend can set through the management API. The check runs before the provider is called and counts the worst-case cost of that user's requests still in flight, so concurrent requests cannot pass the cap together. A request over its cap returns 402 with code end_user_budget and a resets_at time your app can show; a rule can also refuse requests that carry no end-user id with 400 end_user_id_required. Spend is INFRO's estimate of what your provider will bill you.
Around that, model allowlists refuse any model you have not approved with 403 model_not_allowed, guardrails on the Business plan apply pattern checks to prompts and responses (block, redact or flag), budgets cap each gateway key, and spend alerts flag a day that departs from a scope's usual day.
The limits: INFRO cannot verify who your users are, stop account farming at sign-up, or judge whether a prompt fits your feature's purpose unless you configure a check for it; those are your application's job. And INFRO's controls govern traffic that goes through INFRO. A provider key stolen from your systems, or from any platform that stores provider keys, INFRO included, can be used directly against the provider and bypasses every cap here; the backstops are provider-side spend limits, scoped keys and revocation at the provider.
Summary
An AI feature without per-user limits is a free model API with your name on the bill and on the provider's abuse reports. Identify every end user, cap each one's spend before the provider call, slow down new accounts, limit input and output, fix the model on the server, refuse off-purpose prompts, and watch per-user behavior for scripts and harvesting. The cap is the control that bounds everything you did not anticipate.