"Use a gateway to reduce your AI costs" is common advice, and it is half right. A gateway sits between your application and the model providers, so it can change which requests reach a provider and which model serves them. It cannot change what the provider charges per token on your own account. And it charges for itself.
This guide separates the two halves, so you can tell whether a gateway would lower your spend or only reorganize it.
What a gateway does not change
When a gateway routes requests on your own provider keys, the provider bills you exactly as it would if you called it directly: same model, same tokens, same rate. A gateway that says it reduces cost is describing something it does to the number of requests or the choice of model, not the price of either.
When a gateway serves requests on its own provider accounts instead, you pay the gateway's rate for the model, usually the provider's list price. That arrangement does not lower the rate either; it does mean any rate your company negotiated with the provider no longer applies to that traffic.
Neither arrangement changes the price list. Some cost advice blurs this, and it is worth being exact about before comparing products.
What a gateway can change
Four things, roughly in order of how much they usually matter.
It can refuse spend before it happens
The most expensive AI incidents are not inefficient prompts. They are an agent stuck in a loop, a leaked key, a batch job pointed at the wrong model, or one customer of your product running far more traffic than their plan pays for. A budget that is checked before the provider is called turns each of those from an invoice surprise into a refused request.
Two details decide whether a budget does this:
- Does it count your own-key spend? Some gateways' budgets count only usage bought through them.
- Does it check before the call or record after it? Several products record cost when a request completes and enforce on the next one, so concurrent requests can pass the limit.
LLM spend limits compared answers both for ten products from their own documentation.
It can stop paying twice for the same answer
A response cache answers an exact repeat of a request without calling the provider. This only helps for deterministic requests that really repeat — the same classification of the same text, the same lookup — and it does nothing for conversational traffic, where almost no two requests match. Measure your repeat rate before counting on it. Prompt caching is a different mechanism, run by the provider, and it can raise cost when cache entries are written often and read rarely; see Prompt caching break-even.
It can send each workload to a smaller model
The largest single change in spend usually comes from moving a workload to a smaller model that an evaluation shows is good enough. A gateway makes that a configuration change (a route) rather than a code change, and lets you run the old and new model side by side. The decision still needs an evaluation on your data; a gateway that picks models for you on cost alone moves the risk to quality.
Work no user is waiting on — nightly enrichment, backfills, evaluations — can go further: OpenAI prices its Batch API at half the cost of its synchronous APIs1. That is a provider feature, available with or without a gateway.
It can tell you where the money went
Attribution does not reduce spend by itself. It is what tells you which of the three levers above to pull: which project, key, model or end customer accounts for the growth. Without it, cost work is guesswork applied to the largest line on the invoice.
What a gateway costs
Every gateway charges for itself, and the base of the charge matters more over a year than its size. From the vendors' own pricing pages:
| Basis | Example, as published |
|---|---|
| Fee on usage you buy through the gateway | OpenRouter lists a platform fee of 5.5% on pay-as-you-go2; Cloudflare applies a 5% fee to usage bought through its Unified Billing3 |
| Markup on model cost | Requesty's pay-as-you-go plan is a 5% markup4 |
| Fee on your own-key usage above an allowance | OpenRouter: no fee up to $25,000 of list-price usage a month, 5% after2 |
| No fee on model usage, priced add-ons | Vercel: no markup on tokens, some features priced per 1,000 requests5 |
| Free core | Cloudflare: analytics, caching and rate limiting are free6 |
| Free software, you run it | LiteLLM open source; Enterprise at custom pricing7 |
| Flat subscription | A fixed monthly price per plan, with allowances |
A percentage fee grows every time your traffic grows or a model's price rises. A per-request price grows with traffic but not with model prices. A subscription moves only when you change plan. A self-hosted gateway has no fee and costs the engineering time to run, patch and scale it. AI gateway pricing models compared works through one month on each basis.
Questions to ask a gateway vendor about cost
Each answer should be in the vendor's documentation; if it is not, ask before you commit traffic.
- What is your fee charged on: purchases, model cost, own-key usage, requests, logs, models, or a plan?
- Does any fee apply to requests that run on my own provider keys, and above what allowance?
- When my key fails or is rate limited, do you retry on your own provider capacity, and who pays for that request?
- Do your budgets count spend on my own keys, and are they checked before the provider call or after it?
- What happens under a budget to a model you have no price for?
- Can I enter my negotiated rates, or is every estimate at list price?
- Can I export every request and its estimated cost if I leave?
Two of these change the bill directly (the fee basis and the fallback behaviour); the rest decide whether the controls you are paying for act on your actual spend.
When a gateway does not pay for itself
Be honest about these before buying one:
- One provider, one product, low volume. The provider's own spend limit and usage page may be all the control you need.
- No repeats, one model. If every request is unique and already on the right model, caching and routing have nothing to act on.
- Nobody will read the reports. Attribution only lowers spend if someone acts on it.
In those cases, set a hard limit on each provider account, tag requests by feature in your own logs, and revisit when you add a second provider or start passing AI usage through to customers.
Where INFRO sits
INFRO is a hosted gateway over your own provider accounts only, so read this as the vendor's description. Every request runs on a provider account you connect, the provider bills you directly, and INFRO charges a subscription with 0% markup on model spend. Budgets at organization, project, member and key level are checked before the provider is called on every plan, estimated cost is recorded per request at list rates or at your negotiated rates from the Team plan, and there is an exact-match response cache for chat completions. It does not lower any provider's rate, and it has no free plan.
How to decide
Write down your three largest sources of AI spend and ask, for each, which of the four levers would act on it: refuse it, cache it, move it to a smaller model, or attribute it. If none would, a gateway will not lower your bill. If one or more would, compare gateways on whether their controls reach that spend, and on whether their own fee grows with it.