# Do AI gateways reduce API costs? What they can and cannot change

URL: https://infro.io/blog/do-ai-gateways-reduce-costs
Author: INFRO Editorial (https://infro.io/blog/editorial)
Publisher: INFRO
Published: 2026-10-19
Last modified: 2026-10-19
Last reviewed: 2026-10-19 (next review due 2026-11-18)
Topic: Cost and FinOps
Format: Guide

> An AI gateway does not lower the per-token price your provider charges; on your own provider keys you pay the provider's rate either way. It can lower total spend by answering exact repeats from a cache, sending workloads to smaller models, refusing requests that would breach a budget, and showing which project or customer the spend came from. Against that, it adds its own fee, which is charged on one of several bases: a percentage of what you buy through it, a markup on model cost, a fee on your own-key usage, a price per request, or a flat subscription.

Key takeaways:
- On your own provider keys, a gateway does not change the per-token rate; the provider bills you the same either way.
- The levers a gateway does have are fewer requests (exact-match caching), smaller models per workload, refused runaway spend, and attribution that tells you where to look.
- Every gateway charges for itself on some basis; a percentage of spend grows with your traffic and with model prices, a subscription does not.
- Ask whether a gateway's budget counts your own-key spend and whether it refuses before the call or records after it.
- If one provider, one product and a provider-side spend limit cover you, a gateway may add cost without changing the outcome.

"Use a gateway to reduce your AI costs" is common advice, and it is half right. A gateway sits between your application and the model providers, so it can change *which* requests reach a provider and *which* model serves them. It cannot change what the provider charges per token on your own account. And it charges for itself.

This guide separates the two halves, so you can tell whether a gateway would lower your spend or only reorganize it.

## What a gateway does not change

When a gateway routes requests on your own provider keys, the provider bills you exactly as it would if you called it directly: same model, same tokens, same rate. A gateway that says it reduces cost is describing something it does to the *number* of requests or the *choice* of model, not the price of either.

When a gateway serves requests on its own provider accounts instead, you pay the gateway's rate for the model, usually the provider's list price. That arrangement does not lower the rate either; it does mean any rate your company negotiated with the provider no longer applies to that traffic.

Neither arrangement changes the price list. Some cost advice blurs this, and it is worth being exact about before comparing products.

## What a gateway can change

Four things, roughly in order of how much they usually matter.

### It can refuse spend before it happens

The most expensive AI incidents are not inefficient prompts. They are an agent stuck in a loop, a leaked key, a batch job pointed at the wrong model, or one customer of your product running far more traffic than their plan pays for. A budget that is checked before the provider is called turns each of those from an invoice surprise into a refused request.

Two details decide whether a budget does this:

- **Does it count your own-key spend?** Some gateways' budgets count only usage bought through them.
- **Does it check before the call or record after it?** Several products record cost when a request completes and enforce on the next one, so concurrent requests can pass the limit.

[LLM spend limits compared](https://infro.io/blog/llm-spend-limits-compared) answers both for ten products from their own documentation.

### It can stop paying twice for the same answer

A response cache answers an exact repeat of a request without calling the provider. This only helps for deterministic requests that really repeat — the same classification of the same text, the same lookup — and it does nothing for conversational traffic, where almost no two requests match. Measure your repeat rate before counting on it. Prompt caching is a different mechanism, run by the provider, and it can raise cost when cache entries are written often and read rarely; see [Prompt caching break-even](https://infro.io/blog/prompt-caching-break-even).

### It can send each workload to a smaller model

The largest single change in spend usually comes from moving a workload to a smaller model that an evaluation shows is good enough. A gateway makes that a configuration change (a route) rather than a code change, and lets you run the old and new model side by side. The decision still needs an evaluation on your data; a gateway that picks models for you on cost alone moves the risk to quality.

Work no user is waiting on — nightly enrichment, backfills, evaluations — can go further: OpenAI prices its Batch API at half the cost of its synchronous APIs[1]. That is a provider feature, available with or without a gateway.

### It can tell you where the money went

Attribution does not reduce spend by itself. It is what tells you *which* of the three levers above to pull: which project, key, model or end customer accounts for the growth. Without it, cost work is guesswork applied to the largest line on the invoice.

## What a gateway costs

Every gateway charges for itself, and the base of the charge matters more over a year than its size. From the vendors' own pricing pages:

| Basis | Example, as published |
| --- | --- |
| Fee on usage you buy through the gateway | OpenRouter lists a platform fee of 5.5% on pay-as-you-go[2]; Cloudflare applies a 5% fee to usage bought through its Unified Billing[3] |
| Markup on model cost | Requesty's pay-as-you-go plan is a 5% markup[4] |
| Fee on your own-key usage above an allowance | OpenRouter: no fee up to $25,000 of list-price usage a month, 5% after[2] |
| No fee on model usage, priced add-ons | Vercel: no markup on tokens, some features priced per 1,000 requests[5] |
| Free core | Cloudflare: analytics, caching and rate limiting are free[6] |
| Free software, you run it | LiteLLM open source; Enterprise at custom pricing[7] |
| Flat subscription | A fixed monthly price per plan, with allowances |

Table: How gateways charge, quoted from each vendor's page on the access date in the sources.

A percentage fee grows every time your traffic grows or a model's price rises. A per-request price grows with traffic but not with model prices. A subscription moves only when you change plan. A self-hosted gateway has no fee and costs the engineering time to run, patch and scale it. [AI gateway pricing models compared](https://infro.io/blog/ai-gateway-pricing-models-compared) works through one month on each basis.

## Questions to ask a gateway vendor about cost

Each answer should be in the vendor's documentation; if it is not, ask before you commit traffic.

- [ ] What is your fee charged on: purchases, model cost, own-key usage, requests, logs, models, or a plan?
- [ ] Does any fee apply to requests that run on my own provider keys, and above what allowance?
- [ ] When my key fails or is rate limited, do you retry on your own provider capacity, and who pays for that request?
- [ ] Do your budgets count spend on my own keys, and are they checked before the provider call or after it?
- [ ] What happens under a budget to a model you have no price for?
- [ ] Can I enter my negotiated rates, or is every estimate at list price?
- [ ] Can I export every request and its estimated cost if I leave?

Two of these change the bill directly (the fee basis and the fallback behaviour); the rest decide whether the controls you are paying for act on your actual spend.

## When a gateway does not pay for itself

Be honest about these before buying one:

- **One provider, one product, low volume.** The provider's own spend limit and usage page may be all the control you need.
- **No repeats, one model.** If every request is unique and already on the right model, caching and routing have nothing to act on.
- **Nobody will read the reports.** Attribution only lowers spend if someone acts on it.

In those cases, set a hard limit on each provider account, tag requests by feature in your own logs, and revisit when you add a second provider or start passing AI usage through to customers.

## Where INFRO sits

INFRO is a hosted gateway over your own provider accounts only, so read this as the vendor's description. Every request runs on a provider account you connect, the provider bills you directly, and INFRO charges a subscription with 0% markup on model spend. Budgets at organization, project, member and key level are checked before the provider is called on every plan, estimated cost is recorded per request at list rates or at your negotiated rates from the Team plan, and there is an exact-match response cache for chat completions. It does not lower any provider's rate, and it has no free plan.

## How to decide

Write down your three largest sources of AI spend and ask, for each, which of the four levers would act on it: refuse it, cache it, move it to a smaller model, or attribute it. If none would, a gateway will not lower your bill. If one or more would, compare gateways on whether their controls reach that spend, and on whether their own fee grows with it.

## Questions

### What is the best AI gateway for reducing API costs?

There is no single answer, because a gateway does not lower the provider's per-token rate. Pick on two questions instead: does its fee grow with your model spend, and do its controls (caching, budgets that refuse before the call, attribution) act on the waste you actually have? Self-hosted open-source gateways have no software fee but cost engineering time; hosted ones charge a percentage, a per-request price or a subscription.

### What does zero markup mean for an AI gateway?

That the gateway adds nothing to the model's price. It does not mean the gateway is free: zero-markup gateways charge in another way, such as a fee when you buy usage through them, a per-request price, or a subscription. On your own provider keys, the provider bills you directly and the gateway's charge is a separate line.

### Can a gateway make my AI bill higher?

Yes. Its own fee is added, a percentage fee grows with your spend, prompt caching can raise cost when cache entries are written often and read rarely, and a gateway that falls back to its own capacity when your key fails can run traffic on its account at its rates. Check each of these in the gateway's own documentation.

## Sources

1. Batch API — OpenAI. https://developers.openai.com/api/docs/guides/batch (accessed 2026-10-13)
2. Pricing — OpenRouter. https://openrouter.ai/pricing (accessed 2026-10-13)
3. Unified Billing — Cloudflare. https://developers.cloudflare.com/ai-gateway/features/unified-billing/ (accessed 2026-10-13)
4. Pricing — Requesty. https://www.requesty.ai/pricing (accessed 2026-10-13)
5. AI Gateway pricing — Vercel. https://vercel.com/docs/ai-gateway/pricing (accessed 2026-10-13)
6. AI Gateway pricing — Cloudflare. https://developers.cloudflare.com/ai-gateway/reference/pricing/ (accessed 2026-10-13)
7. LiteLLM Enterprise — LiteLLM. https://www.litellm.ai/enterprise (accessed 2026-10-13)