Most advice about AI gateways is written for platform teams with a dozen services and a finance department. A solo founder has a different problem: a product that calls one model, a card on file with the provider, and no time to run infrastructure. This guide is about that situation, and it starts with what you can do for free.
What you need on day one (no gateway required)
Three things protect a solo product from the failures that actually hurt at this stage:
- A hard spend limit on the provider account. OpenAI and Anthropic let you set a limit on the account itself, which works even when your own code has a bug; cloud budgets on AWS and Azure notify rather than block, so check which kind yours is (LLM spend limits compared). Set it to the amount you can lose without the business ending, not to your forecast.
- A feature tag on every request, in your own logs. Record which feature or endpoint sent each call, and the token counts the provider returns. A few lines of code, and the first time spend jumps you will know where to look.
- A written plan for an outage. Decide now what the product does when the provider is down or rate limiting: show an error, queue the work, or switch to a second model. Most founders discover they need failover during the outage.
That is enough for a product with one provider, one model and no per-customer usage. A gateway at this point is mostly a second dashboard.
What to log from day one
The feature tag is the piece most founders skip, and the one they most want back the first time the bill jumps. It costs a few lines around each model call:
import json, logging, time
log = logging.getLogger("model_calls")
def call_model(client, feature: str, customer_id: str, **request):
started = time.monotonic()
response = client.chat.completions.create(**request)
usage = response.usage
log.info(json.dumps({
"feature": feature, # which part of the product asked
"customer": customer_id, # who it was for
"model": request["model"],
"input_tokens": usage.prompt_tokens,
"output_tokens": usage.completion_tokens,
"ms": round((time.monotonic() - started) * 1000),
}))
return responseKeep the rows somewhere you can query by feature and customer. When you later add a gateway, these become the request tags it records for you, and the history you kept is still comparable.
Failover without a gateway
You can fail over in application code: catch a timeout or a rate-limit error from the first provider, retry the same request on a second. It works, and it is how many products start. Three things make it harder than it looks: the two providers expect different request shapes unless both speak the same API, a retry on a second provider doubles the cost of a request that half-completed, and an agent loop can now spend through both accounts with no limit that sees both. That third point is usually what moves a team to a gateway.
The three triggers for a gateway
You start to need something between your code and the providers when one of these becomes true.
1. You add a second provider
The usual reason is availability: the first provider has an outage or rate limits you, and you want requests to fail over to another. The moment you have two, you also have two bills, two spend limits that cannot see each other, and a failover path that can spend through both accounts in a loop. A gateway puts one budget and one record across both. Managing AI API costs across providers covers the work in detail.
2. Your customers' usage becomes your cost
If each of your users can trigger model calls — a chat feature, an agent, a generation tool — then one heavy user can spend what a hundred light ones pay you. You need a cap per customer, checked before the call, with a clear error your app can show. That is awkward to build correctly in application code, especially with concurrent requests, and it is a core gateway feature.
3. You cannot say where the money goes
If the provider bill grows and you cannot tell whether it was one feature, one customer or one prompt change, the cost of not knowing is now larger than the cost of a tool that tells you.
The free routes
Each of these is free in some form, by its vendor's own documentation:
| Option | What is free | What it costs you |
|---|---|---|
| Provider-side limits | Spend caps on the provider account | No cross-provider view, no per-customer caps |
| Cloudflare AI Gateway | Analytics, caching and rate limiting1 | A Cloudflare account; check its logging pricing |
| Vercel AI Gateway | A free tier covering a subset of models, no markup on tokens2 | Best fit when you already deploy on Vercel |
| OpenRouter, own keys | No fee on own-key usage up to $25,000 of list-price usage a month3 | 5% above the allowance; default fallback to its own capacity |
| LiteLLM | Open-source gateway software4 | You deploy, patch and scale it |
| Langfuse | Free Hobby tier; free to self-host5 | Observability only, not in the request path |
Two cautions. First, "free to self-host" is free software, not free operation: count the hours to deploy, upgrade, back up and watch the service, at what your time is worth. Langfuse, for example, documents Docker Compose for a single machine and Helm for production6, and either is a service you now run. Second, read what each free tier's budgets count. Some gateways' limits cover only usage bought through them, not spend on your own provider keys.
When paying for a hosted gateway makes sense
Paying makes sense when either of these is true:
- The hours you would spend running a self-hosted gateway each month cost more than a hosted plan.
- You need something the free options do not do for your setup: budgets per customer that refuse before the call, one limit across several providers, or cost estimated at the rates your own contracts carry.
Look at what the fee is charged on, not only its size. A percentage of model spend grows with your success; a subscription does not. AI gateway pricing models compared lists how each vendor charges.
Where INFRO fits, and where it does not
INFRO is a hosted gateway over your own provider accounts, with no free plan; read this as the vendor's description. The Developer plan is $35 a month (or $29 a month billed annually) for one seat, three projects and 100,000 gateway requests a month, with a 14-day trial that needs a card. Requests run on provider accounts you connect, your providers bill you directly, and INFRO takes 0% of model spend. Budgets at organization, project and key level are checked before the call on every plan; per-customer caps start on the Team plan.
If you are pre-revenue with one provider, the day-one checklist above is the better use of $35. INFRO fits once you are on a second provider or need every request accounted for, and do not want to run the gateway yourself.
Keep the exit open
Whatever you choose, keep the provider accounts in your own name and call the gateway through an OpenAI-compatible API. Then moving from no gateway to a free one, from a free one to a paid one, or back, is a base URL and a key, not a migration.