Migration guidesFrom an in-house AI proxy
From an in-house proxy to a control plane you do not run
Many teams wrote a small service in front of OpenAI and Anthropic: it holds the keys, retries, logs, counts tokens against a cost table and enforces a quota per team. This page maps each of those jobs to INFRO, says which ones stay your code, and shows how to move one service at a time with the proxy still running.
Your own service in front of the providers: keys, retries, logging, quotas
INFRO accounts are self-serve. The catalog, the docs, and the API contract are public, the gateway is deployed, and anything beyond v1 is marked Later. Plans and trial
Where things stand
What these proxies usually do
Hold provider keys, issue internal keys per team, retry and fail over, log requests, price tokens from a hand-kept table, and refuse a team over its quota.
What they cost to keep
Someone owns the price table, the new models, each provider's new parameters, the database behind the logs and the on-call for the proxy itself.
Two ways to move
Replace it: services call INFRO, which calls your providers. Or keep it behind INFRO for a while: connect the proxy as an openai-compatible connection and move its upstreams over one at a time.
A proxy you wrote does exactly what you need, and that is worth something. Keep it if one of these is the reason it exists.
- It changes the request: prompt templating, retrieval injected into context, PII scrubbing with your own models, or response post-processing. INFRO's guardrails are pattern rules and decision-model checks; it does not run your code in the request path.
- It must run inside your network or a named region. INFRO's plans are hosted on Cloudflare's network; a self-hosted or hybrid deployment is an enterprise agreement.
- It serves models no provider API exposes, such as weights you host on internal GPUs, and nothing else. INFRO can route to that server as an
openai-compatibleconnection, but the server stays yours. - It is small, owned, and nobody is asking for budgets, attribution or failover. Then a gateway is a migration with nothing at the end of it.
The edits, one row each. Everything not listed here stays as it is.
| Your proxy | INFRO | |
|---|---|---|
| Base URL | Your proxy's internal host. | https://api.infro.io/v1 (OpenAI shape), /v1/messages (Anthropic shape), /v1/responses (OpenAI Responses). |
| Key | An internal key your proxy issued per team or service. | An INFRO gateway key per service, with its own model allowlist, provider list, IP ranges, environment and spend limit. |
| Model id | Whatever aliases your proxy defined. | vendor/model, or a route name: a route is an alias that resolves to an ordered list of your connections, edited in the console. |
| Headers | Your own team, user or feature headers. | X-INFRO-Feature, X-INFRO-User-Id, X-INFRO-Trace-Id, or metadata.user_id, metadata.session_id, metadata.tags in the body. |
Before · your proxy
import OpenAI from "openai";
const client = new OpenAI({
baseURL: "https://llm-proxy.internal.example.com/v1",
apiKey: process.env.LLM_PROXY_KEY,
defaultHeaders: { "X-Team": "search" },
});
await client.chat.completions.create({
model: "default-chat",
messages: [{ role: "user", content: "Hello" }],
});After · INFRO
import OpenAI from "openai";
const client = new OpenAI({
baseURL: "https://api.infro.io/v1",
apiKey: process.env.INFRO_API_KEY, // one key per service
defaultHeaders: { "X-INFRO-Feature": "search" },
});
await client.chat.completions.create({
model: "default-chat", // a route with the same name
messages: [{ role: "user", content: "Hello" }],
});Keep your aliases
Create a route for each alias your proxy defined, with the same name, and services keep sending the same model. A key can be pinned to named routes, so it cannot call a model directly.
Move one service at a time
Connect the proxy itself as an openai-compatible connection and add it as a route target. Services can switch to INFRO first; the upstreams move from the proxy to direct connections afterwards.
Teams become projects
Per-team quotas become project budgets and per-key spend limits. Members can be confined to the projects they work on.
What your setup does today against INFRO, checked against what the gateway implements. One row says INFRO has nothing.
| Your proxy | What it does there | In INFRO |
|---|---|---|
| Key vaultsame shape | Provider keys in the proxy's config or a secret store. | Provider connections, sealed at rest, several keys per provider with standby keys last. |
| Internal keyssame shape | Keys the proxy issues per team or service. | Gateway keys with model allowlists, provider lists, IP ranges, an environment, a spend limit, rotation and an off switch. |
| Retries and fallbackssame shape | Code in the proxy. | Routes with priority or weighted targets, if-rules on the request, and failure rules per kind (429, 5xx, timeout, connection, credential, moderation). |
| Quotassame shape | Token or dollar counters per team. | Budgets per organization, project, member, key and end customer that refuse at the ceiling with 402, held under concurrency. |
| Price tabledifferent | A hand-kept table of per-token prices. | The vendor's list rate from INFRO's catalog, your negotiated rates where you enter them, and unpriced where neither is known, never zero. |
| Request logsdifferent | Rows in your own database. | The request log and traces in the console, GET /v1/requests, exports, and spans to your OpenTelemetry collector. Prompt capture is off until you switch it on. |
| Prompt and response transformsnothing | Templating, retrieval, scrubbing with your own logic. | Not available. Guardrails block, redact or flag by pattern and by decision-model checks; INFRO does not run your code. |
| Self-hosted modelsdifferent | vLLM, TGI or Ollama behind the proxy. | An openai-compatible connection to that server, routed like any other provider. The server stays yours to run. |
| Running it yourselfdifferent | It runs where you deploy it. | Hosted. A self-hosted or hybrid deployment is an enterprise agreement, not a plan.Enterprise — Available on an enterprise agreement rather than self-serve. |
What INFRO does not do
- Run your own code in the request path: prompt templating, retrieval, custom transforms.
- Embeddings, Batch, Files, fine-tuning and Realtime.
- Region pinning; residency is on the roadmap, not available.
- A self-hosted deployment without an enterprise agreement.
What does not come across
- Your proxy's logs. There is no importer; keep them where they are.
- Its price table. Enter negotiated rates in the console instead; list rates come from the catalog.
- Custom transforms. They stay in your code, before the call to INFRO.
Questions
- Can INFRO sit in front of our existing proxy?
- Yes. Connect the proxy as an
openai-compatibleconnection with its base URL and key, and route to it like any provider while you move its upstreams over. - Do our provider contracts change?
- No. You connect the provider accounts you hold, and your providers keep billing you directly at your own rates. INFRO is a subscription with 0% markup on model spend.
- What stays our code?
- Anything that changes the prompt or the response: templating, retrieval, custom scrubbing, post-processing.
Move with us, or on your own
The migration program maps your an in-house AI proxy setup to INFRO connections, routes and budgets on a call and moves the first service with you. Your providers keep billing you directly; INFRO is a subscription and adds 0% to model spend.
Sources
The SDK behaviour on this page comes from these, read on 19 October 2026.
- OpenAI — openai-node README (baseURL and defaultHeaders)read 2026-10-19
- OpenAI — openai-python README (base_url and default_headers)read 2026-10-19