Skip to content

Migration guidesFrom LiteLLM

From a LiteLLM proxy you run, to one you do not

LiteLLM is two things: a Python SDK that speaks to 100+ providers, and a proxy you deploy with Postgres and Redis that adds virtual keys, budgets, fallbacks and caching. Both run on your own provider keys, as INFRO does. What changes is who runs the proxy. This page maps the proxy's config to INFRO and shows how to keep the SDK.

MIT open source by BerriAI, plus a licensed Enterprise edition; self-hosted

INFRO accounts are self-serve. The catalog, the docs, and the API contract are public, the gateway is deployed, and anything beyond v1 is marked Later. Plans and trial

Where things stand

Licence and editions

Everything outside the enterprise/ directory is MIT; Enterprise is the same image with a licence key, adding SSO, SCIM, audit logs, secret managers and support.

Security advisories in 2026

LiteLLM's own blog lists a PyPI compromise of litellm 1.82.7 and 1.82.8 on 2026-03-24 (the official Docker image was not affected; clean 1.83.0 shipped 2026-03-30), a SQL injection in proxy key verification fixed in 1.83.7 (2026-04-29), and a Host-header authentication bypass fixed in 1.84.0 (2026-06-01). Running the proxy means tracking these yourself.

Whose account serves the request

Yours, in both products. LiteLLM never holds a provider account, and neither does INFRO; your providers bill you directly.

LiteLLM is free, open and runs anywhere. If those are the reasons you chose it, they are still good reasons.

  • Traffic must not leave your network, or you must be able to read and patch the gateway's code. INFRO is hosted and closed source; self-hosted or hybrid is an enterprise agreement.
  • You use semantic caching, context-window or content-policy fallbacks, the A2A agent gateway, or callbacks into a specific logging vendor. INFRO has exact-match caching, route failure rules, an MCP gateway and OpenTelemetry export.
  • You call providers INFRO does not connect to, or endpoints it does not serve: embeddings, batches, files, fine-tuning, Realtime.
  • Running it costs you little because a platform team already owns Postgres, Redis and upgrades.

The edits, one row each. Everything not listed here stays as it is.

 LiteLLMINFRO
Base URLYour proxy, for example http://0.0.0.0:4000.https://api.infro.io/v1
KeyA LiteLLM virtual key (sk-...) from /key/generate.An INFRO gateway key, created in the console or the management API. Provider keys move from litellm_params to INFRO connections.
Model idA model_name from model_list, such as gpt-4o.A route with the same name, so callers keep sending it, or vendor/model directly.
Body fieldsmetadata { tags, trace_id, trace_user_id, generation_name }, cache { ttl, namespace }, user.metadata { user_id, session_id, tags }, cache for the response cache, X-INFRO-Trace-Id for a trace id.

Before · LiteLLM proxy

import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "http://0.0.0.0:4000",
  apiKey: process.env.LITELLM_API_KEY, // virtual key
});

await client.chat.completions.create({
  model: "claude-default", // model_name in model_list
  messages: [{ role: "user", content: "Hello" }],
  user: "user-123",
  // @ts-expect-error LiteLLM extension
  metadata: { tags: ["production"] },
});

After · INFRO

import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://api.infro.io/v1",
  apiKey: process.env.INFRO_API_KEY,
});

await client.chat.completions.create({
  model: "claude-default", // an INFRO route, same name
  messages: [{ role: "user", content: "Hello" }],
  // @ts-expect-error INFRO extension
  metadata: { user_id: "user-123", tags: { env: "production" } },
});

`model_list` becomes routes

Each model_name with several deployments becomes a route with those targets; rpm weights become weighted targets; router_settings.fallbacks become the route's later targets or fallbacks.

Keep the SDK

LiteLLM's OpenAI-compatible provider sends everything after openai/ as the model id, so openai/anthropic/claude-sonnet-5.5 reaches INFRO as anthropic/claude-sonnet-5.5. Fallbacks then happen in INFRO, not in the SDK.

Or keep the proxy behind INFRO

Connect the proxy as an openai-compatible connection while you move. Note that INFRO then sees only the proxy as one provider, so per-provider failover and cost still happen inside LiteLLM until you connect the providers directly.

LiteLLM's objects against INFRO's, from LiteLLM's public documentation read on 19 October 2026 and checked against what the gateway implements. One row says INFRO has nothing.

LiteLLMWhat it does thereIn INFRO
`model_list` and `litellm_params`same shapeDeployments per model name, with provider keys in config.Provider connections (keys sealed, several per provider) and routes over them.
Virtual keyssame shapeProxy-issued keys with allowed models, budget, tpm and rpm, team and user.Gateway keys with allowed models, providers, IP ranges, an environment, a spend limit and request-rate tier; rotation and an off switch.
Teams, users, budgetssame shapemax_budget and budget_duration per key, team or user.Budgets per organization, project, member, key and end customer, held under concurrency, 402 at the ceiling before the provider call.
tpm limitsdifferentToken-per-minute limits per key or deployment.Request-rate tiers per key and dollar budgets. No token-per-minute limit.
Fallbacks and retriessame shapefallbacks, num_retries, cooldowns, allowed_fails.Route targets and failure rules per kind (retry, fail over or stop); fallbacks per request.
Context-window and content-policy fallbacksdifferentA different model when the prompt is too long or a provider's policy refuses it.A moderation failure rule can fail over to the next target. No context-window fallback.
CachingdifferentExact and semantic caches on Redis, S3, GCS, Qdrant and others.Exact-match, opt-in, for chat completions, hosted. No semantic cache.
GuardrailsdifferentPluggable guardrails; several integrations need Enterprise.Pattern rules that block, redact or flag, and checks on a decision model, on every plan that includes guardrails.
Logging callbacksdifferentCallbacks to Langfuse and other tools; spend logs in Postgres.The request log, exports, and spans to your OpenTelemetry collector.
Spend trackingdifferentCost from LiteLLM's model price map.Estimated provider cost from your negotiated rates or the vendor's list rate, unpriced when unknown, reconciled against OpenAI's and Anthropic's cost APIs.
MCP gatewaysame shapeMCP servers behind the proxy.An MCP gateway for your own MCP servers, with approved tools and grants per key.
A2A agent gatewaynothingAgent-to-agent traffic through the proxy.Nothing.
Running it yourselfdifferentMIT, Docker or pip, anywhere.Hosted. Self-hosted or hybrid is an enterprise agreement.Enterprise — Available on an enterprise agreement rather than self-serve.

Our reading of LiteLLM’s published documentation, not a trial or a benchmark. If we have read something wrong, tell us and we will fix the row.

What INFRO does not do

  • Run on your own infrastructure without an enterprise agreement, or let you read its source.
  • Semantic caching, context-window fallbacks, A2A.
  • Serve embeddings, batches, files, fine-tuning or Realtime.
  • Token-per-minute limits; INFRO limits by request rate and by dollars.

What does not come across

  • Spend logs and request history in your LiteLLM Postgres. There is no importer.
  • Virtual keys. Issue INFRO gateway keys and rotate callers over.
  • Callbacks and custom hooks. Use OpenTelemetry export, or keep that logic in your code.

Questions

Can I keep using the LiteLLM Python SDK?
Yes. Call INFRO through LiteLLM's OpenAI-compatible provider: model="openai/<INFRO model id>" with api_base="https://api.infro.io/v1" and your INFRO key.
Is INFRO open source like LiteLLM?
No. INFRO is a hosted service; a self-hosted or hybrid deployment is available under an enterprise agreement.
Who pays the model providers?
You do, directly, as with LiteLLM. INFRO is a subscription and adds 0% to model spend.

Move with us, or on your own

The migration program maps your LiteLLM setup to INFRO connections, routes and budgets on a call and moves the first service with you. Your providers keep billing you directly; INFRO is a subscription and adds 0% to model spend.

Start 14-day trial Book a migration sessionINFRO vs LiteLLMRead the quickstart

Sources

Every statement on this page about LiteLLM comes from one of these, read on 19 October 2026. Nothing is inferred from a trial, a benchmark, or a conversation.