LiteLLM alternatives became a common search after 24 March 2026, when two LiteLLM releases on PyPI were found to contain credential-stealing code. The short answer: alternatives include self-hosted open-source gateways (Portkey's gateway, Bifrost, Agent Router, Kong AI Gateway), managed gateways (Cloudflare AI Gateway, Vercel AI Gateway), aggregators (OpenRouter) and BYOK control planes such as INFRO. But switching products does not by itself fix the risk the incident exposed. Any gateway holds every provider key you have, and the controls around it matter more than the logo on it.
This article sets out what happened, from LiteLLM's own account and a security vendor's analysis, then the controls that apply to any gateway, then the alternatives, described only from their own pages.
What happened in the LiteLLM supply-chain attack
LiteLLM's security update states that litellm==1.82.7 and litellm==1.82.8 were published to PyPI on 24 March 2026, starting at 10:39 UTC, and that both have been removed. LiteLLM believes the compromise originated in the Trivy dependency used in its CI/CD security scanning workflow; the attacker bypassed the official release workflow and uploaded the packages directly to PyPI1.
Datadog Security Labs traced this to a wider campaign it attributes to TeamPCP. On 19 March, attackers published a malicious Trivy release and force-pushed most trivy-action tags to malicious commits; the tainted actions dumped CI runner memory and exfiltrated credentials. The same campaign reached Checkmarx KICS on 23 March, LiteLLM on 24 March and the Telnyx Python SDK on 27 March2.
According to Datadog, the LiteLLM payload collected environment variables, SSH keys, cloud and Kubernetes credentials, Docker configuration, shell history, database credentials and CI/CD secrets, encrypted them and sent them to a lookalike domain. Version 1.82.7 ran when the proxy module was used; 1.82.8 added a .pth file that runs whenever the Python interpreter starts, which made it the higher-risk version2.
How LiteLLM responded
LiteLLM's update lists its response: the packages removed from PyPI, maintainer credentials rotated, Google's Mandiant engaged for forensics, new releases paused for a supply-chain review, confirmation that no malicious code reached main, cosign-signed Docker images from v1.83.0-nightly, and published SHA-256 checksums and scans for releases v1.78.0 to v1.82.6. On 30 March it announced a clean v1.83.0, built by a new CI/CD pipeline with isolated environments and stronger release gates1.
LiteLLM also states who was not affected: the official LiteLLM Proxy Docker image, which pins dependencies in requirements.txt, LiteLLM Cloud, installs from GitHub source, and anyone who stayed on 1.82.6 or earlier1. That detail is the most useful lesson in the whole incident. The deployment path that pinned its dependencies was not exposed.
LiteLLM remains a widely used open-source project with an MIT license, and its proxy offers virtual keys, per-key and per-team budgets, fallbacks, logging integrations and guardrails3. Nothing in this incident is a reason to assume another gateway's release pipeline is safer without asking.
Supply-chain controls for anything that holds every provider key
A gateway is a tier-zero service. It sees every prompt, and in most designs it holds a credential for every provider account you own. Datadog's guidance is to treat any install of the affected versions as a full-credential exposure, and its point that "reverting the compromised packages" is not complete remediation applies to every product in this article2.
- Install by hash, not by version range. pip's hash-checking mode installs only files whose hashes you recorded, and requires every dependency to be pinned once it is on. It protects against a package being altered and against a surprise upgrade through a loose specifier4.
- Publish with short-lived credentials. If you maintain anything that ships to PyPI, Trusted Publishing lets CI prove its identity over OIDC and receive a project-scoped token that expires after about 15 minutes, so a token stolen from runner memory stops working quickly5.
- Pin your CI actions by commit, not tag. The Trivy stage worked because tags were force-pushed to malicious commits2. A tag is a pointer someone else can move.
- Isolate the proxy. Run the gateway in its own container or service account, with only its own secrets mounted, no cloud credentials beyond what it needs and egress limited to provider hosts. The payload harvested whatever the process could read2.
- Scope the keys it holds. Give the gateway inference keys only, one per provider account or project, never an organization admin key. See least privilege for provider admin keys.
- Rehearse rotation. Know how long it takes to replace every provider key the gateway holds, and every key your applications use to reach the gateway. The guide to securing LLM API keys covers inventory and rotation order.
# generated with: pip-compile --generate-hashes requirements.in
litellm==<verified version> \
--hash=sha256:<hash recorded from your verified build>
# every transitive dependency is pinned and hashed the same way
# install with:
# pip install --require-hashes -r requirements.txtThe LiteLLM alternatives, compared
The useful comparison is operational: who runs the gateway, who holds the provider keys, whether your own provider accounts serve the traffic, what happens on failure, whether budgets see the spend, and what is recorded. The table summarises what each product's own pages say; the sections below give the detail and the caveats.
| Option | Hosting | Who holds provider keys | Your own provider accounts (BYOK) | Failover | Budgets | Observability |
|---|---|---|---|---|---|---|
| LiteLLM proxy | You run it | Your deployment | Yes | Fallbacks, load balancing | Per key, team, user | Logging integrations |
| Portkey gateway (OSS) | You run it, or Portkey's cloud | Your deployment | Yes | Fallbacks, retries, load balancing | Not described in README | Usage analytics, local console |
| Bifrost | You run it | Your deployment | Yes | Automatic failover, load balancing | Virtual keys, team and customer budgets | Prometheus, tracing, logs |
| Agent Router (formerly Envoy AI Gateway) | You run it, or hosted by Tetrate | The router | Yes | Provider fallback | Token limits per team, app or model | OpenTelemetry GenAI |
| Kong AI Gateway | Self-hosted or Kong-hosted data planes, managed through Konnect | The gateway's provider entity | Yes | Load balancing and failover | Token rate limiting | Metrics, OpenTelemetry, audit logs |
| Cloudflare AI Gateway | Managed | Cloudflare Secrets Store, or per request | Yes, or Cloudflare billing | Retries and model fallbacks | Spend limits incl. BYOK on priced models | Analytics, logs |
| Vercel AI Gateway | Managed | Vercel team | Yes, falls back to Vercel's credentials | Provider and model fallbacks | Budgets exclude BYOK spend | Request logs |
| OpenRouter | Managed | OpenRouter | Yes, falls back to shared capacity by default | Provider fallbacks by default | Not compared here | Not compared here |
| INFRO | Managed | INFRO, sealed per connection | Only your own accounts | Across your own connections | Hard caps per org, project, member, key | Usage, traces, audit log |
Self-hosted open-source gateways
Running your own gateway keeps keys and prompts inside your network. It also makes the supply chain yours: you choose versions, verify them and patch them.
- Portkey's gateway is MIT-licensed and starts with
npx @portkey-ai/gateway, Docker, Node.js or a Cloudflare Workers deployment. Its README lists fallbacks, up to five retries with backoff, weighted load balancing across keys or providers and pre-built guardrails; it does not describe budgets6. - Bifrost is Apache 2.0, runs with
npx -y @maximhq/bifrostor Docker, and lists automatic failover, load balancing across keys and providers, hierarchical budgets through virtual keys, Prometheus metrics, tracing and an MCP gateway7. - Agent Router, the new name of Envoy AI Gateway since it joined the Agentic AI Foundation on 10 September 2026, keeps its Apache 2.0 license, CRDs,
aigwCLI and maintainers8. Its site lists provider fallback, token limits per team, app or model, provider credentials held and rotated at the router, OpenTelemetry GenAI telemetry and a filtered MCP tool catalog, on Kubernetes, standalone or hosted by Tetrate9. - Kong AI Gateway is managed through Konnect with data planes self-hosted, Kong-hosted or on Kubernetes. It lists token-based rate limiting, semantic caching, prompt guards and PII sanitising, load balancing and failover, and provider credentials centralised in an AI Model Provider entity10.
Managed gateways
A managed gateway moves patching and release engineering to the vendor. You are then trusting their pipeline, so ask how they build, sign and deploy it.
- Cloudflare AI Gateway lists analytics, logging, caching, rate limiting, and request retry and model fallback11. BYOK keys are stored in Cloudflare Secrets Store, and a provider authorization header on the request takes precedence over the stored key12. Its spend limits count BYOK traffic for models with known pricing and return
429at the limit; Cloudflare notes that spend is tracked with eventual consistency, so a concurrent burst can briefly pass it13. - Vercel AI Gateway offers request logs, budgets per team, project, key or member, and provider and model fallbacks14. With BYOK, a failed request is retried on Vercel's own system credentials, billed to your AI Gateway balance, and BYOK spend is not counted in budgets15.
Aggregators
OpenRouter serves traffic through its own provider accounts or your keys. With BYOK it charges a fee of 5% of what the same usage would cost on OpenRouter, and by default falls back to its shared endpoints when your key fails or hits a rate limit; each key can be set to never use shared capacity16. Provider fallbacks are on by default and can be disabled per request with allow_fallbacks: false17. The OpenRouter alternatives comparison covers the commercial model in more depth.
BYOK control planes
INFRO is a hosted gateway that serves traffic only through provider accounts you connect, and it bills a software subscription with no markup on model spend. Each provider key is sealed with AES-256-GCM, bound to your organization and that one connection, and never returned by any endpoint. Applications use INFRO gateway keys, which can be limited to chosen providers or IP ranges and rotated with an overlap window, so the old key keeps working while you deploy the new one. Failover moves between your own connections only; there is no shared pool to fall back to, so budgets, which are hard caps per organization, project, member and key, see all of the spend. Request traces, usage by key and project, and an audit log of key, connection and policy changes are recorded. A hosted gateway is still a party you trust with your keys: ask any vendor, including us, how its release pipeline is protected.
Choosing after an incident
Decide by your constraints, not by the incident:
- Prompts and keys must stay in your network. Self-host. Budget the engineering for version pinning, isolation and upgrades, and compare the work with the build vs buy analysis.
- You want no gateway to operate, and your own contracts must carry all traffic. Check, for each managed option, what happens when your key fails and whether budgets count your own-key spend. Those two answers differ more than the feature lists.
- You stay on LiteLLM. Pin a verified version by hash, prefer the pinned Docker image path LiteLLM recommends, isolate the proxy, scope its keys, and keep up with its later security releases.
Whatever you choose, write the rotation runbook now and run it once. The incident's real cost fell on teams that had to discover, under pressure, which keys a compromised process could read.