# LiteLLM alternatives in 2026: AI gateways compared

URL: https://infro.io/blog/litellm-alternatives-2026
Author: INFRO Editorial (https://infro.io/blog/editorial)
Publisher: INFRO
Published: 2026-10-19
Last modified: 2026-10-19
Last reviewed: 2026-10-19 (next review due 2026-11-18)
Topic: Security and governance
Format: Comparison

> On 24 March 2026, litellm 1.82.7 and 1.82.8 were published to PyPI with credential-stealing code after the attackers obtained publishing access through a compromised Trivy scanner in LiteLLM's CI; LiteLLM removed them, rotated credentials and shipped a clean v1.83.0 through a rebuilt pipeline. The lesson applies to every gateway: pin by hash, publish with short-lived credentials, isolate the proxy and scope the keys it holds. Alternatives are self-hosted open-source gateways (Portkey's gateway, Bifrost, Agent Router, Kong), managed gateways (Cloudflare, Vercel), aggregators (OpenRouter) and BYOK control planes such as INFRO.

Key takeaways:
- litellm 1.82.7 and 1.82.8 were malicious on PyPI on 24 March 2026; LiteLLM says its official Proxy Docker image, which pins dependencies, was not affected.
- The entry point was a compromised security scanner in CI, not LiteLLM's source code. Any gateway with a release pipeline carries the same class of risk.
- A gateway holds every provider key you own, so treat it as a tier-zero service: pin by hash, isolate it, scope its keys and rehearse rotation.
- Alternatives differ most on who holds the provider keys and what happens when your key fails, not on model count.
- Every product here changes often. Check each vendor's own page on the day you decide.

**LiteLLM alternatives** became a common search after 24 March 2026, when two LiteLLM releases on PyPI were found to contain credential-stealing code. The short answer: alternatives include self-hosted open-source gateways (Portkey's gateway, Bifrost, Agent Router, Kong AI Gateway), managed gateways (Cloudflare AI Gateway, Vercel AI Gateway), aggregators (OpenRouter) and BYOK control planes such as INFRO. But switching products does not by itself fix the risk the incident exposed. Any gateway holds every provider key you have, and the controls around it matter more than the logo on it.

This article sets out what happened, from LiteLLM's own account and a security vendor's analysis, then the controls that apply to any gateway, then the alternatives, described only from their own pages.

## What happened in the LiteLLM supply-chain attack

LiteLLM's security update states that `litellm==1.82.7` and `litellm==1.82.8` were published to PyPI on 24 March 2026, starting at 10:39 UTC, and that both have been removed. LiteLLM believes the compromise originated in the Trivy dependency used in its CI/CD security scanning workflow; the attacker bypassed the official release workflow and uploaded the packages directly to PyPI[1].

Datadog Security Labs traced this to a wider campaign it attributes to TeamPCP. On 19 March, attackers published a malicious Trivy release and force-pushed most `trivy-action` tags to malicious commits; the tainted actions dumped CI runner memory and exfiltrated credentials. The same campaign reached Checkmarx KICS on 23 March, LiteLLM on 24 March and the Telnyx Python SDK on 27 March[2].

According to Datadog, the LiteLLM payload collected environment variables, SSH keys, cloud and Kubernetes credentials, Docker configuration, shell history, database credentials and CI/CD secrets, encrypted them and sent them to a lookalike domain. Version 1.82.7 ran when the proxy module was used; 1.82.8 added a `.pth` file that runs whenever the Python interpreter starts, which made it the higher-risk version[2].

### How LiteLLM responded

LiteLLM's update lists its response: the packages removed from PyPI, maintainer credentials rotated, Google's Mandiant engaged for forensics, new releases paused for a supply-chain review, confirmation that no malicious code reached `main`, cosign-signed Docker images from v1.83.0-nightly, and published SHA-256 checksums and scans for releases v1.78.0 to v1.82.6. On 30 March it announced a clean v1.83.0, built by a new CI/CD pipeline with isolated environments and stronger release gates[1].

LiteLLM also states who was not affected: the official LiteLLM Proxy Docker image, which pins dependencies in `requirements.txt`, LiteLLM Cloud, installs from GitHub source, and anyone who stayed on 1.82.6 or earlier[1]. That detail is the most useful lesson in the whole incident. The deployment path that pinned its dependencies was not exposed.

LiteLLM remains a widely used open-source project with an MIT license, and its proxy offers virtual keys, per-key and per-team budgets, fallbacks, logging integrations and guardrails[3]. Nothing in this incident is a reason to assume another gateway's release pipeline is safer without asking.

## Supply-chain controls for anything that holds every provider key

A gateway is a tier-zero service. It sees every prompt, and in most designs it holds a credential for every provider account you own. Datadog's guidance is to treat any install of the affected versions as a full-credential exposure, and its point that "reverting the compromised packages" is not complete remediation applies to every product in this article[2].

1. **Install by hash, not by version range.** pip's hash-checking mode installs only files whose hashes you recorded, and requires every dependency to be pinned once it is on. It protects against a package being altered and against a surprise upgrade through a loose specifier[4].
2. **Publish with short-lived credentials.** If you maintain anything that ships to PyPI, Trusted Publishing lets CI prove its identity over OIDC and receive a project-scoped token that expires after about 15 minutes, so a token stolen from runner memory stops working quickly[5].
3. **Pin your CI actions by commit, not tag.** The Trivy stage worked because tags were force-pushed to malicious commits[2]. A tag is a pointer someone else can move.
4. **Isolate the proxy.** Run the gateway in its own container or service account, with only its own secrets mounted, no cloud credentials beyond what it needs and egress limited to provider hosts. The payload harvested whatever the process could read[2].
5. **Scope the keys it holds.** Give the gateway inference keys only, one per provider account or project, never an organization admin key. See [least privilege for provider admin keys](https://infro.io/blog/provider-admin-key-least-privilege).
6. **Rehearse rotation.** Know how long it takes to replace every provider key the gateway holds, and every key your applications use to reach the gateway. The guide to [securing LLM API keys](https://infro.io/blog/secure-llm-api-keys) covers inventory and rotation order.

```text
# generated with: pip-compile --generate-hashes requirements.in
litellm==<verified version> \
    --hash=sha256:<hash recorded from your verified build>
# every transitive dependency is pinned and hashed the same way

# install with:
#   pip install --require-hashes -r requirements.txt
```

## The LiteLLM alternatives, compared

The useful comparison is operational: who runs the gateway, who holds the provider keys, whether your own provider accounts serve the traffic, what happens on failure, whether budgets see the spend, and what is recorded. The table summarises what each product's own pages say; the sections below give the detail and the caveats.

| Option | Hosting | Who holds provider keys | Your own provider accounts (BYOK) | Failover | Budgets | Observability |
| --- | --- | --- | --- | --- | --- | --- |
| LiteLLM proxy | You run it | Your deployment | Yes | Fallbacks, load balancing | Per key, team, user | Logging integrations |
| Portkey gateway (OSS) | You run it, or Portkey's cloud | Your deployment | Yes | Fallbacks, retries, load balancing | Not described in README | Usage analytics, local console |
| Bifrost | You run it | Your deployment | Yes | Automatic failover, load balancing | Virtual keys, team and customer budgets | Prometheus, tracing, logs |
| Agent Router (formerly Envoy AI Gateway) | You run it, or hosted by Tetrate | The router | Yes | Provider fallback | Token limits per team, app or model | OpenTelemetry GenAI |
| Kong AI Gateway | Self-hosted or Kong-hosted data planes, managed through Konnect | The gateway's provider entity | Yes | Load balancing and failover | Token rate limiting | Metrics, OpenTelemetry, audit logs |
| Cloudflare AI Gateway | Managed | Cloudflare Secrets Store, or per request | Yes, or Cloudflare billing | Retries and model fallbacks | Spend limits incl. BYOK on priced models | Analytics, logs |
| Vercel AI Gateway | Managed | Vercel team | Yes, falls back to Vercel's credentials | Provider and model fallbacks | Budgets exclude BYOK spend | Request logs |
| OpenRouter | Managed | OpenRouter | Yes, falls back to shared capacity by default | Provider fallbacks by default | Not compared here | Not compared here |
| INFRO | Managed | INFRO, sealed per connection | Only your own accounts | Across your own connections | Hard caps per org, project, member, key | Usage, traces, audit log |

### Self-hosted open-source gateways

Running your own gateway keeps keys and prompts inside your network. It also makes the supply chain yours: you choose versions, verify them and patch them.

- **Portkey's gateway** is MIT-licensed and starts with `npx @portkey-ai/gateway`, Docker, Node.js or a Cloudflare Workers deployment. Its README lists fallbacks, up to five retries with backoff, weighted load balancing across keys or providers and pre-built guardrails; it does not describe budgets[6].
- **Bifrost** is Apache 2.0, runs with `npx -y @maximhq/bifrost` or Docker, and lists automatic failover, load balancing across keys and providers, hierarchical budgets through virtual keys, Prometheus metrics, tracing and an MCP gateway[7].
- **Agent Router**, the new name of Envoy AI Gateway since it joined the Agentic AI Foundation on 10 September 2026, keeps its Apache 2.0 license, CRDs, `aigw` CLI and maintainers[8]. Its site lists provider fallback, token limits per team, app or model, provider credentials held and rotated at the router, OpenTelemetry GenAI telemetry and a filtered MCP tool catalog, on Kubernetes, standalone or hosted by Tetrate[9].
- **Kong AI Gateway** is managed through Konnect with data planes self-hosted, Kong-hosted or on Kubernetes. It lists token-based rate limiting, semantic caching, prompt guards and PII sanitising, load balancing and failover, and provider credentials centralised in an AI Model Provider entity[10].

### Managed gateways

A managed gateway moves patching and release engineering to the vendor. You are then trusting their pipeline, so ask how they build, sign and deploy it.

- **Cloudflare AI Gateway** lists analytics, logging, caching, rate limiting, and request retry and model fallback[11]. BYOK keys are stored in Cloudflare Secrets Store, and a provider authorization header on the request takes precedence over the stored key[12]. Its spend limits count BYOK traffic for models with known pricing and return `429` at the limit; Cloudflare notes that spend is tracked with eventual consistency, so a concurrent burst can briefly pass it[13].
- **Vercel AI Gateway** offers request logs, budgets per team, project, key or member, and provider and model fallbacks[14]. With BYOK, a failed request is retried on Vercel's own system credentials, billed to your AI Gateway balance, and BYOK spend is not counted in budgets[15].

### Aggregators

**OpenRouter** serves traffic through its own provider accounts or your keys. With BYOK it charges a fee of 5% of what the same usage would cost on OpenRouter, and by default falls back to its shared endpoints when your key fails or hits a rate limit; each key can be set to never use shared capacity[16]. Provider fallbacks are on by default and can be disabled per request with `allow_fallbacks: false`[17]. The [OpenRouter alternatives comparison](https://infro.io/blog/openrouter-alternatives-2026) covers the commercial model in more depth.

### BYOK control planes

INFRO is a hosted gateway that serves traffic only through provider accounts you connect, and it bills a software subscription with no markup on model spend. Each provider key is sealed with AES-256-GCM, bound to your organization and that one connection, and never returned by any endpoint. Applications use INFRO gateway keys, which can be limited to chosen providers or IP ranges and rotated with an overlap window, so the old key keeps working while you deploy the new one. Failover moves between your own connections only; there is no shared pool to fall back to, so budgets, which are hard caps per organization, project, member and key, see all of the spend. Request traces, usage by key and project, and an audit log of key, connection and policy changes are recorded. A hosted gateway is still a party you trust with your keys: ask any vendor, including us, how its release pipeline is protected.

## Choosing after an incident

Decide by your constraints, not by the incident:

- **Prompts and keys must stay in your network.** Self-host. Budget the engineering for version pinning, isolation and upgrades, and compare the work with the [build vs buy analysis](https://infro.io/blog/build-vs-buy-llm-gateway).
- **You want no gateway to operate, and your own contracts must carry all traffic.** Check, for each managed option, what happens when your key fails and whether budgets count your own-key spend. Those two answers differ more than the feature lists.
- **You stay on LiteLLM.** Pin a verified version by hash, prefer the pinned Docker image path LiteLLM recommends, isolate the proxy, scope its keys, and keep up with its later security releases.

Whatever you choose, write the rotation runbook now and run it once. The incident's real cost fell on teams that had to discover, under pressure, which keys a compromised process could read.

## Questions

### Was the LiteLLM Docker image affected by the March 2026 compromise?

LiteLLM states that customers running the official LiteLLM Proxy Docker image were not affected, because that image pins its dependencies, and it lists LiteLLM Cloud, source installs from GitHub and installs that stayed on 1.82.6 or earlier as unaffected too. Environments that installed 1.82.7 or 1.82.8 from PyPI should be treated as fully exposed.

### Is LiteLLM safe to use after the supply-chain attack?

LiteLLM removed the two malicious releases, rotated maintainer credentials, engaged Mandiant, published checksums for earlier releases and shipped v1.83.0 from a rebuilt CI/CD pipeline. Whether it fits you is a question of your own controls: pin a verified version by hash, run the proxy isolated, and keep upgrading for later security fixes as LiteLLM publishes them.

### What is the difference between LiteLLM and Portkey's gateway?

Both are open-source gateways you can run yourself. LiteLLM is a Python SDK and proxy with virtual keys, budgets, fallbacks, logging integrations and guardrails. Portkey's gateway is an MIT-licensed gateway you can start with npx or Docker or deploy to Cloudflare Workers, with fallbacks, retries, load balancing and guardrails; its README does not describe budgets.

### Do I need to rotate provider keys if I never installed the bad versions?

Not because of this incident. Rotation is still worth rehearsing: a gateway holds every provider key you own, and the time to learn how long a rotation takes is before you need one.

## Sources

1. Security Update: Suspected Supply Chain Incident — LiteLLM. https://docs.litellm.ai/blog/security-update-march-2026 (accessed 2026-10-19)
2. LiteLLM and Telnyx compromised on PyPI: Tracing the TeamPCP supply chain campaign — Datadog Security Labs. https://securitylabs.datadoghq.com/articles/litellm-compromised-pypi-teampcp-supply-chain-campaign/ (accessed 2026-10-19)
3. LiteLLM AI Gateway (proxy server) — LiteLLM. https://docs.litellm.ai/docs/simple_proxy (accessed 2026-10-19)
4. Secure installs — pip (PyPA). https://pip.pypa.io/en/stable/topics/secure-installs/ (accessed 2026-10-19)
5. Trusted Publishers — Python Package Index. https://docs.pypi.org/trusted-publishers/ (accessed 2026-10-19)
6. Portkey-AI/gateway — GitHub. https://github.com/Portkey-AI/gateway (accessed 2026-10-19)
7. maximhq/bifrost — GitHub. https://github.com/maximhq/bifrost (accessed 2026-10-19)
8. Envoy AI Gateway is becoming Agent Router — Agent Router. https://theagentrouter.ai/blog/envoy-ai-gateway-is-now-agent-router (accessed 2026-10-19)
9. Agent Router — Agent Router. https://theagentrouter.ai/ (accessed 2026-10-19)
10. Kong AI Gateway — Kong. https://developer.konghq.com/ai-gateway/ (accessed 2026-10-19)
11. AI Gateway overview — Cloudflare. https://developers.cloudflare.com/ai-gateway/ (accessed 2026-10-19)
12. Bring your own keys (BYOK) — Cloudflare. https://developers.cloudflare.com/ai-gateway/configuration/bring-your-own-keys/ (accessed 2026-10-19)
13. Spend limits — Cloudflare. https://developers.cloudflare.com/ai-gateway/features/spend-limits/ (accessed 2026-10-19)
14. Vercel AI Gateway: Models, Routing, and Observability — Vercel. https://vercel.com/docs/ai-gateway (accessed 2026-10-19)
15. Bring Your Own Key (BYOK) to AI Gateway — Vercel. https://vercel.com/docs/ai-gateway/authentication-and-byok/byok (accessed 2026-10-19)
16. BYOK — OpenRouter. https://openrouter.ai/docs/guides/overview/auth/byok (accessed 2026-10-19)
17. Provider selection — OpenRouter. https://openrouter.ai/docs/guides/routing/provider-selection (accessed 2026-10-19)