Skip to content

Migration guidesFrom Kong AI Gateway

From Kong AI Gateway, or alongside it

Kong AI Gateway puts AI traffic on the same gateway as the rest of your APIs: AI Models with targets and balancers, AI Policies for limits, caching and guardrails, run on your own Kong data planes or Konnect. If you run Kong for everything else, that is a strong reason to stay. This page is for teams who want the AI part as a hosted control plane instead.

AI Gateway 2.x (2.0 on 2026-07-10, 2.2 on 2026-09-30); Konnect-hosted or self-hosted

INFRO accounts are self-serve. The catalog, the docs, and the API contract are public, the gateway is deployed, and anything beyond v1 is marked Later. Plans and trial

Where things stand

Version 2

Kong AI Gateway is versioned on its own: 2.0.0 shipped on 2026-07-10 and 2.2.0 on 2026-09-30. 2.x replaces the AI Proxy plugins on Services and Routes with entities: AI Model, AI Model Provider, AI Policy, AI Auth Strategy, AI Consumer Group, AI MCP Server.

Whose account serves the request

Yours. Provider credentials are declared on the AI Model Provider and injected by the gateway; Kong does not sell inference.

Who runs it

Konnect hosts the control plane; data planes run on your infrastructure, Kubernetes or Kong's Dedicated Cloud Gateways. The OpenAI SDK how-to for on-prem says it requires Kong Gateway Enterprise.

Kong is a general API gateway with deep AI policies. Moving AI traffic off it is a real cost, and these are the cases where it does not pay.

  • Kong already fronts your APIs, and your platform team operates it. One gateway, one policy model, one on-call.
  • Traffic must stay on your own data planes or inside your network. INFRO's plans are hosted; self-hosted or hybrid is an enterprise agreement.
  • You use semantic caching, the semantic or lowest-latency balancer, prompt templates or the prompt compressor. INFRO has exact-match caching, priority and weighted routes, and no prompt tooling.
  • You rely on AWS Guardrails, Azure Content Safety or GCP Model Armor through Kong policies, or on A2A traffic. INFRO has its own guardrails and no A2A.
  • You proxy embeddings, files, batches or assistants. INFRO does not serve them.

The edits, one row each. Everything not listed here stays as it is.

 Kong AI GatewayINFRO
Base URLYour Kong proxy or $KONNECT_PROXY_URL plus the AI Model's route path.https://api.infro.io/v1
KeyA Kong consumer key in apikey (key-auth) or an OIDC bearer token.An INFRO gateway key in Authorization: Bearer. Provider credentials move from AI Model Providers to INFRO connections.
Model idThe upstream model name (gpt-4o) or an alias set by config.route.model.vendor/model, or a route whose name is your old alias, so callers keep sending it.
HeadersRead from responses: X-Cache-Status, X-AI-RateLimit-*.X-INFRO-Cache and X-RateLimit-Limit on responses; budgets answer 402 in the body.

Before · Kong AI Gateway

from openai import OpenAI

kong_url = "https://kong.internal.example.com"
kong_route = "ai"

client = OpenAI(
    api_key="my-consumer-key",
    base_url=f"{kong_url}/{kong_route}",
)

client.chat.completions.create(
    model="my-gpt",  # alias on the AI Model
    messages=[{"role": "user", "content": "Hello"}],
)

After · INFRO

import os
from openai import OpenAI

client = OpenAI(
    api_key=os.environ["INFRO_API_KEY"],
    base_url="https://api.infro.io/v1",
)

client.chat.completions.create(
    model="my-gpt",  # an INFRO route with the same name
    messages=[{"role": "user", "content": "Hello"}],
)

Aliases become routes

Each AI Model alias becomes a route with the same name and the same targets in order. A gateway key can be pinned to named routes so it cannot call a model directly, as a Kong consumer is confined to its routes.

Consumer groups become projects

Per-consumer and per-group token or cost limits become key spend limits and project budgets. Request-rate tiers per key are reported in X-RateLimit-Limit.

Run both while you move

Nothing stops Kong and INFRO from running side by side. INFRO can also be an upstream of Kong, since it answers the OpenAI shape.

Kong AI Gateway's objects against INFRO's, from Kong AI Gateway's public documentation read on 19 October 2026 and checked against what the gateway implements. Two rows say INFRO has nothing.

Kong AI GatewayWhat it does thereIn INFRO
AI Model Providersame shapeReusable provider definition holding credentials.A provider connection, sealed, several keys per provider with standby keys last.
AI Model and targetssame shapeA virtual model with targets and a load balancer.A route with ordered or weighted targets, if-rules on the request and failure rules per kind.
Balancer algorithmsdifferentRound-robin, consistent hashing, least connections, lowest latency, lowest usage, semantic, priority.Priority and weighted only, plus a prompt classifier that picks a class of targets. No latency, usage or semantic balancing.
Retries, failover criteria, circuit breakerdifferentRetries, failover_criteria, max_fails and fail_timeout.Failure rules per kind (retry, fail over or stop) for 429, 5xx, timeout, connection, credential and moderation. No circuit breaker setting.
AI Auth Strategy and consumersdifferentkey-auth or OIDC identities, grouped for policies.Gateway keys per service, with allowlists, IP ranges and environments. Members sign in with SSO; gateway calls use keys.
AI Rate Limiting AdvanceddifferentToken or cost limits per consumer, group, model, provider, header or path.Budgets that refuse at a dollar ceiling per organization, project, member, key and end customer; request-rate tiers per key. No token-count windows.
AI Semantic CachedifferentSemantic and exact caching in Redis, Valkey or pgvector.Exact-match only, opt-in, for chat completions.
Prompt Guard, Sanitizer, Semantic guardsdifferentPattern and semantic prompt rules, PII sanitising, response guards.Guardrails that block, redact or flag by pattern, plus checks run on a decision model. No semantic similarity guard.
Cloud guardrail integrationsnothingAWS Guardrails, Azure Content Safety, GCP Model Armor.Nothing.
Prompt Template, Prompt CompressornothingTemplated prompts and prompt compression in the gateway.Nothing. Prompts stay in your code.
AI MCP Serversame shapeMCP traffic on the same gateway.An MCP gateway for your own MCP servers, with approved tools and grants per key.
AnalyticsdifferentToken usage, latency and cost in Konnect analytics.Estimated provider cost per request, reconciled daily against OpenAI's and Anthropic's own cost APIs where you add an admin key.
Self-hosted data planesdifferentRun on your own infrastructure.Hosted. Self-hosted or hybrid is an enterprise agreement.Enterprise — Available on an enterprise agreement rather than self-serve.

Our reading of Kong AI Gateway’s published documentation, not a trial or a benchmark. If we have read something wrong, tell us and we will fix the row.

What INFRO does not do

  • Gateway your non-AI APIs.
  • Semantic caching or semantic, latency- or usage-based balancing.
  • Prompt templates or compression.
  • Integrate cloud guardrail services, or carry A2A traffic.
  • Proxy embeddings, files, batches or assistants.
  • Run on your own data planes without an enterprise agreement.

What does not come across

  • Konnect analytics and logs. There is no importer.
  • Policies as configuration (decK). Recreate them as routes, budgets, allowlists and guardrails; INFRO's management API can create them from a script.

Questions

Can INFRO and Kong run together?
Yes. INFRO answers the OpenAI shape, so it can be an upstream of Kong, or services can move from Kong to INFRO one at a time.
Can I keep my model aliases?
Yes. Create an INFRO route with the alias as its name; callers keep sending the same model.
Who pays for model usage?
You do, directly to your providers, as you do with Kong. INFRO is a subscription with 0% markup on model spend.

Move with us, or on your own

The migration program maps your Kong AI Gateway setup to INFRO connections, routes and budgets on a call and moves the first service with you. Your providers keep billing you directly; INFRO is a subscription and adds 0% to model spend.

Start 14-day trial Book a migration sessionINFRO vs Kong AI GatewayRead the quickstart

Sources

Every statement on this page about Kong AI Gateway comes from one of these, read on 19 October 2026. Nothing is inferred from a trial, a benchmark, or a conversation.