AI gateway in Azure API Management and Microsoft Foundry
INFRO vs Azure AI Gateway
Azure's AI gateway is a set of AI capabilities in Azure API Management, also switched on from Microsoft Foundry for model deployments there. It fronts Foundry and Azure OpenAI, other providers' OpenAI-, Anthropic- and Vertex-shaped APIs, self-hosted models, MCP servers and agent APIs, and applies APIM policy to them: token-per-minute limits and token quotas per key, semantic caching, Azure AI Content Safety checks, weighted and priority load balancing with circuit breakers, and token metrics in Azure Monitor. For an enterprise that already runs API Management, it extends a gateway the platform team operates today, and that is hard to beat.
- You already run Azure API Management. Your AI traffic gets the same instance, networking, identity, developer portal and policy language as the rest of your APIs, operated by the team that already owns them.
- You are an Azure-native enterprise on Foundry. Turning on the gateway from the Foundry portal gives each project its own token limits and quotas on a shared instance, with managed identity to the models and no API keys to hand out.
- Semantic caching is the requirement. APIM returns cached answers for prompts similar in meaning, using Azure Managed Redis and an embeddings deployment; INFRO's response cache is exact-match only, for chat completions.
- You need a programmable policy pipeline. APIM policies with expressions can transform, route and limit on anything in the request; INFRO's controls are fixed features configured in a console and an API, not a policy language.
- Your capacity is Provisioned Throughput and the problem is tokens per minute, not dollars. Token limits and priority load balancing across PTU and pay-as-you-go deployments are what APIM's AI policies are built around.
This section is not a courtesy. If one of these describes you, pick Azure AI Gateway — a customer who adopts the wrong layer on our advice is worse for us than one we never had.
Where the two actually diverge.
- A gateway you operate, or a service you use
- Azure's AI gateway is API Management: an instance in your subscription, at a service tier you choose, configured with XML policies and operated by your platform team — Foundry creates a Basic v2 instance by default and recommends Standard v2 or Premium v2 for production. That is the right shape for a company that already runs APIM and has the people for it. INFRO's plans are hosted, with budgets, routes, allowlists and an audit log configured in a console or through its management API, and nothing to deploy; an enterprise agreement can put it in your own infrastructure instead.
- Tokens, or dollars
- APIM's llm-token-limit policy limits tokens: a rate per minute, a quota per hour, day, week, month or year, or both, keyed by subscription, IP address or any expression, answering 429 or 403. Tokens are the right unit for sharing a deployment's capacity. They are the wrong unit for a budget, because a token on one model and a token on another differ in price by orders of magnitude, and a quota written in tokens has to be rewritten every time a team changes model. INFRO's budgets are dollars of estimated provider cost — at your negotiated rates from the Team plan — at organization, project, member and key, checked before the provider is called and refused with 402 when there is no room.
- Cost as a record, not a metric to build
- APIM emits token counts to Azure Monitor and Application Insights with whatever dimensions you add, and logs prompts and completions if you turn it on. Turning those into cost by team means pricing the tokens yourself. INFRO prices every request when it is recorded, says which rate answered — your negotiated rate, the vendor's published rate, or unpriced — and attributes it to a project, key, model, route or end user. On the Business plan the month's estimate is checked against OpenAI's and Anthropic's own cost APIs and the variance is stated.
- Not either-or
- Azure OpenAI is one of the provider kinds INFRO connects, through your resource's deployments and its own key. A company can keep APIM in front of its Azure estate and use INFRO for the budgets, attribution and audit trail across that estate and the direct provider contracts beside it.
Side by side.
Our column carries dashes too. A comparison where one side is all checks is one nobody believes, and rightly.
| Capability | Azure AI Gateway | INFRO |
|---|---|---|
| One API across many providers | yesFoundry, Azure OpenAI, OpenAI-, Anthropic- and Vertex-shaped APIs | yeswhatever your connected providers serve |
| Nothing to deploy or operate | noan API Management instance in your subscription | yeshosted service |
| Limits in tokens | yesper minute and per period, by any key | norequest rate limits per key instead |
| Budgets in dollars that refuse a request | notoken quotas only | yesorganization, project, member, key; 402 |
| Cost per request, attributed | partialtoken metrics by custom dimension | yesproject, key, model, route, end user |
| Response caching | yessemantic, with Azure Managed Redis | partialexact match, chat completions only |
| Content safety | yesAzure AI Content Safety policy | partialpatterns plus optional classifier checks; chat completions; Business plan |
| Load balancing and failover | yesweighted, priority, circuit breaker | yespriority or weighted routes, any modality |
| Programmable policy | yesAPIM policy expressions | noconfigured features, not a policy language |
| MCP servers and agent APIs | yesMCP and A2A | partialMCP gateway, Business plan |
| Available today | yes | noprivate early access |
Our reading of each product's public documentation as of October 2026. We do not quote competitors' prices — theirs change and a wrong number about someone else's business is worse than none. Think a row is wrong? Tell us and we will fix it.
Work out what the difference is worth
Estimate what your current model mix costs at your providers' list rates and the plan it needs, or have us read your real usage and tell you where we would and would not help.