# Azure OpenAI and Bedrock behind one gateway: ids and quotas

URL: https://infro.io/blog/multi-cloud-azure-bedrock-one-gateway
Author: INFRO Editorial (https://infro.io/blog/editorial)
Publisher: INFRO
Published: 2026-09-29
Last modified: 2026-09-29
Last reviewed: 2026-09-29 (next review due 2026-10-29)
Topic: Architecture and migration
Format: Guide

> Azure OpenAI addresses a deployment you named and meters it in tokens per minute you assign; Bedrock addresses a model or inference profile id and meters tokens per minute per model and Region. Put both behind one gateway by mapping a stable model name to each cloud's own id, reading each cloud's 429s on their own terms, and checking per model that the API you call is supported on that cloud.

Key takeaways:
- On Azure, `model` is a deployment name you chose. On Bedrock, it is a model id or an inference profile id AWS chose.
- Both clouds meter tokens per minute, but Azure counts an estimate that includes max_tokens and Bedrock can burn output tokens at a model-specific rate.
- Not every model supports Chat Completions on every cloud. On Bedrock, check the model card before mapping a model.
- Where inference runs is a deployment choice on Azure and a model-id choice on Bedrock; a global option trades location for capacity.
- Put both behind one stable name, and let failover move a request only before its first token.

Teams end up with both Azure OpenAI and Amazon Bedrock for ordinary reasons: a cloud commitment on each side, a model only one of them offers, a second source of capacity for a critical feature, or two business units that chose differently. Both clouds now expose an OpenAI-style API, which makes it tempting to treat them as interchangeable endpoints. They are not. The same request body means different things on each, is counted against different limits, and runs in places chosen by different controls.

This guide lines the two up concept by concept, then shows how to put them behind one stable name.

[Diagram: The same six questions, answered by each cloud. Read from each provider's documentation on 2026-09-29.]

## What `model` means on each cloud

On Azure OpenAI, you create a **deployment** of a model and give it a name, and the API's `model` parameter carries that deployment name, not the model's name. Microsoft's v1 examples say so in the code comment: replace the value "with your model deployment name"[1]. Two deployments of the same model in one resource can have different names, quotas and deployment types.

On Bedrock, `model` carries an id AWS defines. For many newer models on the `bedrock-runtime` endpoint it has to be a cross-Region **inference profile** id, not the bare model id. GPT-5.6 Sol is an example: on `bedrock-runtime` it is called as `us.openai.gpt-5.6-sol` or `global.openai.gpt-5.6-sol`, and in-Region inference is not supported there[2]. The prefix is not decoration: it selects where the request may run.

So the first job of anything that sits in front of both clouds is a mapping from one name your application uses to each cloud's own id.

## Two OpenAI-style endpoints, two different surfaces

Azure's v1 API takes the standard OpenAI client with a base URL of `https://YOUR-RESOURCE-NAME.openai.azure.com/openai/v1/`, with no `api-version` parameter, and authenticates with an `api-key` header or a Microsoft Entra ID token[1].

Bedrock serves OpenAI Chat Completions at `https://bedrock-runtime.{region}.amazonaws.com/openai/v1`, authenticated with SigV4 or an Amazon Bedrock API key sent as a bearer token[3]. It does not implement the OpenAI `GET /models` operation on that endpoint, so `client.models.list()` does not work there[3].

```python
import os
from openai import OpenAI

azure = OpenAI(
    base_url="https://my-company-openai.openai.azure.com/openai/v1/",
    api_key=os.environ["AZURE_OPENAI_API_KEY"],
)
bedrock = OpenAI(
    base_url="https://bedrock-runtime.us-east-1.amazonaws.com/openai/v1",
    api_key=os.environ["AWS_BEARER_TOKEN_BEDROCK"],
)

messages = [{"role": "user", "content": "Summarise this ticket in two lines."}]

azure.chat.completions.create(model="gpt56-sol-prod", messages=messages)          # your deployment name
bedrock.chat.completions.create(model="us.openai.gpt-5.6-sol", messages=messages)  # AWS's profile id
```

The important difference is which models answer on those endpoints. On Azure, the GPT-5.6 models list the Chat Completions API among their capabilities[4]. On Bedrock, API support is per model and per endpoint. AWS's compatibility table lists Chat Completions for OpenAI, DeepSeek, Mistral, Qwen and several other families, while the Anthropic Claude rows list Invoke, Converse and Messages without Chat Completions[5]. The Claude Sonnet 5 model card is explicit: on `bedrock-runtime`, Chat Completions is not supported[6]. A Chat Completions call for Claude on that endpoint is not a supported combination, however correct the id mapping is.

## Credentials

Azure resource keys and Entra ID tokens are both documented for the v1 API[1]. Bedrock has two kinds of API key: short-term keys last up to 12 hours and are recommended for production, and long-term keys last until a configured expiry and are recommended "only for exploration"[7]. AWS also provides an IAM action, `bedrock:CallWithBearerToken`, with a condition key that can restrict which key type an identity may use[7].

A long-lived credential stored in a gateway is therefore a trade-off on Bedrock, and it should be made knowingly: set an expiry, attach the narrowest Bedrock policy, and rotate before expiry rather than after a failure.

## Quotas: both say TPM, neither means the same thing

Azure assigns quota "on a per-region, per-model, per-deployment-type basis in units of Tokens-per-Minute"; each deployment gets a share, and an RPM limit is set in proportion to it[8]. The TPM check uses an estimate made when the request arrives, and the estimate includes `max_tokens`, so an oversized `max_tokens` consumes limit even when the answer is short[8]. RPM is enforced over short windows, typically 1 or 10 seconds, so a burst can be throttled while the per-minute total is under the limit[8]. Throttled responses carry `retry-after-ms`[8].

Bedrock's `bedrock-runtime` quotas are tokens per minute per model per Region, with separate quotas for on-demand and cross-Region inference, and they are shared by every API you call the model with, Chat Completions included[9]. Output tokens are converted into quota at a model-specific burndown rate[9]; for GPT-5.6 Sol, AWS states one output token consumes ten[2].

> **Example: Why a quota looks bigger than it is**
> A deployment receives requests with about 2,000 prompt tokens, `max_tokens` set to 8,000, and answers of about 300 tokens. Azure's rate-limit estimate counts roughly 10,000 tokens per request at arrival, not 2,300. On a model with a 10x output burndown on Bedrock, the same 300-token answer consumes about 3,000 tokens of quota. Both effects make a workload hit 429s well below the token counts on its bill. Illustrative numbers.

Azure also distinguishes kinds of 429: exceeding your rate limit, transient system capacity, and a temporary reduction of a Standard deployment's effective limit when shared capacity is under pressure, visible when `x-ratelimit-limit-tokens` is below your configured TPM[8].

[Diagram: A 429 is a question, not an answer. The right response depends on which limit produced it.]

## Where inference runs

On Azure, location is a property of the deployment type. Global types may process data in any Azure region; Data Zone types keep processing inside the US, EU or APAC data zone; Standard and Regional Provisioned keep it within the chosen Azure geography, while data at rest stays in the designated geography for all of them[10].

On Bedrock, location is a property of the model id. Geographic profiles route within a geography such as the US or EU; global profiles route to any supported commercial Region[11]. CloudTrail records the Region that processed each cross-Region request in `additionalEventData.inferenceRegion`[11].

That symmetry matters when the two clouds back each other up: a failover from an Azure Data Zone deployment to a Bedrock global profile moves the request's processing location. Pair targets whose locations your policy accepts.

## Connecting Azure OpenAI and Bedrock in INFRO

INFRO has a connection kind for each, and both call your own accounts with your own credentials; each cloud bills you directly ([connect a provider](https://infro.io/docs/providers)).

- **Azure OpenAI** asks for the resource name and an API key. INFRO calls `https://{resource}.openai.azure.com/openai/v1` with the key in the `api-key` header. Each catalog model you route is mapped to your deployment name on the connection; an unmapped model is sent with its model name as the deployment name.
- **AWS Bedrock** asks for a long-lived Bedrock API key and a Region, and calls the `bedrock-runtime` OpenAI-compatible endpoint in that Region. Bedrock ids are Region-specific, so nothing is guessed: a model is served through the connection only when you name it.

Because the Bedrock connection speaks Chat Completions, it reaches only models whose Bedrock model card lists Chat Completions on `bedrock-runtime`. For Claude, connect your Anthropic account directly today. And because it stores a long-lived key, apply the AWS guidance above: an expiring key, a narrow policy, and rotation with `POST /provider_connections/{id}/credential`, which takes effect on the next request ([management API resources](https://infro.io/docs/manage/resources)).

```bash
curl https://api.infro.io/manage/v1/provider_connections \
  -H "Authorization: Bearer $INFRO_MANAGEMENT_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "kind": "azure-openai",
    "name": "Azure EU production",
    "environment": "production",
    "secret": "'"$AZURE_OPENAI_API_KEY"'",
    "config": {"resource": "my-company-openai"},
    "model_map": {"openai/gpt-5.6-sol": "gpt56-sol-prod"},
    "timeout_ms": 60000,
    "validate": true
  }'
```

Then one [route](https://infro.io/docs/routes) gives the application a stable name over both:

[Diagram: The application names a route. INFRO tries the Azure target first and moves to the Bedrock target only if Azure fails before the first token.]

```json
{
  "slug": "chat-primary",
  "strategy": "priority",
  "max_retries": 0,
  "timeout_ms": 20000,
  "targets": [
    {"connection_id": "conn_azure_eu", "model": "openai/gpt-5.6-sol"},
    {"connection_id": "conn_bedrock_us", "model": "us.openai.gpt-5.6-sol"}
  ]
}
```

The second target names the exact profile id Bedrock expects. A target whose model is not an INFRO catalog id is sent as written and recorded as unpriced, so its `usage.cost` reads `null` rather than a guess ([chat completions](https://infro.io/docs/api/chat-completions)). Under a blocking budget, a request to an unpriced model is refused by default with `402 unpriced_model_under_budget` ([errors](https://infro.io/docs/api/errors)); decide that setting deliberately before the fallback is needed. Send a test request through each target and read its record before relying on the chain.

A provider 429 does not count against a connection's health, and it moves the request to the next target; if every target is throttled, the application receives `429 provider_rate_limited` with the provider's `Retry-After` ([connect a provider](https://infro.io/docs/providers)). Every attempt, with the connection that made it, is on the request's [trace](https://infro.io/docs/observability/traces), which is how you find out that the Azure deployment's quota, not the model, is what failed.

Note what the route does not do: it does not make the two targets identical. They are the same model family on two clouds, with different quota behaviour, different feature support per API and different processing locations. Evaluate the fallback target as its own deployment, with the same prompts, before depending on it.

## Checklist

- [ ] One stable application-side name per workload, mapped to an Azure deployment name and a Bedrock id.
- [ ] Every mapped Bedrock model checked on its model card for Chat Completions on `bedrock-runtime`.
- [ ] `max_tokens` sized to real answers, so Azure's estimate and Bedrock's burndown do not waste quota.
- [ ] Azure deployment type and Bedrock profile prefix chosen to match your location policy, for primary and fallback alike.
- [ ] Bedrock key with an expiry, a narrow policy and a rotation date in the calendar.
- [ ] 429s classified by cloud and cause in dashboards, not summed into one error rate.
- [ ] Failover tested by disabling the primary connection and finding the Bedrock attempt on the trace.

## Sources

1. Azure OpenAI in Microsoft Foundry Models v1 API — Microsoft Learn. https://learn.microsoft.com/en-us/azure/foundry/openai/api-version-lifecycle (accessed 2026-09-29)
2. GPT-5.6 Sol model card — Amazon Bedrock — AWS. https://docs.aws.amazon.com/bedrock/latest/userguide/model-card-openai-gpt-56-sol.html (accessed 2026-09-29)
3. Chat Completions API — Amazon Bedrock — AWS. https://docs.aws.amazon.com/bedrock/latest/userguide/inference-chat-completions.html (accessed 2026-09-29)
4. Foundry Models sold by Azure — Microsoft Learn. https://learn.microsoft.com/en-us/azure/foundry/foundry-models/concepts/models-sold-directly-by-azure (accessed 2026-09-29)
5. API compatibility — Amazon Bedrock — AWS. https://docs.aws.amazon.com/bedrock/latest/userguide/models-api-compatibility.html (accessed 2026-09-29)
6. Claude Sonnet 5 model card — Amazon Bedrock — AWS. https://docs.aws.amazon.com/bedrock/latest/userguide/model-card-anthropic-claude-sonnet-5.html (accessed 2026-09-29)
7. API keys — Amazon Bedrock — AWS. https://docs.aws.amazon.com/bedrock/latest/userguide/api-keys.html (accessed 2026-09-29)
8. Manage Azure OpenAI in Microsoft Foundry Models quota — Microsoft Learn. https://learn.microsoft.com/en-us/azure/foundry/openai/how-to/quota (accessed 2026-09-29)
9. Quotas for the bedrock-runtime endpoint — AWS. https://docs.aws.amazon.com/bedrock/latest/userguide/quotas-runtime.html (accessed 2026-09-29)
10. Understanding deployment types in Microsoft Foundry Models — Microsoft Learn. https://learn.microsoft.com/en-us/azure/foundry/foundry-models/concepts/deployment-types (accessed 2026-09-29)
11. Route model inference requests across AWS Regions with cross-Region inference — AWS. https://docs.aws.amazon.com/bedrock/latest/userguide/cross-region-inference.html (accessed 2026-09-29)