Azure OpenAI and Bedrock behind one gateway: ids and quotas

Teams end up with both Azure OpenAI and Amazon Bedrock for ordinary reasons: a cloud commitment on each side, a model only one of them offers, a second source of capacity for a critical feature, or two business units that chose differently. Both clouds now expose an OpenAI-style API, which makes it tempting to treat them as interchangeable endpoints. They are not. The same request body means different things on each, is counted against different limits, and runs in places chosen by different controls.

This guide lines the two up concept by concept, then shows how to put them behind one stable name.

What model means on each cloud

On Azure OpenAI, you create a deployment of a model and give it a name, and the API's model parameter carries that deployment name, not the model's name. Microsoft's v1 examples say so in the code comment: replace the value "with your model deployment name"¹. Two deployments of the same model in one resource can have different names, quotas and deployment types.

On Bedrock, model carries an id AWS defines. For many newer models on the bedrock-runtime endpoint it has to be a cross-Region inference profile id, not the bare model id. GPT-5.6 Sol is an example: on bedrock-runtime it is called as us.openai.gpt-5.6-sol or global.openai.gpt-5.6-sol, and in-Region inference is not supported there². The prefix is not decoration: it selects where the request may run.

So the first job of anything that sits in front of both clouds is a mapping from one name your application uses to each cloud's own id.

Two OpenAI-style endpoints, two different surfaces

Azure's v1 API takes the standard OpenAI client with a base URL of https://YOUR-RESOURCE-NAME.openai.azure.com/openai/v1/, with no api-version parameter, and authenticates with an api-key header or a Microsoft Entra ID token¹.

Bedrock serves OpenAI Chat Completions at https://bedrock-runtime.{region}.amazonaws.com/openai/v1, authenticated with SigV4 or an Amazon Bedrock API key sent as a bearer token³. It does not implement the OpenAI GET /models operation on that endpoint, so client.models.list() does not work there³.

import os
from openai import OpenAI

azure = OpenAI(
    base_url="https://my-company-openai.openai.azure.com/openai/v1/",
    api_key=os.environ["AZURE_OPENAI_API_KEY"],
)
bedrock = OpenAI(
    base_url="https://bedrock-runtime.us-east-1.amazonaws.com/openai/v1",
    api_key=os.environ["AWS_BEARER_TOKEN_BEDROCK"],
)

messages = [{"role": "user", "content": "Summarise this ticket in two lines."}]

azure.chat.completions.create(model="gpt56-sol-prod", messages=messages)          # your deployment name
bedrock.chat.completions.create(model="us.openai.gpt-5.6-sol", messages=messages)  # AWS's profile id

The important difference is which models answer on those endpoints. On Azure, the GPT-5.6 models list the Chat Completions API among their capabilities⁴. On Bedrock, API support is per model and per endpoint. AWS's compatibility table lists Chat Completions for OpenAI, DeepSeek, Mistral, Qwen and several other families, while the Anthropic Claude rows list Invoke, Converse and Messages without Chat Completions⁵. The Claude Sonnet 5 model card is explicit: on bedrock-runtime, Chat Completions is not supported⁶. A Chat Completions call for Claude on that endpoint is not a supported combination, however correct the id mapping is.

Credentials

Azure resource keys and Entra ID tokens are both documented for the v1 API¹. Bedrock has two kinds of API key: short-term keys last up to 12 hours and are recommended for production, and long-term keys last until a configured expiry and are recommended "only for exploration"⁷. AWS also provides an IAM action, bedrock:CallWithBearerToken, with a condition key that can restrict which key type an identity may use⁷.

A long-lived credential stored in a gateway is therefore a trade-off on Bedrock, and it should be made knowingly: set an expiry, attach the narrowest Bedrock policy, and rotate before expiry rather than after a failure.

Quotas: both say TPM, neither means the same thing

Azure assigns quota "on a per-region, per-model, per-deployment-type basis in units of Tokens-per-Minute"; each deployment gets a share, and an RPM limit is set in proportion to it⁸. The TPM check uses an estimate made when the request arrives, and the estimate includes max_tokens, so an oversized max_tokens consumes limit even when the answer is short⁸. RPM is enforced over short windows, typically 1 or 10 seconds, so a burst can be throttled while the per-minute total is under the limit⁸. Throttled responses carry retry-after-ms⁸.

Bedrock's bedrock-runtime quotas are tokens per minute per model per Region, with separate quotas for on-demand and cross-Region inference, and they are shared by every API you call the model with, Chat Completions included⁹. Output tokens are converted into quota at a model-specific burndown rate⁹; for GPT-5.6 Sol, AWS states one output token consumes ten².

Azure also distinguishes kinds of 429: exceeding your rate limit, transient system capacity, and a temporary reduction of a Standard deployment's effective limit when shared capacity is under pressure, visible when x-ratelimit-limit-tokens is below your configured TPM⁸.

Where inference runs

On Azure, location is a property of the deployment type. Global types may process data in any Azure region; Data Zone types keep processing inside the US, EU or APAC data zone; Standard and Regional Provisioned keep it within the chosen Azure geography, while data at rest stays in the designated geography for all of them¹⁰.

On Bedrock, location is a property of the model id. Geographic profiles route within a geography such as the US or EU; global profiles route to any supported commercial Region¹¹. CloudTrail records the Region that processed each cross-Region request in additionalEventData.inferenceRegion¹¹.

That symmetry matters when the two clouds back each other up: a failover from an Azure Data Zone deployment to a Bedrock global profile moves the request's processing location. Pair targets whose locations your policy accepts.

Connecting Azure OpenAI and Bedrock in INFRO

INFRO has a connection kind for each, and both call your own accounts with your own credentials; each cloud bills you directly (connect a provider).

• Azure OpenAI asks for the resource name and an API key. INFRO calls https://{resource}.openai.azure.com/openai/v1 with the key in the api-key header. Each catalog model you route is mapped to your deployment name on the connection; an unmapped model is sent with its model name as the deployment name.
• AWS Bedrock asks for a long-lived Bedrock API key and a Region, and calls the bedrock-runtime OpenAI-compatible endpoint in that Region. Bedrock ids are Region-specific, so nothing is guessed: a model is served through the connection only when you name it.

Because the Bedrock connection speaks Chat Completions, it reaches only models whose Bedrock model card lists Chat Completions on bedrock-runtime. For Claude, connect your Anthropic account directly today. And because it stores a long-lived key, apply the AWS guidance above: an expiring key, a narrow policy, and rotation with POST /provider_connections/{id}/credential, which takes effect on the next request (management API resources).

curl https://api.infro.io/manage/v1/provider_connections \
  -H "Authorization: Bearer $INFRO_MANAGEMENT_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "kind": "azure-openai",
    "name": "Azure EU production",
    "environment": "production",
    "secret": "'"$AZURE_OPENAI_API_KEY"'",
    "config": {"resource": "my-company-openai"},
    "model_map": {"openai/gpt-5.6-sol": "gpt56-sol-prod"},
    "timeout_ms": 60000,
    "validate": true
  }'

Then one route gives the application a stable name over both:

{
  "slug": "chat-primary",
  "strategy": "priority",
  "max_retries": 0,
  "timeout_ms": 20000,
  "targets": [
    {"connection_id": "conn_azure_eu", "model": "openai/gpt-5.6-sol"},
    {"connection_id": "conn_bedrock_us", "model": "us.openai.gpt-5.6-sol"}
  ]
}

The second target names the exact profile id Bedrock expects. A target whose model is not an INFRO catalog id is sent as written and recorded as unpriced, so its usage.cost reads null rather than a guess (chat completions). Under a blocking budget, a request to an unpriced model is refused by default with 402 unpriced_model_under_budget (errors); decide that setting deliberately before the fallback is needed. Send a test request through each target and read its record before relying on the chain.

A provider 429 does not count against a connection's health, and it moves the request to the next target; if every target is throttled, the application receives 429 provider_rate_limited with the provider's Retry-After (connect a provider). Every attempt, with the connection that made it, is on the request's trace, which is how you find out that the Azure deployment's quota, not the model, is what failed.

Note what the route does not do: it does not make the two targets identical. They are the same model family on two clouds, with different quota behaviour, different feature support per API and different processing locations. Evaluate the fallback target as its own deployment, with the same prompts, before depending on it.

Checklist

• One stable application-side name per workload, mapped to an Azure deployment name and a Bedrock id.
• Every mapped Bedrock model checked on its model card for Chat Completions on bedrock-runtime.
• max_tokens sized to real answers, so Azure's estimate and Bedrock's burndown do not waste quota.
• Azure deployment type and Bedrock profile prefix chosen to match your location policy, for primary and fallback alike.
• Bedrock key with an expiry, a narrow policy and a rotation date in the calendar.
• 429s classified by cloud and cause in dashboards, not summed into one error rate.
• Failover tested by disabling the primary connection and finding the Bedrock attempt on the trace.