"We need to be multi-region" usually arrives as one requirement and turns out to be four. A reliability review wants the feature to survive a regional outage. A capacity plan wants more tokens per minute than one Region grants. A product manager wants faster answers for users on another continent. A legal review wants personal data processed inside a jurisdiction.
Each of those is a legitimate goal, and each needs a different control. Worse, they conflict: the option with the most capacity is often the one with the least control over location, and a failover target added for availability can move data somewhere the legal review never approved. Separate the goals before you design anything.
Why do you want more than one region?
- A provider or Region outage must not stop the featureTwo connections in different Regions or providers, one route, failover.
- One Region's quota is too smallSplit load with a weighted route across deployments or accounts.
- Users far from your provider endpoint wait too longMeasure first: the gateway hop and the provider endpoint both add distance.
- Data must stay in a jurisdictionPin processing at the provider, and review every fallback target. INFRO regional processing is roadmap.
Where is a request, exactly?
For a request that goes through a gateway to a hosted model, there are up to five locations worth naming. Most multi-region confusion comes from treating them as one.
1 · client
Your application's regionWhere the request starts. You choose it.2 · gateway
INFRO on CloudflareEnters at the nearest edge; the Worker is placed near its primary database in Frankfurt. A performance setting, not residency.3 · endpoint
The provider endpoint you connectedAn Azure resource, a Bedrock Region, api.openai.com, or your own server.4 · processing
Where the provider runs inferenceSet by the provider product: deployment type, inference profile or data residency project.5 · records
What INFRO storesRequest metadata always; payloads only if you opt in. No regional pinning today: residency is roadmap.
Layer 4 is the one most regulations care about, and it is set almost entirely at the provider. Layer 3, the endpoint you call, is not the same thing: an Azure resource in an EU region with a Global deployment may still process prompts in any Azure region1.
Goal 1: availability across regions
A regional outage at a provider affects everything routed to that Region. Microsoft notes that with Global Standard and Data Zone Standard deployments, "if the primary region experiences an interruption in service, all traffic initially routed to this region is affected"1. The remedy is a second, independent place to send the request: a deployment in another Region, an account at another provider, or both.
What makes the second target useful is independence. Two deployments behind the same quota, the same account policy and the same model version share more failure modes than two different providers do. And the second target has to be proven, not assumed: disable the primary in a drill and confirm the standby answers.
Goal 2: more quota than one Region grants
Provider quotas are regional. Azure assigns quota per region, per model and per deployment type, in tokens per minute, and its guidance for workloads that need more than one deployment supports is to "spread requests across multiple deployments or regions"2. Bedrock's bedrock-runtime quotas are per model, per Region, with on-demand and cross-Region inference counted separately3.
The provider-managed alternative is a global option. Azure's Global Standard "provides the highest default quota and eliminates the need to load balance across multiple resources"1; Bedrock's global inference profiles route to any supported commercial Region4. Both buy capacity by giving up control of layer 4. If location does not matter for a workload, that is often the simplest answer. If it does, spread load across regional deployments yourself.
Goal 3: latency for distant users
Latency is where intuition fails most often, because a model call's time is dominated by the model: queueing, prompt processing and token generation. Network distance matters, but it is usually a smaller share than it is for an ordinary API call, and it can be spent in places that are not obvious. A gateway is one more hop, and where it runs matters as much as where the provider runs.
Measure before moving anything. From the client, record time to first token and total time per provider endpoint. Then compare with the provider-side timing where one is available. Moving a workload to a nearer Region only helps if distance, not generation, is where the time goes.
Goal 4: keeping data in a jurisdiction
Each provider offers its own control over where inference runs.
| Provider control | Keeps inference inside | Set where |
|---|---|---|
| Azure Data Zone Standard | The US, EU or APAC data zone | Deployment type on the deployment |
| Azure Standard | The Azure geography | Deployment type on the deployment |
| Azure Global Standard | ○Any Azure region | Deployment type on the deployment |
| Bedrock geo profile (us., eu.) | The profile's geography | The model id you send |
| Bedrock global profile | ○Any supported commercial Region | The model id you send |
| OpenAI data residency project | US, Europe or UAE for processing; more regions for storage only | Project setting and regional domain |
| Self-hosted server | Wherever you run it | Your infrastructure |
- Azure OpenAI sets it by deployment type. Data Zone deployments process prompts and responses only within the US, EU or APAC data zone; Standard keeps processing within the chosen Azure geography; Global may process in any Azure region. Data stored at rest remains in the designated geography for all types1.
- Amazon Bedrock sets it by the model id you send. A geographic inference profile keeps routing within a geography such as the US or EU; a global profile routes to any supported commercial Region4. CloudTrail records the Region that processed each cross-Region request4.
- OpenAI sets it per project. Data residency is a project configuration with regional domains such as
eu.api.openai.com; regional processing is offered for the US, Europe and the UAE, while several other regions offer storage only, and non-US regions require approval for abuse-monitoring controls and a retention amendment5.
Availability of a location option differs by model, too. On Bedrock's bedrock-runtime endpoint, GPT-5.6 Sol is offered through a US geographic profile or a global profile; in the EU Regions the model card lists only global inference for it6. A location policy therefore has to be checked per model, not per provider.
The rule that ties this to failover: every target in a chain must satisfy the same location policy as the primary. A route that falls back from an EU Data Zone deployment to a global profile keeps the feature up by moving the data. If that is acceptable during an outage, write it down as a decision. If it is not, the fallback has to be another in-zone target, or an error.
Multi-region routing with INFRO today
INFRO calls the provider accounts and deployments you connect, so layer 4 is governed by the provider settings above, on your accounts, under your agreements (EU compliance). What INFRO adds is the chain: several connections, one name, and failover before the first token.
- One connection per regional deployment or account. An Azure connection names one resource; a Bedrock connection names one Region; the model id you map decides geographic or global processing (connect a provider).
- A route per workload.
priorityfor a primary with in-policy standbys,weightedto split load across regional deployments; a target at weight 0 is a standby that never takes a first attempt (routes). - Scoped by environment. A connection or route scoped to
productionis invisible to a key that does not name that environment, which keeps a staging key from reaching production deployments (connect a provider). - Evidence per request. Each record names the connection that served and every failed attempt, so an audit can show which Region-bound deployment answered (request traces).
- Day 0Second connection addedDeployment or account in another Region, weight 0: a standby that takes no first attempts.
- Day 1Failover drillDisable the primary connection; confirm the standby serves and the record names it.
- Day 2Small shareGive the second target a small weight and compare latency and errors per connection.
- Day 7Target splitSettle weights from quota and measured latency, not from symmetry.
What INFRO does not do today matters as much:
- No regional processing guarantee at the gateway. INFRO runs on Cloudflare; requests enter at the nearest edge location and the gateway Worker is placed near its primary database in Frankfurt, which is a performance setting, not a residency guarantee (EU compliance). For users and providers far from Frankfurt, that placement is part of the latency you should measure. Pinning an organization or project to a region is on the roadmap and is not claimed before it is enforced.
routing.regionsis not a residency control. The legacy request field is recorded and checked against a stored region policy, but it does not constrain or rank your connections (smart routing). Location comes from which connections a route contains.- The OpenAI connection has no region setting. It calls
api.openai.com, so an OpenAI data residency project's regional domain is not selectable through it today.
On layer 5, prompt and response capture is off by default, can be disabled per request with logging: false, and INFRO's settings do not change what your provider retains (privacy).
Checklist
- The goal is named: availability, quota, latency, data location, or a stated combination.
- All five locations written down for each workload: client, gateway, endpoint, processing, records.
- Provider location control chosen per model: deployment type, profile prefix or residency project.
- Every fallback target checked against the same location policy as its primary.
- Quota headroom per Region recorded, with a weighted split where one Region is not enough.
- Client-side time to first token measured per endpoint before any move for latency.
- A failover drill that disables the primary Region and finds the standby on the request record.
- Gateway-level residency treated as absent until your gateway enforces it.