How to block direct AI provider calls with egress rules

Why detection is not enough

If you already compare provider usage with gateway records, you know the gap exists: some code calls a provider directly. Detection reports it after the spend and after the prompt has left. Prevention makes the direct call fail at the moment it is attempted.

No single control does that reliably. A network rule is bypassed by a laptop on home wifi. A key restriction is bypassed by a second key someone created. An identity rule is bypassed by a workload that still holds a static secret. So the approach below stacks four layers, and each layer gets a test that must fail if the layer is open. The risk it addresses is the same one OWASP's LLM Top 10 work and NIST's generative AI profile frame at program level¹²; this article is the mechanical part.

The checklist assumes a gateway already carries your production traffic. If it does not, start with a canary migration first, because default-deny egress on a half-migrated estate is an outage.

Layer 1: default-deny egress with one AI destination

The base layer is a network rule that denies outbound traffic to every AI provider hostname from application workloads, and allows the gateway.

• Inventory every provider hostname your organization uses, including regional and alternate hosts, plus any aggregator or proxy hosts you do not intend to allow.
• Enforce by hostname through an egress proxy or DNS-aware firewall, not by provider IP ranges, which move and share infrastructure with unrelated services.
• Make the default for workload subnets deny, with the gateway hostname as an explicit allow. A deny list of providers fails open the day a new provider appears.
• Block direct DNS-over-HTTPS and any outbound DNS that bypasses your resolver, or hostname rules can be sidestepped.
• Block outbound traffic to arbitrary HTTPS proxies and tunnel services on the same default-deny basis.
• Put the rule in code (Terraform, policy repo) and require review for any exception, so each exception has an owner and an expiry.

Log denials. A denied call to a provider hostname is exactly the signal your detection work was looking for, now with the source workload attached and no spend behind it.

Layer 2: cloud-hosted models need their own boundary

If your models run on a hyperscaler, the provider endpoint is reachable from inside your cloud account, and an egress proxy never sees it. This path needs its own rule.

On AWS, Bedrock supports interface endpoints for several API groups, including bedrock-runtime. The default endpoint policy allows full access through the endpoint, and you narrow it with a custom endpoint policy naming principals, actions and resources. With private DNS enabled, calls to the standard regional name route through the endpoint with no code change³. That is useful for the gateway's own network and a hole for everyone else: if workloads can reach the endpoint, they can call the model directly. Attach a policy that allows only the principals your gateway path uses.

On Azure, Microsoft's guidance for Foundry Tools is to first deny traffic from all networks by default, then grant specific virtual networks; the rules apply to REST and WebSocket alike, and callers still need valid authorization, either Microsoft Entra credentials or an API key⁴. Each resource supports up to 100 virtual network rules⁴.

On Google Cloud, VPC Service Controls defines perimeters so that clients inside a perimeter cannot reach unauthorized resources outside it, and it offers a dry run mode for perimeters so you can see what a rule would block before enforcing it⁵.

• For each cloud-hosted model account, set the default network action to deny and name the allowed network or principal.
• Attach a restrictive endpoint policy or perimeter rather than relying on the permissive default.
• Run new perimeters in dry run first where the platform offers it, and read the violations before enforcing.

Layer 3: no provider keys on workloads, gateway keys tied to your addresses

Network rules fail when a credential travels. The credential layer has two halves.

Remove provider keys from applications. With a BYOK gateway, provider credentials are saved once, encrypted and scoped to your organization, and applications hold only gateway keys⁶. That makes the old provider key a thing to revoke, not to hide. After cutover, rotate or delete every provider key that application code, CI, laptops or .env files ever held. A direct call that gets past the network then fails with a 401 from the provider.

Tie each gateway key to where it may be used. INFRO gateway keys can carry an allowed-networks list of up to 32 IPv4 or IPv6 addresses or ranges, matched against the client address the edge reports and never against X-Forwarded-For; a call from elsewhere is refused with 403 ip_not_allowed⁷⁶. Set it to your egress NAT addresses and a copied key is useless off your network.

• One gateway key per service or environment, with a label, so spend and refusals are attributable.
• Allowed networks set to your NAT egress ranges on every production key.
• Providers and models limited per key or project; anything else returns a policy error, not a line item.
• A spend limit and a requests-per-minute ceiling per key; both are enforced per key, so a key at its spend limit returns 402 budget_exceeded while other keys keep working⁶⁸.
• An expiry on keys for contractors, demos and migrations.
• Rotation with an overlap window planned, so the old key can keep working up to seven days while you deploy the new one⁷.

Keep a spending limit at each provider too. Gateway caps count an estimate of provider cost, not your provider's invoice⁷.

Layer 4: workload identity decides who may ask

Static secrets are what you are trying to remove, so identity should do the gatekeeping where the platform supports it.

• Run each workload under its own service identity (pod identity, instance role, managed identity), never a shared account.
• Fetch the gateway key from your secrets manager using that identity at start-up, so the key never lives in an image, repository or environment file.
• For cloud-hosted models, grant the invoke permission only to the gateway's identity and deny it to application roles. On AWS this is the principal list in the endpoint policy above³.
• Separate environments: a key belongs to one environment and cannot reach a connection outside it, which stops a staging workload from borrowing production credentials⁷.

Be clear about the limit here. INFRO authenticates requests with sk_infro_ bearer keys⁶; workload identity is how you control who can read that key and who can call cloud endpoints, not a replacement for the gateway key itself. For more depth, see zero trust for AI API access.

Layer 5: prove each layer fails closed

Every control above gets a test that is expected to fail. Run these from a workload in the protected network, in CI after each policy change, and on a schedule.

#!/usr/bin/env bash
# Expect FAIL (non-200 or connect error) for every direct call.
for host in api.openai.com api.anthropic.com generativelanguage.googleapis.com; do
  code=$(curl -s -o /dev/null -m 5 -w '%{http_code}' "https://$host/" || true)
  echo "$host -> ${code:-blocked}   (want: blocked/000)"
done

# Expect 200 through the gateway with the workload's own key.
curl -s -o /dev/null -w 'gateway -> %{http_code}\n' \
  https://api.infro.io/v1/models \
  -H "Authorization: Bearer $INFRO_API_KEY"

Then test the other layers separately, because passing one can mask failing another.

1. Credential layer. Use a revoked provider key against the provider from a network that is allowed out. It must return an authentication error.
2. Address layer. Call the gateway with a production key from outside its allowed networks. Expect 403 ip_not_allowed and no provider call.
3. Model layer. Request a model outside the key's approved list. Expect a policy error.
4. Off switch. Disable a test key and confirm every request returns 403 key_disabled⁶.
5. Cloud endpoint. Call the model endpoint from an application role. It must be denied by the endpoint policy or perimeter.
6. Alert path. Confirm a denied direct call appears in your egress log and reaches the on-call channel.

The holes that survive

Teams that pass the tests above still leak through these.

• SDK telemetry and update checks. Some SDKs and agents contact vendor hosts for telemetry, version checks or feature flags. Default-deny will break or log them. Decide per host: allow it explicitly with an owner, or disable the feature in configuration. Test with the SDK pinned to the version you deploy, since behaviour can change on upgrade.
• Browsers. Egress rules on your servers say nothing about end-user browsers. A provider key shipped to a front end is a bypass by design. The fix is a backend call through the gateway, with a per-user cap; see AI API keys in frontend and mobile apps.
• Laptops and split tunnels. Corporate egress rules do not follow a developer off VPN. Treat laptop keys as separately issued, expiring gateway keys with their own limits, not as a reason to loosen the server rules.
• CI runners and serverless. These often have a different egress path than your application subnets. Include them in the inventory and the tests.
• Coding agents. Claude Code and Codex speak their provider's API, so they bypass any proxy that only watches Chat Completions. Point them at the gateway: Claude Code takes a base URL and token through environment variables or the env block of managed settings, and Codex takes a model provider in ~/.codex/config.toml⁹. Push those settings through managed configuration, give each developer or agent its own key with a limit, and let the egress rule deny the provider hosts as the backstop⁹. More on the keys side in AI coding agents and secrets.
• Aggregator and proxy hosts. A third-party router is a provider for this purpose. If it is not on the allowlist it should be denied, and it should appear in the inventory.

Where a gateway helps, and what to keep watching

A gateway gives the allowlist one destination to point at, and per-key controls to enforce on the far side of it: allowed networks, provider and model limits, environment scoping, spend limits and an audit log of key and policy changes. INFRO's connection storage keeps provider credentials out of applications, and its key controls handle the address and provider limits described above⁷.

Keep detection running. Cost reconciliation with provider usage access reports usage on a connected provider key that never went through the gateway, and without that access it says "not checked", never "no bypass"⁷. Also decide what happens when the gateway itself is down: a fail-open exception reopens the bypass, so read gateway down: fail open or fail closed before you write that exception.

Rollout order

1. Inventory hosts, keys and egress paths, including CI, serverless and laptops.
2. Move traffic to the gateway and confirm with request logs.
3. Issue per-service gateway keys with allowed networks and limits.
4. Switch egress to log-only deny, and read the denials for a full business cycle.
5. Rotate or delete provider keys held outside the gateway.
6. Enforce default deny, add cloud endpoint policies, and turn the bypass tests into a scheduled job.