# AI gateway keys: environments, limits, rotation, off switch

URL: https://infro.io/blog/gateway-key-controls-environments
Author: INFRO Editorial (https://infro.io/blog/editorial)
Publisher: INFRO
Published: 2026-10-13
Last modified: 2026-10-13
Last reviewed: 2026-10-13 (next review due 2026-11-12)
Topic: Security and governance
Format: Guide

> Give applications a gateway key, never a provider key, and issue one per service and environment. Scope each key to the providers and networks it needs, give it a spend limit and a rate limit, rotate it with an overlap window so deploys do not break, and keep an off switch for pausing it without destroying it. When a key leaks, contain first (turn it off or rotate with no overlap), then scope the damage from request records and the audit log, then harden.

Key takeaways:
- An application should hold a key the gateway issued, scoped to what that application needs; provider keys stay sealed server-side.
- One key per service and per environment makes spend attributable and lets you revoke one integration without touching the rest.
- A lifetime spend cap bounds what a leaked key can ever cost; a rate limit bounds how fast. Use both.
- Rotate with an overlap window for routine changes, and with no overlap when a key has leaked.
- An off switch that keeps the key's secret and settings is the right tool for a pause or an investigation; revocation is for good.
- Leak response runs contain, scope, harden — in that order.

A model API key is a spending instrument. Whoever holds it can run inference that someone else pays for, at whatever rate the account's quotas allow. Most advice about these keys stops at "keep them out of source control". This guide covers what comes after: how a team that runs AI traffic through a gateway should issue, scope, limit, rotate and pause the keys its applications hold, and what to do in the first hour after one leaks. Storage basics are in [how to secure LLM API keys](https://infro.io/blog/secure-llm-api-keys); this article assumes them.

## Two kinds of key, and why the application holds only one

A gateway setup has two credentials per call. The **provider key** authenticates the gateway to OpenAI, Anthropic, Bedrock or whichever provider serves the request. The **gateway key** authenticates your application to the gateway. They should never be the same thing, and the application should only ever see the second.

The reason is blast radius. A provider key is usually broad: it can call every model the account can reach, at the account's full quota, from anywhere on the internet. Sysdig's threat research team named the abuse of stolen model credentials *LLMjacking* in May 2024: attackers who took cloud credentials from a server with an unpatched framework vulnerability tested them against ten AI services, checked whether prompt logging was on, and fronted the access with a reverse proxy so others could use it while the victim paid. Sysdig estimated the worst case for the account it observed at about $46,000 a day in model charges[1].

A gateway key can be much narrower than that: one environment, a chosen set of providers, a list of models, a set of source networks, a spend ceiling and a rate. If it leaks, the attacker gets that, not the provider account. The provider key stays sealed in one place, held by one consumer. How a gateway should seal and scope those provider credentials is covered in [BYOK credential architecture](https://infro.io/blog/byok-credential-architecture), and how to narrow the provider-side keys themselves in [least privilege for provider admin and usage keys](https://infro.io/blog/provider-admin-key-least-privilege).

[Diagram: The application holds a scoped gateway key; the provider credentials it ultimately spends against stay server-side.]

## One key per service, per environment

The default that causes the most trouble is one key shared by everything. It makes spend impossible to attribute, turns every rotation into a coordinated deploy across services, and means revoking a leaked key takes every service down with it.

Issue one key per service and per environment instead. OpenAI's production guidance recommends separate projects for development and production so testing cannot disturb the live application, restricting who can reach the production project, and setting rate and spend limits per project[2]. OWASP's secrets guidance says the same about secrets in general: different per environment, and ideally kept apart from development secrets entirely[3].

Environment scoping is only useful if it fails closed. A key that says nothing about its environment should not reach a provider connection someone deliberately reserved for production. Otherwise the easiest key to create, the one where nobody thought about environments, is also the most powerful.

[Diagram: Fail-closed matching: a key reaches connections that list its environment or serve every environment, and a key with no environment reaches only the latter.]

## Limit what a key can reach

**Providers and models.** A summarization service has no business calling an image model, and a service approved for one vendor's terms should not quietly fail over to another's. Limit each key to the providers it needs, and to the models it needs, and make sure the limit holds on every path: a direct model name, a named route, and a fallback tried after a failure. The governance side of model lists is in [model access governance](https://infro.io/blog/model-access-governance).

**Networks.** An IP allowlist turns a stolen key into a useless string unless the attacker is also inside your network. It works well for servers with stable egress addresses (a NAT gateway, fixed egress IPs, a VPC with a known range) and badly for anything whose addresses change, such as some serverless platforms or developer laptops. Two details matter. The gateway must use the client address it observes at its edge, not a header the client can set, such as `X-Forwarded-For`. And when no trusted address is available, a key with an allowlist should be refused, not let through.

## Cap what a key can spend, and how fast

OWASP's unbounded consumption risk names *denial of wallet*, running up cost against a pay-per-use AI service, and lists rate limiting and user quotas among the mitigations[4]. On a key, that means two independent limits:

- **A spend limit** bounds money. A resetting limit (daily, weekly, monthly) catches a runaway loop within one window. A **lifetime** limit that never resets is the strongest leak control there is: it bounds the total a key can ever cost, however long it goes unnoticed.
- **A rate limit** bounds speed. It cannot cap total cost on its own, but it slows abuse enough for an alert to reach someone, and it stops one misbehaving service from consuming the provider quota other services need.

OpenAI also supports hard spend limits on its side[2], and those are worth setting too: a gateway's spend figure is an estimate of what the provider will bill, not the provider's own meter.

> **Example: Sizing a key's lifetime cap**
> Illustrative. A batch job that summarizes support tickets is expected to cost about $300 a month at the provider's rate. The team gives its key a $500 monthly limit and, separately, a $2,000 lifetime cap reviewed each quarter. If the key leaks and nobody notices for a week, the most it can cost is whatever room is left under both: never more than $500 in the month, and never more than $2,000 in total.

## Rotate with an overlap window

Rotation is where most teams cause their own outages. Revoke-then-replace breaks every service still holding the old key. The pattern that avoids it is the one OpenAI describes for its own keys — create the replacement, update applications, revoke the old key once the new one is confirmed working[2] — with a defined overlap during which both keys work. OWASP's cheat sheet describes the same sequence as create, set, test, finish[3].

[Diagram: Routine rotation with an overlap: both keys work until the window ends, so deploys never race the revocation.]

Three rules make rotation safe. The new key must inherit every setting of the old one, or rotation quietly widens or narrows access. Spend limits must carry across, or rotation becomes a way to reset a cap. And a leaked key rotates with no overlap at all.

## An off switch that is not revocation

Revocation is final. That is right for a leaked key and wrong for most other reasons to stop traffic: a misbehaving deploy, a customer dispute, an investigation in progress, a load test that got out of hand. For those, a key needs an off switch that refuses every request but keeps the secret, the scope and the limits, so turning it back on restores the same key without redeploying anything.

The off switch is also the fastest containment step when you suspect a leak but have not confirmed it: it stops spend immediately and can be undone if the alarm was false. Once a leak is confirmed, follow it with rotation or revocation, because a disabled secret that someone later re-enables is still the leaked secret.

## People who manage keys

Keys are created by people, and scoping people matters as much as scoping keys. A developer working on one product should be able to create and rotate that product's keys without being able to read other projects' traffic, change organization-wide budgets or edit provider connections. Every key change (created, rotated, disabled, revoked) belongs in an append-only audit log with the actor and time, so an investigation can tell who issued the key that leaked and what else they changed.

## When a key leaks

Leaks are usually found in one of three ways: a secret scanner, a spend alert, or a provider notice. GitHub runs secret scanning automatically and for free on public repositories, across the whole Git history, and for partner-validated secrets it notifies the issuing provider, which can revoke the credential[5]. That means a provider key pushed to a public repository may be revoked by the provider before you hear about it — another reason applications should not hold provider keys.

OWASP's remediation order for an exposed secret is revoke, rotate, delete the secret from systems, code and logs, and keep the access and rotation history for responders[3]. Applied to AI keys:

[Diagram: Contain in minutes, scope in hours, harden in days.]

Scoping the damage is where per-key records pay off. With one key per service you can pull exactly that key's requests (which models, how many, from which networks), compare the gateway's estimate with the provider's own usage data, and read the audit log for anything else changed with the same access. Sysdig observed attackers checking whether prompt logging was enabled before using stolen credentials[1], so look for unusual reads and configuration changes as well as inference. The broader incident workflow is in [LLM incident response](https://infro.io/blog/llm-incident-response).

## Gateway keys in INFRO: scope, limits, rotation and the off switch

INFRO is a BYOK control plane: customers connect their own provider accounts, and applications call the gateway with an INFRO key. Provider credentials stay sealed in INFRO and are never returned to an application.

**A key's scope.** Keys start with `sk_infro_` and are created in the console under **Keys**. Each carries a label, an optional spend limit, a rate-limit tier, and optionally an environment, a project, an expiry, a model allowlist, a route allowlist, the providers it may use and its allowed networks. `GET /v1/key` returns the label, limit, usage, remaining room and next reset of whichever key made the call, which is how a deployed service confirms it picked up the right key.

**Environments.** Every organization starts with `production`, `staging` and `development` and can add its own. A provider connection or route serves one environment, several or all of them; a key names one environment or none. Matching is fail-closed in both directions: a key reaches connections that list its environment and connections set to every environment, and a key with no environment reaches only the latter.

**Providers.** By default a key may use every connected provider, including ones connected later. **Only selected** (or `allowed_providers` on `POST` / `PATCH /console/keys` and the management API) narrows it to provider ids such as `openai`. The limit holds on a model name, a provider's own model id, a route and per-request fallbacks; a route target on another provider is skipped. When nothing the key may use can serve a request, the gateway answers `503` with code `connection_out_of_scope`, naming the providers the key allows. A rotated key keeps the same providers.

**Allowed networks.** `allowed_ips` takes up to 32 IPv4 or IPv6 addresses or CIDR ranges per key. The check uses the client address the Cloudflare edge reports, never `X-Forwarded-For`, and fails closed when no trusted address is available. A refused request returns `403` with code `ip_not_allowed` before any budget, routing or provider call, and the response never lists the allowed networks. It applies to every request a gateway key authenticates (inference, `/v1/usage`, `/v1/requests`, `/v1/key` and the FOCUS export), not to management tokens or console sign-in.

**Spend and rate limits.** A key's spend limit is a ceiling on estimated provider cost that resets daily (midnight UTC), weekly (Monday, midnight UTC), monthly (on the organization's budget cycle day) or never (a lifetime cap). At the limit, requests return `402 budget_exceeded` with code `key_budget`, or `key_lifetime_limit` for a lifetime cap, while other keys keep working. Under a blocking limit, a model with no known price is refused (`unpriced_model_under_budget`) by default, because its cost cannot be counted. Rate limits are per key, from fixed tiers of 60 to 50,000 requests per minute (200 by default), independent of the plan; over the limit, the gateway returns `429 rate_limit_exceeded` with `Retry-After`.

**Rotation.** **Rotate…** in the console, `POST /console/keys/{id}/rotate`, or `POST /manage/v1/gateway_keys/{id}/rotate` issues a new secret, shown once, with every setting of the old key. The console offers an overlap of none (the old key is revoked at once), 1 hour, 24 hours (the default) or 7 days; the API's `overlap_seconds` takes any whole number of seconds up to 7 days. The old key never outlives its own expiry. Both keys share one spend limit and one lifetime cap, so rotating never creates fresh spending room, and analytics show the old key's history on its replacement.

**The off switch.** A key can be turned off from the console or with `PATCH` and `"disabled": true`. Requests with it return `403` with code `key_disabled`; its secret, scope and limits are kept, and turning it back on makes the same secret work on the next request. Whoever may revoke a key may turn it off. Revocation itself is immediate and cannot be undone: later requests return `401`.

[Diagram: What the INFRO gateway returns for each key-level refusal, from the authentication, limits and providers documentation.]

**People.** An owner or admin can limit a developer or analyst to some projects. A member limited this way manages the gateway keys in those projects, and every key they create or move belongs to one of them; budgets, provider and routing settings, policies, members and the audit log are closed to them, and the gateway enforces this on every request.

**Audit.** The audit log records `api_key.created`, `api_key.updated`, `api_key.rotated`, `api_key.disabled`, `api_key.enabled` and `api_key.revoked`, plus `member.project_access_changed` and environment changes, each with the actor and time.

**Plans.** Environments, provider limits, allowed networks, rotation, per-key spend and rate limits and the audit log are on every plan. The [authentication reference](https://infro.io/docs/authentication) and [rate and spend limits](https://infro.io/docs/api/limits) have the full request and response shapes.

## A key hygiene checklist

- [ ] Applications hold gateway keys only; provider keys stay sealed server-side.
- [ ] One key per service and per environment, each with a label someone recognizes.
- [ ] Each key limited to the providers and models it needs.
- [ ] Production keys restricted to your servers' egress networks.
- [ ] A resetting spend limit and a lifetime cap on every key, plus a rate tier sized to the service.
- [ ] Rotation rehearsed with an overlap window; leak rotation with none.
- [ ] Someone knows where the off switch is, and the audit log is reviewed after every key change.

## Sources

1. LLMjacking: Stolen Cloud Credentials Used in New AI Attack — Sysdig Threat Research Team. https://www.sysdig.com/blog/llmjacking-stolen-cloud-credentials-used-in-new-ai-attack (accessed 2026-10-13)
2. Production best practices — OpenAI. https://developers.openai.com/api/docs/guides/production-best-practices (accessed 2026-10-13)
3. Secrets Management Cheat Sheet — OWASP Cheat Sheet Series. https://cheatsheetseries.owasp.org/cheatsheets/Secrets_Management_Cheat_Sheet.html (accessed 2026-10-13)
4. LLM10:2025 Unbounded Consumption — OWASP GenAI Security Project. https://genai.owasp.org/llmrisk/llm102025-unbounded-consumption/ (accessed 2026-10-13)
5. About secret scanning — GitHub Docs. https://docs.github.com/en/code-security/secret-scanning/introduction/about-secret-scanning (accessed 2026-10-13)