# Zero trust for AI API access: a reference architecture

URL: https://infro.io/blog/zero-trust-ai-api-access
Author: INFRO Editorial (https://infro.io/blog/editorial)
Publisher: INFRO
Published: 2026-10-21
Last modified: 2026-10-21
Last reviewed: 2026-10-21 (next review due 2026-11-20)
Topic: Security and governance
Format: Guide

> Zero trust for AI API access means no caller is trusted because of where it runs or which shared key it holds: every service, agent and developer authenticates with its own credential, each request is checked against a policy that limits models, providers, environments, networks and spend before the provider is called, decisions and usage are logged, and provider credentials stay in one enforcement point instead of in applications. Provider-side scope and spend limits cover what the enforcement point cannot see.

Key takeaways:
- A shared provider key grants every model to every caller with no per-request decision; zero trust starts by giving each service, agent and developer its own credential.
- Put one policy enforcement point between callers and providers, and keep provider credentials there rather than in applications.
- Least privilege for AI calls has five axes: models, providers, environments, networks and spend.
- Evaluate continuously: budgets and content checks on every request, revocation that takes effect at once, and alerts on usage that departs from normal.
- An enforcement point governs only the traffic that passes through it; provider-side scope, spend limits and usage reports cover the rest.

**What does zero trust mean for AI API access?** No caller is trusted because of where it runs or which shared key it holds. Every service, agent and developer that calls a model authenticates with its own credential; every request is checked against policy — which models, which providers, which environment, which networks, how much spend — before the provider is called; decisions and usage are logged; and provider credentials stay in one enforcement point rather than in applications. Provider-side controls cover the traffic that enforcement point cannot see.

The common starting point is the opposite: one provider key in an environment variable, shared by every service and several laptops, able to call every model the account can, with no decision made per request. This guide maps the zero trust tenets of NIST SP 800-207 onto AI API calls, lays out a reference architecture in a table, and names its limits.

## What NIST SP 800-207 says, briefly

NIST SP 800-207, *Zero Trust Architecture*, was published in August 2020. Its abstract describes zero trust as shifting defenses from network perimeters to users, assets and resources, and states that no implicit trust is granted to assets or user accounts based solely on their network location or ownership[1]. Section 2.1 sets out seven tenets; section 3.1 defines the logical components that enforce them, chiefly a policy engine that decides, a policy administrator that executes the decision, and a policy enforcement point that enables, monitors and terminates connections between a subject and a resource[2].

CISA's Zero Trust Maturity Model builds on the same ideas with maturity stages from traditional through initial and advanced to optimal[3]. Neither document mentions AI APIs. The mapping below is an interpretation, and it is ours.

## The seven tenets, applied to AI API calls

| NIST tenet (Section 2.1) | What it means for AI API calls |
| --- | --- |
| 1. All data sources and computing services are resources | Each provider account, model and MCP tool is a resource with its own owner and policy, not a utility anyone can reach |
| 2. All communication is secured regardless of network location | A call from inside your VPC gets no more trust than one from outside; network location is at most one extra condition |
| 3. Access is granted per session, with least privilege | Each caller gets only the models, providers and environment it needs; access to one does not grant another |
| 4. Access is determined by dynamic policy | Decisions use who the caller is, which project and environment it belongs to, its network, its spend so far, and the request itself |
| 5. The enterprise monitors the integrity and posture of assets | Know which services, agents and tools call models, and treat an unmanaged or unknown caller differently |
| 6. Authentication and authorization are dynamic and enforced before access | Every request is checked before the provider is called; revocation takes effect at once |
| 7. Collect information to improve posture | Log calls, decisions and spend, and feed what you learn back into policy |

The tenets' own text supports two details. Tenet 3 says access should be granted with the least privileges needed and that authorization to one resource does not grant access to another. Tenet 4 lists observed behaviour and measured deviations from usage patterns among the attributes a policy may use[2] — which, for AI calls, is where spend comes in.

## A reference architecture

This is the architecture in components, the zero trust role each plays, and what it does for an AI call.

| Component | Zero trust role | What it does |
| --- | --- | --- |
| **Caller** (service, agent, developer tool) | Subject | Holds its own credential, never a provider key |
| **Caller credential** (a per-caller key or token) | Identity | Names one caller, one project and one environment; can expire, be rotated and be revoked on its own |
| **Identity provider** (SSO for people) | Identity source | Authenticates the humans who configure policy and issue credentials, with multi-factor authentication |
| **Policy store** (allowlists, budgets, routes, content rules) | Policy | Declares what each caller may do, per organization, project, member and key |
| **Gateway** (in the request path) | Policy engine and enforcement point | Authenticates the caller, evaluates policy, refuses or forwards the request, records the decision |
| **Credential vault** (inside the gateway) | Protected resource credentials | Holds provider credentials encrypted, never returns them, uses them only to forward allowed requests |
| **Provider account** (OpenAI, Anthropic, Google, a cloud) | Resource | Serves the model; its own scope and spend limits are the last layer |
| **Telemetry** (request log, usage, alerts) | Monitoring and analytics | Records who called what, the decision, latency and estimated cost; raises alerts on departures from normal |
| **Provider usage data** | External data source | Shows usage on each provider key, including any that did not go through the gateway |

The flow for one request, in order:

1. The caller sends a request with its own credential to the gateway.
2. The gateway authenticates the credential and resolves its project, environment and owner.
3. Network conditions are checked: a request from outside the credential's allowed ranges is refused.
4. Policy is evaluated: is this model allowed for this organization, project and key; is this provider allowed; is this environment's connection in scope; does the budget at every level have room; do content rules pass on the prompt.
5. If any check fails, the request is refused with a specific error and the provider is never called.
6. If all pass, the gateway uses a provider credential from its vault to forward the request, applies content rules to the response, and records the outcome and estimated cost.
7. Telemetry compares usage with the usual pattern and raises an alert when it departs from it.

## The five axes of least privilege

For AI calls, least privilege is not one setting. It has five axes, and a credential should be narrow on each.

- **Models.** An allowlist per organization, project and key, all of which must pass. A support bot does not need an image model or the most expensive reasoning tier. [Model access governance](https://infro.io/blog/model-access-governance) covers the policy side.
- **Providers.** A caller that should use only one provider's models should not be able to reach another's, even when the model name is ambiguous.
- **Environments.** A development credential must never reach a production provider connection. The default must narrow: a credential that names no environment should reach less, not everything. [Gateway key controls and environments](https://infro.io/blog/gateway-key-controls-environments) explains why.
- **Networks.** An IP allowlist on a server-side credential means a copy used from elsewhere is refused. Under tenet 2 it is a condition added to identity, never a substitute for it.
- **Spend.** A hard cap per credential, project and organization, checked before the provider is called. OWASP's entry on unbounded consumption recommends rate limiting and user quotas per source, resource allocation management, and logging and monitoring to detect unusual patterns[4].

Providers offer some of these axes on their own keys, and they belong in the design. OpenAI recommends setting an expiration date on project keys and supports organization-level limits on key lifetime[5]. Anthropic keys are tied to the workspace they were created in, and a workspace can carry its own spend limit below the organization's[6]. Google recommends API restrictions to reduce the impact of a compromised key, and short-lived service account credentials over API keys where possible[7].

## Continuous evaluation

Tenet 6 describes a constant cycle of obtaining access, assessing threats, adapting and re-evaluating trust[2]. For AI calls that cycle has four parts:

- **Per-request checks.** Budgets and allowlists evaluated on every call, not at key issue time. A key that was fine yesterday is refused today when its project's budget is spent.
- **Content checks.** Pattern or classifier checks on prompts and responses — for credentials, card numbers, personal data — that block, redact or flag. Detection can miss things; treat it as one layer. [LLM guardrails at the gateway](https://infro.io/blog/llm-guardrails-at-the-gateway) goes deeper.
- **Behavioural alerts.** Spend or volume that departs from a scope's usual day, or a forecast that crosses a limit, raised to a person who can act. [LLM spend anomalies](https://infro.io/blog/llm-spend-anomalies) explains the signals.
- **Revocation that takes effect at once.** OpenAI says a revoked key stops working within a few seconds[8]. A gateway credential should do the same, including at any cache in front of it.

## Logging what the policy decided

Tenet 7 asks the enterprise to collect information and use it to improve policy. For AI calls, log at least the caller, project, environment, model, provider, decision, error code, token counts and estimated cost for every request, including refused ones. Whether to record prompt and response content is a separate decision with its own risks; [an LLM prompt logging policy](https://infro.io/blog/llm-prompt-logging-policy) helps make it. Keep configuration changes — who changed an allowlist, a budget, a route — in an audit log that the people being audited cannot edit.

> **Example: One agent, one credential**
> A team runs a nightly agent that summarizes support tickets. Instead of the team's shared OpenAI key, the agent gets its own gateway credential: project `support-summaries`, environment `production`, models limited to one small model, provider limited to the team's OpenAI connection, an allowlist of the job runner's egress range, a daily cap of $40 and a 90-day expiry. One night a bug loops the agent. Its requests pass every check until the cap is reached, then the gateway refuses them before calling OpenAI, and the spend alert names the credential. Nothing else the team runs is affected, and revoking the credential stops the agent without touching anyone else.

## Where the architecture stops

An enforcement point governs the traffic that passes through it. Three gaps remain, and naming them is part of the design.

- **Provider keys outside the vault.** If a provider key also exists in an application's environment, on a laptop, or in a third-party tool, traffic on it never meets your policy. Keep provider keys in one place, and compare provider usage by key with what the gateway served to find traffic that bypassed it; [detecting gateway bypass](https://infro.io/blog/detect-gateway-bypass-provider-usage) shows how.
- **A stolen provider key.** A key taken from the vault, or from anywhere else, can be used directly at the provider. Only provider-side scope, spend limits and revocation apply. [What happens when a BYOK platform is breached](https://infro.io/blog/byok-platform-breach-blast-radius) models this.
- **The enforcement point itself.** It is now a high-value system. It needs strong sign-in, roles, an audit log and careful release practices of its own.

## How INFRO maps to this architecture

INFRO is a gateway that can play the policy engine and enforcement point role for calls to the provider accounts you connect. What it enforces today:

- **Identity per caller.** INFRO gateway keys (`sk_infro_`, stored as a SHA-256 digest), each in one project and exactly one [environment](https://infro.io/docs/authentication), never reaching a connection outside it. People sign in with two-factor authentication or passkeys, or with SAML or OIDC SSO and SCIM on the Business plan ([SSO](https://infro.io/docs/org/sso)); roles and [project access](https://infro.io/docs/org/members) limit what each member can see and change.
- **Least privilege.** [Model allowlists](https://infro.io/docs/org/model-allowlists) at organization, project and key level (`403 model_not_allowed`); keys limited to chosen providers; IP allowlists that refuse a request from outside them with `403 ip_not_allowed` before any provider call; expiry and rotation with an overlap of none, 1 hour, 24 hours or 7 days.
- **Continuous evaluation.** Hard [budgets](https://infro.io/docs/org/spend-controls) per organization, project, member and key, checked before the provider is called (`402`); [guardrails](https://infro.io/docs/features/guardrails) on prompts and responses on the Business plan; [spend alerts](https://infro.io/docs/observability/alerts) when a scope's day departs from its usual day or a forecast crosses a limit, by email, Slack, Teams, PagerDuty or webhook.
- **Logging.** Every request in the request log with its decision and estimated cost ([traces](https://infro.io/docs/observability/traces)), prompt and response recording off unless you turn it on, and configuration changes in the [audit log](https://infro.io/docs/org/audit-log).
- **The vault.** Provider credentials encrypted with AES-256-GCM under a key held outside the database, never returned after saving.

The same limits apply to INFRO as to any enforcement point: its controls govern traffic that goes through it. A provider key used directly at the provider, including one stolen from INFRO, is limited only by the provider's own scope, spend limits and revocation, so set those too.

## Summary

Zero trust for AI API access replaces one shared provider key with an identity per caller, a policy decision on every request, and a log of each decision. Put a single enforcement point between callers and providers, keep provider credentials there, narrow every credential on models, providers, environments, networks and spend, and evaluate on every call. Then cover what the enforcement point cannot see with provider-side scope, spend limits and usage reports.

## Questions

### What is zero trust for AI APIs?

Applying zero trust principles to calls to model providers: no caller is trusted because it runs inside your network or holds a shared key. Each service, agent and developer has its own credential, each request is checked against policy for models, providers, environment, network and spend before the provider is called, and every decision is logged.

### Does NIST SP 800-207 cover AI or LLM APIs?

No. SP 800-207, published in August 2020, defines zero trust tenets and logical components for enterprise resources in general. Applying them to AI API calls is an interpretation; this guide makes that mapping explicitly.

### Is an IP allowlist zero trust?

Not on its own. NIST's second tenet says network location alone does not imply trust. An IP allowlist is a useful extra condition on a credential — a stolen key used from elsewhere is refused — but identity and per-request policy come first.

### How do zero trust principles apply to AI agents?

Give each agent its own credential rather than a developer's or a shared key, limit it to the models, providers and environment it needs, cap its spend per run or per key, and log its calls separately so its usage can be evaluated and revoked on its own.

## Sources

1. SP 800-207: Zero Trust Architecture — NIST. https://csrc.nist.gov/pubs/sp/800/207/final (accessed 2026-10-21)
2. NIST SP 800-207, Zero Trust Architecture (full text) — NIST. https://nvlpubs.nist.gov/nistpubs/SpecialPublications/NIST.SP.800-207.pdf (accessed 2026-10-21)
3. Zero Trust Maturity Model — CISA. https://www.cisa.gov/zero-trust-maturity-model (accessed 2026-10-21)
4. LLM10:2025 Unbounded Consumption — OWASP GenAI Security Project. https://genai.owasp.org/llmrisk/llm102025-unbounded-consumption/ (accessed 2026-10-21)
5. Production best practices — OpenAI. https://developers.openai.com/api/docs/guides/production-best-practices (accessed 2026-10-21)
6. Creating and managing Workspaces in the Claude Console — Anthropic. https://support.claude.com/en/articles/9796807-creating-and-managing-workspaces-in-the-claude-console (accessed 2026-10-21)
7. Best practices for managing API keys — Google Cloud. https://docs.cloud.google.com/docs/authentication/api-keys-best-practices (accessed 2026-10-21)
8. API reference: Authentication — OpenAI. https://developers.openai.com/api/reference/overview (accessed 2026-10-21)