Most architecture diagrams draw an LLM gateway as a thin box between the application and the model provider. Security reviews that accept that drawing miss the point: the box holds every provider credential the organization uses, decides which models may be called and how much may be spent, and — if content capture is on — keeps copies of prompts and completions. It is an asset worth attacking in its own right.
This analysis threat-models the gateway itself, then maps the OWASP Top 10 for LLM Applications 2025 onto it, to separate the risks a gateway reduces from the ones it cannot touch. The distinction matters for procurement as much as for design: a vendor who claims its gateway "addresses the OWASP Top 10" is claiming something no gateway can do.
What the gateway holds
Start with assets, because threats only make sense against something of value:
- Provider credentials. Keys and tokens that spend money on the organization's own provider accounts. In a bring-your-own-key design they belong to the customer, not the gateway operator.
- Gateway keys and configuration tokens. Inbound credentials that decide who can send traffic and who can change policy.
- Policy. Model allowlists, routes, budgets, guardrail rules, environment scopes. Weakening policy quietly is a more patient attack than stealing a key.
- Request records. Metadata about every call — model, provider, tokens, latency, estimated cost, the caller's own tags — which is operational evidence and can contain personal data in metadata fields.
- Captured content. Prompts and completions, if capture is enabled. The most sensitive asset, and the one a good default does not create at all.
- Spend authority. The ability to cause provider charges. OWASP lists unbounded consumption as its own risk, including attacks that exploit pay-per-use pricing to create unsustainable cost1.
Four trust boundaries
Each asset is exposed across one or more boundaries. Drawing them explicitly is what turns a list of worries into a set of controls with owners.
Boundary 1
Your applications → gatewayAuthenticated by gateway keys. Threats: a leaked key, a key used from the wrong network, a key used for the wrong model or project.Asset
The gatewayHolds sealed provider credentials, policy, budgets, request records and, if enabled, captured content.Boundary 2
Gateway → provider accountsYour credentials on your accounts. Threats: a credential disclosed, a request sent to a provider or region the data should not reach.Boundary 3
People → consoles and management APISessions, two-factor, SSO, management tokens. Threats: account takeover, a policy quietly weakened.Boundary 4
Gateway → your telemetry and exportsOpenTelemetry spans, CSV and FOCUS exports. Threats: content or identifiers leaking into a system with wider access.
Applications to gateway. The inbound boundary is authenticated by gateway keys. Threats are the familiar ones: a key committed to a repository, shipped in a mobile app, or used by a service that should not have it. Controls are scoping (project, environment, model, network), spend limits per key, and fast revocation.
Gateway to providers. The outbound boundary carries the customer's credentials and the full request content. Threats are disclosure of a credential from storage, a request sent to a provider or deployment the data should not reach, and silent fallback to a target nobody approved. Controls are authenticated encryption bound to the owning row, no read path for secrets, and model and route policy that applies to fallbacks as well as primaries. Provider-side retention sits on the far side of this boundary and outside the gateway's control; OpenAI, for example, documents that abuse-monitoring logs may be retained for up to 30 days unless an organization has approved data controls2.
People to consoles and configuration APIs. Threats are account takeover, a departing employee who keeps access, and a policy change nobody reviews. Controls are phishing-resistant second factors, single sign-on with directory deprovisioning, configuration tokens that expire and follow their creator's role, and an audit log the application itself cannot rewrite.
Gateway to telemetry and exports. The least-watched boundary. Spans sent to an observability backend and CSV exports sent to a finance team both leave the gateway's access model. Threats are content or identifiers leaking into a system with far wider read access. Controls are field-by-field allowlists for telemetry and exports that exclude content by construction.
Threats that follow from the gateway's position
A few threats are specific to sitting in the middle, and deserve their own line in the model:
- Cross-tenant credential confusion. In a multi-tenant gateway, a bug or a write to the credentials table that places one tenant's sealed credential in another's row. Binding each ciphertext to its row makes this fail closed instead of authenticating as the wrong account.
- Policy bypass through indirection. A route or fallback list that reaches a model the allowlist forbids. Policy must apply to what is actually called, not to the name the client sent.
- Header trust. Network allowlists evaluated against
X-Forwarded-Foror another client-controlled header can be spoofed; they must use the address the edge observed. - Retry amplification of cost. Automatic retries and fallbacks multiply the requests a single abusive call produces. Spend controls have to count every attempt, not only the one that succeeded.
- Error-message disclosure. Upstream provider errors and internal identifiers passed through verbatim can reveal account structure. The gateway should translate them into its own documented error surface.
Holder of a leaked key
Gateway
Provider account
- Holder of a leaked key to Gateway: Request from an unknown network
- Gateway to Holder of a leaked key: 403 ip_not_allowed, before any budget or provider call
- Holder of a leaked key to Gateway: Request from an allowed network, for an unapproved model
- Gateway to Holder of a leaked key: 403 model_not_allowed
- Holder of a leaked key to Gateway: Approved model, repeated until the key's limit
- Gateway to Provider account: Served while the key's spend limit has room
- Gateway to Holder of a leaked key: 402 key_budget once the limit is reached
Mapping the OWASP Top 10 for LLM Applications
OWASP's 2025 list names ten risks, from prompt injection (LLM01) to unbounded consumption (LLM10)3. They are written for LLM applications as a whole, so the honest question for a gateway is which of them it can reduce at its boundary.
| OWASP 2025 | Can a gateway help? | What it can do at the boundary |
|---|---|---|
| LLM01 Prompt injection | ◐Partly | Limit the blast radius: model allowlists, spend caps, pattern redaction of secrets. It cannot tell an injected instruction from a real one. |
| LLM02 Sensitive information disclosure | ●Yes | Content capture off by default, payload access control, redaction before the provider |
| LLM03 Supply chain | ◐Partly | Approve which models and providers may be called, and record which one served |
| LLM04 Data and model poisoning | ○No | Happens in training and fine-tuning data, upstream of any request |
| LLM05 Improper output handling | ○No | The application must validate and encode output for its destination |
| LLM06 Excessive agency | ◐Partly | Narrow models and spend per key; tool permissions live in the application |
| LLM07 System prompt leakage | ◐Partly | Output checks for credential formats; the fix is keeping secrets out of prompts |
| LLM08 Vector and embedding weaknesses | ○No | Lives in the retrieval store and its access control |
| LLM09 Misinformation | ○No | An evaluation and product-design problem |
| LLM10 Unbounded consumption | ●Yes | Rate limits, per-key and per-project budgets, end-customer caps |
Two risks are squarely in the gateway's reach. Sensitive information disclosure (LLM02) is reduced by never capturing content by default, restricting who can read it when captured, and redacting known formats before a request leaves. Unbounded consumption (LLM10) is reduced by rate limits, budgets per key and project, and caps per end customer — the operational controls OWASP itself lists, such as rate limiting, user quotas and resource allocation limits1.
Four are partial. For prompt injection (LLM01), OWASP states that it is unclear whether fool-proof prevention methods exist4; a gateway cannot distinguish an injected instruction from a genuine one, but it can bound what a manipulated call reaches — which models, what spend, whether a credential-shaped string leaves. Excessive agency (LLM06) is mostly an application concern, because tool permissions live where tools execute. System prompt leakage (LLM07) is best addressed, per OWASP, by not treating the system prompt as a secret or as a security control at all5; output checks for credential formats help only with the formats they know. Supply chain (LLM03) benefits from an approved-model list and a record of which provider served each call.
Four are out of reach. Poisoning (LLM04) happens in training data. Vector and embedding weaknesses (LLM08) live in the retrieval store. Improper output handling (LLM05) must be solved by the application that renders or executes output. Misinformation (LLM09) is an evaluation problem.
Organizing the model with NIST AI RMF
The NIST AI Risk Management Framework, released in January 2023 for voluntary use, organizes AI risk work into four functions: Govern, Map, Measure and Manage6. A gateway threat model fits into it cleanly. The asset and boundary inventory above is Map work. Request records, refusal counts and audit events are Measure inputs. Budgets, allowlists and revocation are Manage controls. And who may change policy, and how that is reviewed, is Govern. Using the framework's vocabulary makes the gateway's controls legible to a risk team that already uses it for the rest of the AI program.
INFRO against this threat model
INFRO is a gateway, so it is subject to this model like any other. The controls it has today, from its documentation and Trust Center:
| Boundary | Control in INFRO | Where documented |
|---|---|---|
| Applications → gateway | Keys scoped by project, environment, model allowlist and up to 32 allowed networks; per-key spend limits; rotation with overlap | Authentication |
| Gateway → providers | AES-256-GCM sealing bound to organization and connection; no reveal; model policy applied to route targets and fallbacks | Connect a provider, Model allowlists |
| People → consoles | Two-factor and passkeys on every plan; SAML/OIDC SSO and SCIM from the Business plan; expiring, role-bound management tokens; append-only audit log | SSO & SCIM, Management tokens, Audit log |
| Gateway → exports | Content capture off by default; CSV export never includes content; OpenTelemetry attributes built from an allowlist | Privacy & data, Request traces |
The gaps belong in the model too, and INFRO publishes them. Guardrails are pattern-based and do not detect prompt injection, jailbreaks or toxicity. There is no control that refuses a single unusually expensive request; budgets cap totals. Rate limits are counted per location rather than by one global counter, so a key can briefly exceed its nominal limit. There is no independent penetration test, and no SOC 2 or ISO 27001 certification is held. A reviewer should weigh those as stated rather than infer their absence from silence.
Using the model
A threat model is useful when it produces decisions. For each boundary, write down the credential that crosses it, the control that limits it, the evidence that the control works, and the person who owns it. For each OWASP risk the gateway cannot reach, name the component that must carry it instead — the retrieval store, the output renderer, the evaluation suite — so that buying a gateway does not quietly leave those risks unowned.