"Guardrails" covers a lot of different things: a regular expression that catches card numbers, a provider's harm classifier, a list of approved models, a schema validator on tool arguments, a human approval step. They have one thing in common. Each is a check that runs at a known point in a request and has a decision attached to it. This guide is about the checks that sit on the path of the model call itself, on the prompt before it leaves and on the response before it returns: what they can catch, where each one belongs, what it costs, and what should happen when it cannot run.
What a guardrail is, and what it is not
A useful working definition has three parts: what is inspected (which fields, which endpoints, which stage), what is detected (a pattern, a list, a classifier score) and what happens (refuse, rewrite, or record and continue). If you cannot write those three down for a guardrail, you cannot test it, and you cannot explain an incident involving it.
What a guardrail is not is a safety guarantee. OWASP's entry on prompt injection, the first risk in its 2025 Top 10 for LLM Applications1, says it is unclear whether fool-proof prevention exists, and treats input and output filtering as one mitigation among several, beside privilege control, human approval for high-risk actions, segregating untrusted content and adversarial testing2. A filter reduces what gets through. The controls that limit damage when something does get through are authorization and blast radius, and those are covered in defense in depth for prompt injection. This article covers the filter layer and keeps that boundary in view.
Input checks
Input checks run on the prompt before the provider sees it. Four families cover most needs.
Secrets. API keys, tokens and private keys pasted into prompts are common and dangerous: the prompt goes to a third party, may be logged there, and may be logged by you. Secret formats are a good fit for deterministic detection because most vendors use recognizable prefixes (sk-, AKIA, ghp_, xoxb-) or structures (JWTs, PEM blocks). The gap is secrets with no prefix, such as an AWS secret access key or a database password, which no pattern can recognize reliably.
Personal data. OWASP's sensitive information disclosure entry recommends input validation that detects and filters sensitive data before it reaches the model, alongside sanitization, tokenization or redaction3. Pattern detection works for data with a format and a checksum: card numbers pass a Luhn check, IBANs a mod-97 check, emails and phone numbers have recognizable shapes. It does not work for names, street addresses, dates of birth or anything written out in words. NIST's generative AI profile adds that what counts as sensitive data can be highly contextual4. A pattern detector finds formats. It does not know what is sensitive to you.
Prompt-injection and jailbreak signals. These are about meaning, not format. Text that tells the assistant to ignore its instructions can be phrased in endless ways, encoded or split across a retrieved document. Detecting it takes a classifier: a provider's (Azure's Prompt Shields classify user prompt attacks and indirect attacks embedded in documents5), an open model you host, or a decision model answering a yes/no question. All of them return a score or a label, and you choose the threshold.
Topic and policy. "Is this request within what this application is for?" is also a classifier question. It is the most application-specific of the four, so the instructions to the classifier have to describe your scope, and the threshold has to be tuned on your own traffic.
Output checks
Output checks run on what the model returned. They catch secrets or PII the model reproduced from its context or training, harmful content a provider's own filter allowed, and text that matches a term you never want to show. Improper output handling, fifth in OWASP's list1, is the reason structured output also needs validation in code before anything acts on it. That is an application concern rather than a filter, covered in the defense-in-depth guide.
Two properties of output checks catch people out. First, the provider has already generated the response and will bill for it; blocking it afterwards changes what the user sees, not what you pay. Second, streaming. A check that has to see a whole match before deciding must hold back the tail of the stream, which delays the first visible token slightly. A block mid-stream can stop what follows, but it cannot recall what was already sent. Azure documents the same shape for its own filter: a filtered streaming completion ends with finish_reason: "content_filter" after the segments already returned5.
Model and provider policy
The least glamorous guardrail is often the most effective: deciding which models a workload may call at all. An allowlist stops a developer pointing production traffic at an unevaluated model, keeps regulated data on models whose terms you have reviewed, and closes off expensive models where a budget would only notice afterwards. Block entries matter as much as allow entries. "Everything from this vendor except one model" and "every model except this vendor" are common policies, and a list that can only allow forces you to keep updating it as catalogs grow.
Model policy has to hold on every path a request can take: a direct model name, a named route, and any fallback the gateway tries after a failure. A policy that a route or a fallback can step around is not a policy. The broader governance process (who approves, how exceptions work) is in model access governance.
Where each check belongs
| Where a check belongs | In the application | At the gateway | At the provider |
|---|---|---|---|
| Knows | The user, the task, the documents and tools in play | Every app's traffic, keys, projects and providers | Its own model and its own policy categories |
| Good for | Authorization, tool approval, output validation against a schema | Uniform rules: secrets, PII formats, model allow and block lists, spend | Harm categories the vendor trains and maintains |
| Weak at | Consistency across teams and services | Business meaning it was never told about | Rules that must hold when you switch providers |
The application knows the user, the task and which tools are in play. Authorization of tool calls, validation of output against a schema and human approval steps belong there, because nothing outside the application can know whether this user may do this thing.
A gateway sees every application's traffic and knows which key, project and environment sent each request. Checks that should be the same everywhere belong there: secret and PII-format detection, literal deny terms, model allow and block lists, spend limits. Putting them in one place means a new service inherits them on day one, and a policy change takes effect everywhere at once rather than in whichever services were redeployed.
The provider runs its own safety systems. OpenAI's moderation endpoint returns flagged, per-category booleans and per-category scores from 0 to 1 for text and images, and OpenAI recommends treating the scores as signals for your own policy, noting that the model will change and thresholds may need recalibration6. Azure runs every prompt and completion through classifiers for hate, sexual, violence and self-harm at configurable severity levels, refuses a filtered prompt with HTTP 400 and code content_filter, and marks a filtered completion with finish_reason: "content_filter"5. These are valuable, but they are per provider. When you route across several providers, or fail over from one to another, the provider layer gives you a different policy on each path. Keep it as a second layer, not the only one.
Latency, cost and false positives
Each check has a price.
- Pattern checks run in memory, with no network call, so their added latency is small next to a model call. Their visible cost is mostly the streaming hold-back on output.
- Classifier checks add a network call before the model call, so their latency adds directly to time to first token. They need a timeout, and the timeout needs a decision about what happens when it fires. They also cost money if the classifier is a paid model.
- Provider filters are part of the provider's own latency and are not separately visible to you.
False positives are the cost people underestimate. A card detector with no checksum flags order numbers; a phone detector with loose rules flags invoice IDs; an injection classifier at a low threshold flags security researchers and anyone pasting a support ticket that quotes an attack. The remedies are the same for each: prefer detectors with structural tests (issuer prefix plus Luhn, country length plus mod-97), start new rules in record-only mode and read what they would have blocked, and tune classifier thresholds on labelled samples of your own traffic rather than on defaults. NIST's profile warns against extrapolating system performance from narrow, anecdotal assessments (MS-2.5-001), which applies directly to a threshold picked from a handful of test prompts4.
Fail open or fail closed
Every check eventually fails to run: the input is larger than the scanner accepts, the classifier times out, its provider is down. What happens next has to be decided in advance, per check.
The check cannot finish: timeout, oversized input, classifier unreachable. What happens?
- Fail closed (refuse)Nothing unchecked passes; an outage of the check becomes an outage of the feature
- Fail open (serve, mark it)Traffic keeps flowing; you must alert on the unchecked marker
- Regulated data, tools with side effectsUsually fail closed
- Low-stakes chat where availability matters mostFail open with an alert can be defensible
Provider filters make this choice for you, and it is worth knowing which way. Azure documents that when its content filtering system is down or cannot finish in time, the request still completes without filtering, and the only signal is an error object inside content_filter_results5. That is fail-open, and an application that never reads that field never knows. If you fail open, mark the request as unchecked and alert on the rate of unchecked requests. If you fail closed, size the checker's capacity and timeout like any other dependency on the critical path. NIST's profile asks that the system architecture can monitor outputs and handle and recover from errors when security anomalies are detected (MS-2.6-005), which is the same requirement in framework language4.
Logging policy for guardrails
A guardrail that redacts a card number and then writes the card number to a log has moved the problem, not solved it. The rule is simple: record that a check fired, which detector or classifier, at which stage, what action and how many matches — never the matched text. Keep policy changes in an append-only audit log, with the shape of each rule but not secret term lists, so you can answer "who turned this off, and when" during a review. The wider question of whether prompt content is captured at all is covered in the prompt logging policy guide.
Guardrails in INFRO: what is checked, and what a block returns
INFRO is a BYOK control plane: requests run on the customer's own provider accounts, and the gateway applies policy on the way. Its guardrails are the gateway layer described above, and the docs are explicit that detection is pattern-based and can miss things.
What is inspected. POST /v1/chat/completions, streaming and non-streaming: text in messages[].content for every role, tool-call arguments, and the response's text and tool-call arguments. The prompt of image and video requests is checked on input. Tool definitions, image and audio parts, and /v1/audio/* are not inspected. An organization setting, unguarded endpoints, can refuse requests to endpoints guardrails cannot inspect while any input rule is active.
Detectors. email, phone, card (issuer prefix plus Luhn), iban (country length plus mod-97), ssn (the dashed US form), ip, secret (prefixed keys from OpenAI, Anthropic, AWS access key ids, GitHub, Slack, Stripe live keys, Google, plus JWTs and PEM private keys) and deny_terms (up to 200 literal terms per rule, never regular expressions). Text is normalized with Unicode NFKC and stripped of zero-width characters before matching. Detectors run inside the gateway and call no outside service.
Rules. A policy holds up to 24 rules, each a detector, an action (block, redact with placeholders such as [EMAIL], or flag) and a stage (input, output or both). Policies can be set on the organization, a project and a gateway key; every applicable scope is evaluated and the strictest action wins, so a project or key can tighten what the organization set but never loosen it.
authentication
Key checks
403 model_not_allowed
Model allowlists
block, redact or flag
Pattern rules on the prompt
optional
Classifier checks
your own account
Budget, then the provider
output stage
Pattern rules on the response
Errors and records. A block returns 403 with error.type: "permission_denied" and code: "guardrail_blocked"; the message names the detector and where it matched (for example messages[1]), never the text. When a check cannot finish (over the 2 MB scan limit, or an engine failure), each policy's on_engine_error decides: block, the default, returns 403 guardrail_unavailable; allow serves the request and records it as flagged. Neither code is retryable. When a rule acts, the response carries an X-INFRO-Guardrails header and a guardrails field with the action and a count per detector, and request records from GET /v1/requests carry the same counts.
| INFRO rule actions | block | redact | flag |
|---|---|---|---|
| Request continues | ○No | ●, with placeholders such as [EMAIL] | ●, unchanged |
| Caller sees | 403 guardrail_blocked | X-INFRO-Guardrails: redacted | X-INFRO-Guardrails: flagged |
| Provider receives | Nothing (input stage) | The redacted text | The original text |
| Record keeps | Detector, stage and count | Detector, stage and count | Detector, stage and count |
Classifier checks. For meaning-level checks, a policy can add up to 10 yes/no questions about the prompt, answered in one call by a decision model (Jev by default, or another decision model) on the customer's own connection, which that provider bills. Presets cover prompt injection, jailbreak, harmful requests, self-harm risk (flag only by default) and off-topic use. Each check has block_at and flag_at thresholds on the probability of yes, the call has a timeout_ms from 300 to 5,000 milliseconds (default 2,000), and classifiers.on_error chooses fail-closed (block, the default) or fail-open (allow, recorded as an unavailable finding). Classifier checks run after the pattern rules have redacted the prompt and before the budget check, read prompts only, never responses, and the classifier call is its own row in the request log. The Jev guardrails guide covers how to write and tune those questions.
Model allowlists apply at organization, project and key scope, and a model must satisfy every non-empty list. Entries are catalog ids or vendor wildcards; a ! prefix blocks: !openai/gpt-4o one model, !openai/* a vendor. A block entry always wins in its list, and a list of block entries alone allows everything else, including models added later. Route targets and fallbacks stay subject to the lists. A denied model returns 403 with code: "model_not_allowed", naming the list that refused it.
What is stored. Matched text is never stored, in request records, the audit log, alerts, webhooks or OpenTelemetry. The audit log records guardrail.updated, guardrail.deleted and guardrail.settings_updated with each rule's shape and the number of terms, not the terms; allowlist changes are recorded as policy or key updates. The console's Try it runs sample text through the policies that would apply without storing it.
Plans. Model allowlists and the audit log are on every plan. Guardrails, including classifier checks, are included on the Business plan ($599 a month) and Enterprise. The plan gate applies to turning a policy on; turning one off or deleting it works on every plan, and a policy already set keeps being enforced after a move to a smaller plan. The guardrails reference lists every detector's exact match rules.
A checklist before you turn a rule on
- For each check, write down what is inspected, what is detected and what happens.
- Run new rules in record-only mode first and read what they would have done.
- Tune classifier thresholds on labelled samples of your own traffic.
- Decide fail-open or fail-closed per check, and alert on unchecked requests.
- Put shared rules (secrets, PII formats, model lists) at the gateway; keep authorization in the application.
- Confirm no log, trace or alert stores the text a check matched.
- Keep policy changes in an audit log someone reviews.