A prompt and payload logging policy for LLM traffic

Every organization that sends traffic to language models has a logging policy. Often it is implicit: whatever the SDK's debug mode prints, whatever the proxy happens to keep, whatever a developer turned on during an incident and never turned off. An explicit policy is shorter than people expect, and it answers the questions a security reviewer, a data-protection officer and an on-call engineer will each ask sooner or later.

This checklist is organized in the order those decisions should be made. Each section ends with items you can tick.

Separate metadata from content

Metadata about a call — which model, which provider and account, how many tokens, how long it took, what it is estimated to cost your provider account, whether it failed and why, and the tags your application attached — is what budgets, cost attribution, failover analysis and incident response run on. It is useful for every request and rarely sensitive, with one exception: fields you fill yourself. A user_id that is an email address, or a tag holding a customer name, turns metadata into personal data.

Content — the prompt, the system instructions, retrieved documents, tool results and the completion — is a different kind of data. It is where personal and confidential information lives, and it is needed only for specific jobs: debugging a bad answer, building an evaluation set, investigating abuse.

• Metadata is recorded for every request, including failed and refused ones.
• Caller-supplied identifiers are opaque (a hashed or internal ID), never an email address or name.
• Content is stored in a separate place from metadata, with its own access rules and its own retention.

Decide the default for content

The GDPR's data minimisation principle asks that personal data be adequate, relevant and limited to what is necessary for the purpose¹. For LLM traffic the practical reading is that content capture should be off unless there is a named purpose for it. OWASP's guidance on sensitive information disclosure points the same way, recommending restricted access and clear policies on retention, usage and deletion².

Off by default does not mean never. It means capture is a decision someone takes for a reason, at the narrowest scope that serves it, with an end date.

• Content capture is off by default at the organization level.
• Turning it on requires a named owner and a stated purpose.
• Individual requests can opt out even when capture is on, for flows that must never be stored.
• A per-request flag cannot turn capture on when the organization policy says off.

Set retention and prove deletion

The storage limitation principle requires that personal data be kept in identifiable form for no longer than necessary for its purpose¹. A retention window on a settings page is only half of that. The other half is a job that deletes expired content and a read path that stops returning content at the deadline, whether or not the job has run yet.

Pick windows by purpose: hours for live debugging, days to a few weeks for incident investigation, longer only for a deliberately curated evaluation set that has its own review.

• Every captured payload carries its own expiry, set when it is written.
• Reads refuse content past its expiry, independently of deletion.
• A scheduled job deletes expired content, and its lag is monitored and alerted on.
• Shortening the window applies to content already captured, not only to new content.
• Deletion removes rows; it does not only set a flag.

Control who can read content

The ability to call a model, the ability to read request metadata and the ability to read captured content are three different permissions. A service key that sends traffic should not, by default, be able to read what every other service sent. A dashboard for cost owners needs metadata and never content.

Article 32 of the GDPR asks for technical and organizational measures appropriate to the risk, including the ability to ensure ongoing confidentiality of processing systems³. Separate grants are the simplest such measure to put in place.

• Reading content requires an explicit grant, given by an administrator, separate from reading metadata.
• History access defaults to the caller's own project or key, not the whole organization.
• Support staff at your gateway vendor cannot read your captured content.
• Administrative reads and exports are recorded in an audit log.

Redact before you store

Some content should never be stored, even when capture is on: card numbers, national identifiers, credentials that users paste into chat. Redacting at the gateway, before the request reaches the provider and before capture, means the stored copy and the provider's copy both hold a placeholder instead.

Pattern-based redaction is deterministic and cheap, and it misses anything outside the patterns it knows: a name, an address, a number written in words. Treat it as reducing exposure, not as a data-loss control.

• Known formats (card numbers, account numbers, credentials) are redacted before capture.
• The policy says plainly what redaction does not catch.
• Matched sensitive text is never written to logs, alerts or audit records; only counts are.

Control what leaves

Exports and telemetry are where logging policies usually leak. A CSV sent to finance, a span sent to an observability vendor, a webhook to an alerting tool — each moves data into a system with its own, often wider, access model.

• Request exports contain metadata only, never content.
• Telemetry attributes are built from an explicit allowlist, not by copying a database row.
• Telemetry and webhook destinations require HTTPS.
• Every export is recorded with who ran it and what it contained.

Remember the provider's logs

Your gateway's policy governs your gateway. The provider keeps its own records under your agreement with it. OpenAI, for example, states that API data is not used for training unless you opt in, that abuse-monitoring logs may be retained for up to 30 days unless longer retention is required by law, and that eligible organizations can apply for Modified Abuse Monitoring or Zero Data Retention⁴. Other providers document their own terms.

• The policy lists each provider account in use and its retention terms.
• Every fallback target is included, not only the primary provider.
• Anyone who sets logging: false knows it does not change provider-side retention.

Configuring payload capture in INFRO

INFRO records metadata for every request and captures content only when an organization opts in on a plan that includes it; the database column controlling capture defaults to off. A request can send logging: false to skip capture even when the organization has enabled it, and logging: true cannot override organization policy or plan limits. See Privacy & data.

{
  "model": "anthropic/claude-sonnet-5",
  "messages": [{ "role": "user", "content": "Summarize this contract clause." }],
  "logging": false
}

When capture is on, the organization picks a window of 24 hours, 14, 30, 60 or 90 days, or one year, and the stored window is the shorter of that choice and the plan's ceiling:

Reading captured content needs a separate payload permission on the gateway key, granted by an owner or admin, on top of its history scope; content is never returned past its retention deadline, even if deletion has not run. GET /v1/requests/export never includes prompts, responses or captured payloads, and every export is written to the audit log. If guardrails redact a prompt, the redacted text is what capture, failover targets and the OpenTelemetry export see; with the flag action nothing is changed, so capture stores the original. Guardrails are included from the Business plan and are pattern-based.