DocsFeatures
Prompt management
Keep prompts in INFRO as numbered versions with labels such as production, name one on a chat completion, and see which version served every request.
A managed prompt is a slug, such as support-reply, with numbered versions and movable labels. A version holds messages with {{variable}} placeholders, and can carry a default model and request settings. A version never changes once saved; a change is a new version, and going back is moving a label. A chat completion names the prompt, and the gateway puts the version's messages in front of the request's own before routing, model allowlists, guardrails and the spend check, so every check reads the final messages. The version is recorded on the request.
Name a prompt on a request
curl https://api.infro.io/v1/chat/completions \
-H "Authorization: Bearer $INFRO_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"model": "openai/gpt-5",
"prompt": {
"id": "support-reply",
"label": "production",
"variables": { "customer_name": "Ada" }
},
"messages": [{ "role": "user", "content": "Where is my order?" }]
}'completion = client.chat.completions.create(
model="openai/gpt-5",
messages=[{"role": "user", "content": "Where is my order?"}],
extra_body={
"prompt": {"id": "support-reply", "variables": {"customer_name": "Ada"}},
},
)prompt.idstringrequired- The prompt's slug.
prompt.labelstring- The label to serve, such as
productionorstaging. prompt.versioninteger- A version number, from 1. Name a label or a version, not both; with neither, the request is served from
production. prompt.variablesobject- Values for the version's
{{variables}}: text, numbers or true/false (numbers and booleans are written as text). At most 50, each up to 100,000 characters.
Or name it in a header, for a client that cannot add a body field: X-INFRO-Prompt: support-reply@production, support-reply@3, or a bare support-reply for production. A header carries no variables, so it suits versions whose variables are optional. When both are sent, the body's prompt wins.
curl https://api.infro.io/v1/chat/completions \
-H "Authorization: Bearer $INFRO_API_KEY" \
-H "X-INFRO-Prompt: support-reply@3" \
-H "Content-Type: application/json" \
-d '{"model": "openai/gpt-5", "messages": [{"role": "user", "content": "Where is my order?"}]}'How a request is built
- The version's messages come first, then the request's
messages.messagesmay be left out when the version holds every message. - Each
{{name}}(spaces inside the braces allowed) is replaced by its value once. There is no logic, no HTML escaping and no recursion: a value that itself contains{{x}}is sent as written. A required variable the request did not send is400prompt_variable_missing, naming it; an optional one takes its default, or empty text. - A field the request sets wins over the version's default. The version's
modelis used only when the request sends none, and its settings (temperature,top_p,max_tokens,max_completion_tokens,stop,frequency_penalty,presence_penalty,seed,response_format,tools,tool_choice) only where the request leaves them out;max_tokensandmax_completion_tokenscount as one setting. - The response carries
X-INFRO-Prompt-Version: support-reply@3, on errors after resolution too. The request record carriespromptwith itsid,slugandversion, and runs list the versions they used. - A label moved in the console or the management API reaches every request within 5 seconds: versions and labels are read through the gateway's configuration cache, never with a database query per request.
- Prompts are resolved on
POST /v1/chat/completions, and on/v1/messagesand/v1/responsesas described below.
A prompt can be limited to one project. It then serves that project's gateway keys (and the console's Playground and Studio); for any other key it answers 404 prompt_not_found, as if it did not exist.
On /v1/messages and /v1/responses
Claude Code, Codex and the agent SDKs can name a prompt too, resolved at the same point (before routing, guardrails and the spend check), with the same errors, X-INFRO-Prompt-Version header and request record. It is applied to the body before anything else reads it, so a provider that takes the API as written (Anthropic for Messages, OpenAI and Azure OpenAI for Responses) and a provider INFRO translates for both receive it.
/v1/messages: thepromptfield (INFRO's; Anthropic's API has none, and it is never sent on) orX-INFRO-Prompt. The version'ssystemanddevelopermessages become the request'ssystem, before its own: joined with a blank line to asystemstring, or as a first text block of asystemlist. Itsuserandassistantmessages go beforemessages.POST /v1/messages/count_tokenscounts them too./v1/responses:X-INFRO-Promptonly. The body'spromptis OpenAI's own (a prompt stored at OpenAI) and passes through untouched. The version'ssystemanddevelopermessages becomeinstructions, before its own; the rest go beforeinput(a textinputbecomes a user message after them). A header carries no variables, so a version whose required variables have no default answers400prompt_variable_missinghere.- Of the version's settings, only those that mean the same in the API are applied:
temperatureandtop_p,max_tokens(asmax_tokenson Messages,max_output_tokenson Responses) and, on Messages,stopasstop_sequences.seed, the penalties,response_format,toolsandtool_choiceare written in chat's shape and are not applied; keep them in the request.
curl https://api.infro.io/v1/messages \
-H "x-api-key: $INFRO_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"model": "anthropic/claude-sonnet-5.5",
"max_tokens": 1024,
"prompt": {"id": "code-review", "label": "production", "variables": {"lang": "Go"}},
"messages": [{"role": "user", "content": "Review this diff."}]
}'curl https://api.infro.io/v1/responses \
-H "Authorization: Bearer $INFRO_API_KEY" \
-H "X-INFRO-Prompt: code-review@production" \
-H "Content-Type: application/json" \
-d '{"model": "openai/gpt-5.4", "input": "Review this diff."}'Fetch a version without routing through INFRO
GET /v1/prompts/{slug} returns the production version, or ?label= / ?version=: its messages, variables, model, params, version and prompt_id, with any gateway key that may use the prompt. Use it in an application that calls a provider directly. A version fetched this way is not enforced: INFRO never sees the call it is used in, so no guardrail, allowlist or spend limit applies and no request records the version. The answer says so with "enforced": false. Naming the prompt on a chat completion is what makes it enforced.
Versions and labels
Creating a prompt saves version 1 and points production at it. Labels are 1 to 32 lower-case letters, digits or dashes; slugs are up to 64 characters of lower-case letters, digits and _ . -. A version holds at most 50 messages (system, developer, user or assistant) and 100,000 characters. production is protected: moving or protecting a protected label is a publish, made by an owner or admin in the console with a recent confirmation. Owners, admins and developers create prompts, save versions and move unprotected labels such as staging; every role that reads analytics reads prompts. Every change is in the audit log.
The console's Prompts screen shows each version's change against the previous one and, over a window you choose, its requests, error rate, p50 and p95 latency, estimated provider cost per request, mean score and feedback share. Rolling back is moving the label to the earlier version.
Rollback rules
A rule on a label moves it back to the previous version, or alerts, when the version it points at scores below a limit over a window: the mean of a score, the pass rate of a true/false score, or your end users' thumbs-down share. See Prompt rollback rules.
Management API
Scripts and CI use /manage/v1/prompts with a management token holding prompts:read or prompts:write. prompts is a data family: *:read does not include it, and a project-restricted token cannot hold it. A token acts with its creator's role.
| Operation | Scope | What it does |
|---|---|---|
GET /prompts | prompts:read | Every prompt, with its labels and their versions. |
GET /prompts/{slug} | prompts:read | One prompt with its versions (newest first) and labels. ETag is the prompt's version. |
POST /prompts | prompts:write | slug, description, project_id, and version 1: messages, variables, model, params, note. production points at version 1, protected. |
POST /prompts/{slug}/versions | prompts:write | A new version; If-Match optional. same_as_previous is true when it is identical to the newest. |
PUT /prompts/{slug}/labels/{label} | prompts:write | {"version": 3}; creates the label when it does not exist. If-Match required. A protected label answers 403 prompt_label_protected: a token never publishes. |
curl https://api.infro.io/manage/v1/prompts \
-H "Authorization: Bearer $INFRO_MANAGEMENT_TOKEN" \
-H "Content-Type: application/json" \
-d '{
"slug": "support-reply",
"description": "First answer to a support ticket",
"messages": [
{ "role": "system", "content": "You answer for Acme support. The customer is {{customer_name}}." }
],
"params": { "temperature": 0.2 },
"note": "First version"
}'curl https://api.infro.io/manage/v1/prompts/support-reply/labels/staging \
-X PUT \
-H "Authorization: Bearer $INFRO_MANAGEMENT_TOKEN" \
-H 'If-Match: "4"' \
-H "Content-Type: application/json" \
-d '{"version": 3}'A variables entry is {name, required, default}; every placeholder a version uses is a variable, required unless declared otherwise. The INFRO MCP server reads prompts with list_prompts and get_prompt under the same prompts:read scope; it does not change them.