Chat completions

The chat completions endpoint is the core of the API and is wire-compatible with OpenAI's. Any SDK or tool that speaks the OpenAI protocol works against it by changing the base URL.

POST https://api.infro.io/v1/chat/completions

Request body

modelstringrequired

A configured route slug, or a model ID in vendor/model-name form, e.g. anthropic/claude-opus-5. Use the model ids your connected providers serve; GET /v1/models lists what your key can call.

messagesarrayrequired

Conversation history. Each message has role (system | developer | user | assistant | tool) and content (string, or an array of text/image parts for vision).

streamboolean

When true, tokens arrive as server-sent events. See Streaming.

max_tokensinteger

Upper bound on generated tokens. When omitted, the provider's own default applies, with two exceptions: Anthropic requires a value, so INFRO sends the model's maximum output (32,768 tokens where the catalog lists none); and under a blocking budget INFRO sends that same ceiling to every provider, so a response cannot outrun what the budget reserved. max_completion_tokens is accepted as an alias. INFRO sends the ceiling in the field each model takes: OpenAI's reasoning models (GPT-5 and later, the o-series) refuse max_tokens, so they are sent max_completion_tokens whichever name you used.

temperaturenumber

Sampling temperature, 0–2. Higher is more random. A model that refuses it is not sent it: Claude 4.7 and later (Opus 4.7 and 4.8, the Claude 5 family, Fable) refuse temperature, top_p and top_k, and OpenAI's o-series takes only its default. The response then lists what was dropped in X-INFRO-Params-Dropped (see below).

top_pnumber

Nucleus sampling. Use temperature or top_p, not both: Claude 4.5 and 4.6 refuse the pair, so INFRO keeps temperature and drops top_p for them.

stopstring | array

Up to 4 sequences where generation halts.

frequency_penalty / presence_penaltynumber

-2.0–2.0. Forwarded to the provider; whether it is honoured depends on the provider.

toolsarray

Function definitions the model may call. Translated to each provider's own format — see Tool calling.

tool_choicestring | object

"auto" (default), "none", "required", or {"type": "function", "function": {"name": "..."}} to force one tool.

response_formatobject

{"type": "json_object"} for JSON mode or {"type": "json_schema", "json_schema": {...}} for validated output. See Structured outputs.

seedinteger

Best-effort deterministic sampling on providers that support it.

userstring

Opaque end-user ID, forwarded to the provider as sent. To attribute cost to your users in INFRO's request log, send metadata.user_id.

stream_optionsobject

{"include_usage": true} appends a final chunk with token counts when streaming.

INFRO extensions

INFRO-specific fields sit at the top level of the request body (use extra_body in the OpenAI SDKs). Other OpenAI-compatible backends ignore them, so requests stay portable.

routingobject

routing.providers chooses which of your providers may serve this request, by provider id (openai, aws-bedrock, openrouter): {"allow": ["aws-bedrock"]} keeps only those, {"deny": ["openrouter"]} removes those. It applies to the model's connections, a route's targets and the fallbacks, and narrows what your API key allows without widening it. Leave it out to let the gateway choose among every connection that serves the model. An id that is not a provider is 400 unknown_provider; when nothing it allows can serve, the 503 connection_out_of_scope says so. Legacy: routing.policy (cheapest, fastest, balanced) is accepted and ignored. routing.regions is checked against your organization's region policy and recorded, but does not move a request between your connections. Use a named route to set the order connections are tried in.

fallbacksarray

Up to 5 ordered model IDs to try after every connection for model has failed, e.g. ["openai/gpt-5.6-terra"]. An ID that is not a model you can call is refused with 400 code unknown_fallback_model, naming it; a known model is skipped when none of your connections serves it, when your policy does not allow it, or when it is a different modality. Ignored when model is a route slug (an unknown ID is still refused): a route's failover is set on the route. See Failover & fallbacks.

loggingboolean

Set false to disable optional prompt and response capture. Organization opt-in and plan limits also apply. Usage metadata is still recorded for platform metering and estimated provider cost. See Privacy & data.

metadataobject

Your own annotation, recorded on the request and never sent to the provider. user_id, session_id and tags.<key> can be filtered in the request log.

Response cache

The response cache extension. It narrows what the organization's settings allow and never turns the cache on where they have it off; the headers X-INFRO-Cache, X-INFRO-Cache-TTL and X-INFRO-Cache-Namespace are equivalents, and the body wins field by field. logging: false also means the answer is never written to the cache.

cacheobject

{mode, ttl_seconds, namespace}. mode: default, bypass, refresh or read_only. ttl_seconds: 60–2592000 and at most the plan's maximum, otherwise 400 cache_ttl_too_long. namespace: 1–128 characters of letters, digits and . _ : -, partitioning the key. A malformed header is 400 invalid_cache_control.

Example request

curl https://api.infro.io/v1/chat/completions \
  -H "Authorization: Bearer $INFRO_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "anthropic/claude-sonnet-5",
    "messages": [
      {"role": "system", "content": "You are a concise assistant."},
      {"role": "user", "content": "Summarize SSE in one sentence."}
    ],
    "max_tokens": 200,
    "fallbacks": ["openai/gpt-5.6-terra"]
  }'

completion = client.chat.completions.create(
    model="anthropic/claude-sonnet-5",
    messages=[
        {"role": "system", "content": "You are a concise assistant."},
        {"role": "user", "content": "Summarize SSE in one sentence."},
    ],
    max_tokens=200,
    extra_body={
        "fallbacks": ["openai/gpt-5.6-terra"],
    },
)

const completion = await client.chat.completions.create({
  model: "anthropic/claude-sonnet-5",
  messages: [
    { role: "system", content: "You are a concise assistant." },
    { role: "user", content: "Summarize SSE in one sentence." },
  ],
  max_tokens: 200,
  // @ts-expect-error INFRO extensions
  fallbacks: ["openai/gpt-5.6-terra"],
});

Response

{
  "id": "req_9f27c1d0a4b84e6f",
  "object": "chat.completion",
  "created": 1755950000,
  "model": "anthropic/claude-sonnet-5",
  "route": "primary",
  "route_name": null,
  "provider": "anthropic",
  "connection": "conn_4b1e9a07",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "Server-sent events stream one-way updates from server to client over a single HTTP connection."
      },
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 31,
    "completion_tokens": 19,
    "total_tokens": 50,
    "cost": 0.000378,
    "price_source": "list"
  }
}

routestring

INFRO extension: which position in the resolved chain served the request — primary, or standby_a, standby_b and so on after a failover. It tells you whether your traffic moved between your connections. The top-level id is INFRO's request id, the same value as the X-INFRO-Request-Id header.

route_namestring | null

INFRO extension: the slug of the named route the request called, or null when it named a model directly.

providerstring

INFRO extension: the provider kind whose account served the request — openai, anthropic, aws-bedrock and so on. It is one of your own connections; connection names which.

connectionstring

INFRO extension: the id of the provider connection that served the request.

usage.costnumber

INFRO extension: what your provider will bill you for this request, in USD. An estimate, unless the provider reported the cost in its own response (price_source is then provider, and the provider's figure is also in usage.cost_usd). INFRO takes no part of it. null when the model is unpriced; that does not mean free.

usage.price_sourcestring

INFRO extension: where cost came from — provider (reported by the provider in its response), custom_provider (your negotiated rate for the provider that served it), custom (your rate for the model, whatever serves it) or list (the model vendor's published rate). Absent when cost is null. See billing.

guardrailsobject

INFRO extension, present only when your organization's guardrails redacted, flagged or blocked something: {action, findings}, where each finding is {detector, stage, action, count}. Counts only — the matched text is never returned. On a stream it is on each choice's finish chunk and on the usage chunk.

finish_reasonstring

stop, length (hit max_tokens), tool_calls, or content_filter.

When the model that served refuses a sampling parameter you sent, INFRO does not send it and the response carries X-INFRO-Params-Dropped naming each one, for example X-INFRO-Params-Dropped: temperature, top_p. Only sampling parameters a model takes at its default and nothing else are ever dropped (temperature, top_p, top_k, frequency_penalty, presence_penalty, seed, logprobs, top_logprobs, logit_bias); a refused stop, tools or response_format is returned to you as the provider's 400. When a provider refuses a parameter INFRO has no rule for yet, INFRO drops it and asks the same connection once more, and remembers the answer for the following requests. The header is absent when nothing was dropped.

When a guardrail acted on the request, the response also carries the header X-INFRO-Guardrails: redacted, flagged or blocked. On a stream the header reflects the input check only, because it is sent before any response text exists. A request refused by a guardrail returns 403 with code: "guardrail_blocked" or "guardrail_unavailable" — see Guardrails.

Response cache fields

Every chat response that reached the cache step carries X-INFRO-Cache: HIT | MISS | BYPASS | DISABLED | UNCACHEABLE; a hit adds X-INFRO-Cache-Age and X-INFRO-Cache-Source. The body gains a top-level cache object, absent when the cache is off for the request: {status} with miss, bypass or uncacheable, or on a hit {status: "hit", age_seconds, source_request_id, provider_billed: false}. On a hit, usage carries the original token counts, usage.cost is 0 and usage.price_source is gateway_cache; model, provider, connection and route describe what produced the answer originally, and route_name is the route this request named. See Response cache.

Compatibility notes

• Top-level parameters INFRO does not model are forwarded to the provider unchanged, so a new OpenAI parameter does not break your requests. Whether the provider honours one is up to the provider; a sampling parameter the serving model refuses is dropped and reported in X-INFRO-Params-Dropped.
• usage token counts are the counts your provider reported for the request; INFRO does not re-tokenize.
• The legacy POST /v1/completions endpoint is not supported; use chat completions.