Structured outputs

Structured outputs turn a model's reply into JSON you can parse directly, instead of prose you have to scrape. INFRO supports both standard response_format modes: json_object, which produces syntactically valid JSON, and json_schema, which constrains output to a schema you define.

response_format is part of the standard OpenAI request shape on POST /v1/chat/completions, so it works unchanged through the OpenAI SDKs — no extra_body needed. On OpenAI-protocol connections INFRO forwards it unchanged; where a provider uses OpenAI's Responses protocol it is translated to that protocol's equivalent. INFRO never rewrites what comes back.

JSON object mode

Set response_format to {"type": "json_object"} and the model emits a single JSON value instead of prose. Nothing enforces which keys appear, though — describe the shape you want in the prompt and validate the result.

{
  "model": "openai/gpt-5.6-terra",
  "messages": [
    {"role": "system", "content": "Reply in JSON with keys sentiment and confidence."},
    {"role": "user", "content": "The checkout flow keeps timing out and support has not replied in two days."}
  ],
  "response_format": {"type": "json_object"}
}

JSON schema mode

json_schema mode constrains generation to a schema, so the response contains exactly the fields you declared. Pass a json_schema object inside response_format:

namestringrequired

An identifier for the schema, such as invoice_extraction. Letters, digits, underscores, and dashes.

strictboolean

Set true to enforce the schema during decoding. Without it, the schema is advisory and output can drift.

schemaobjectrequired

A standard JSON Schema object. For strict mode, set additionalProperties: false on every object and list every property in required — mark optional fields with a nullable type like ["string", "null"] instead of omitting them.

Strict mode works only on models whose vendor supports it. INFRO does not record this per model yet — the capabilities array in GET /v1/models is empty — so check the vendor's documentation and test before relying on it. If a provider rejects the schema, INFRO returns a 400 carrying the provider's own message.

If the request also carries fallbacks, or names a route whose targets span providers, make sure every model in the chain supports the mode you use — and remember a fallback to an Anthropic connection drops response_format. The response's model and provider fields report what actually served.

Example: extracting fields from text

The request below pulls four typed fields out of a free-text invoice line. With strict: true the response contains exactly those fields.

curl https://api.infro.io/v1/chat/completions \
  -H "Authorization: Bearer $INFRO_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "openai/gpt-5.6-sol",
    "messages": [
      {"role": "system", "content": "Extract invoice fields from the text."},
      {"role": "user", "content": "Invoice #4821 from Acme Corp, due March 15, 2026, total $1,240.50."}
    ],
    "response_format": {
      "type": "json_schema",
      "json_schema": {
        "name": "invoice_extraction",
        "strict": true,
        "schema": {
          "type": "object",
          "properties": {
            "invoice_number": {"type": "string"},
            "vendor": {"type": "string"},
            "due_date": {"type": "string", "description": "ISO 8601 date"},
            "total": {"type": "number"}
          },
          "required": ["invoice_number", "vendor", "due_date", "total"],
          "additionalProperties": false
        }
      }
    }
  }'

import os
import json
from openai import OpenAI

client = OpenAI(
    base_url="https://api.infro.io/v1",
    api_key=os.environ["INFRO_API_KEY"],
)

response = client.chat.completions.create(
    model="openai/gpt-5.6-sol",
    messages=[
        {"role": "system", "content": "Extract invoice fields from the text."},
        {"role": "user", "content": "Invoice #4821 from Acme Corp, due March 15, 2026, total $1,240.50."},
    ],
    response_format={
        "type": "json_schema",
        "json_schema": {
            "name": "invoice_extraction",
            "strict": True,
            "schema": {
                "type": "object",
                "properties": {
                    "invoice_number": {"type": "string"},
                    "vendor": {"type": "string"},
                    "due_date": {"type": "string", "description": "ISO 8601 date"},
                    "total": {"type": "number"},
                },
                "required": ["invoice_number", "vendor", "due_date", "total"],
                "additionalProperties": False,
            },
        },
    },
)

invoice = json.loads(response.choices[0].message.content)
print(invoice["vendor"], invoice["total"])

import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://api.infro.io/v1",
  apiKey: process.env.INFRO_API_KEY,
});

const response = await client.chat.completions.create({
  model: "openai/gpt-5.6-sol",
  messages: [
    { role: "system", content: "Extract invoice fields from the text." },
    { role: "user", content: "Invoice #4821 from Acme Corp, due March 15, 2026, total $1,240.50." },
  ],
  response_format: {
    type: "json_schema",
    json_schema: {
      name: "invoice_extraction",
      strict: true,
      schema: {
        type: "object",
        properties: {
          invoice_number: { type: "string" },
          vendor: { type: "string" },
          due_date: { type: "string", description: "ISO 8601 date" },
          total: { type: "number" },
        },
        required: ["invoice_number", "vendor", "due_date", "total"],
        additionalProperties: false,
      },
    },
  },
});

const invoice = JSON.parse(response.choices[0].message.content!);
console.log(invoice.vendor, invoice.total);

The structured object arrives as a JSON string in choices[0].message.content, the same place as any other completion — parse it with json.loads or JSON.parse. It also works with stream: true: deltas arrive as ordinary content chunks and the JSON parses once the stream completes. See Streaming.

Prompting tips

• In json_object mode, say "JSON" explicitly and sketch the shape you want — a one-line example object in the system prompt beats a paragraph of description.
• Use description strings inside the schema. They reach the model and steer field content like prompt text, without bloating the conversation.
• Keep extraction requests at low temperature (0 to 0.3). You want determinism, not creativity.
• Budget max_tokens for the whole object. Truncated JSON never parses; finish_reason: "length" means the output was cut off mid-object.
• If you send the same schema on every request, keep the system prompt stable and put the varying input last so provider-side prompt caching can kick in.

Handling validation failures

INFRO does not rewrite or repair model output. If a model emits invalid JSON in json_object mode, or a provider's strict decoder fails, the raw text comes back in a normal 200 response — there is no special error status for malformed output.

Treat parsing as part of the request lifecycle: parse, validate against your schema client-side (pydantic, zod, or a plain JSON Schema validator), and retry on failure. A cheap, effective retry appends the parse error and the bad output to the conversation and asks the model to correct it. Check finish_reason first — if it is "length", the fix is a higher max_tokens, not a retry. Transport-level failures are a separate concern; see Errors.

Combining with tool calling

Tools and response_format solve different problems. Use tool calling when the model should decide to take an action or fetch data mid-conversation; use response_format when you always want the final answer in a fixed shape. Don't define a dummy tool just to get structured output — json_schema mode is simpler and skips a round trip.

The two combine cleanly: run the tool loop as usual, returning each result as a tool role message with its tool_call_id, and once the model stops requesting tools its final message conforms to response_format. While tools are being called, finish_reason is tool_calls and content may be empty; the structured payload arrives on the final stop message.