# Structured outputs

> Request JSON from any model on INFRO: json_object and strict json_schema modes, model support, prompting tips, and handling validation failures.

Source: https://infro.io/docs/features/structured-outputs

> **Note:** INFRO is in private early access. Running these examples requires an early-access account, an active trial or subscription entitlement, and your own provider connection. Self-serve purchasing is not generally open. See [Early access](https://infro.io/early-access).

Structured outputs turn a model's reply into JSON you can parse directly, instead of prose you have to scrape. INFRO supports both standard `response_format` modes: `json_object`, which produces syntactically valid JSON, and `json_schema`, which constrains output to a schema you define.

`response_format` is part of the standard OpenAI request shape on [POST /v1/chat/completions](https://infro.io/docs/api/chat-completions), so it works unchanged through the OpenAI SDKs — no `extra_body` needed. On OpenAI-protocol connections INFRO forwards it unchanged; where a provider uses OpenAI's Responses protocol it is translated to that protocol's equivalent. INFRO never rewrites what comes back.

> **Warning:** Anthropic connections do not support `response_format` today: INFRO drops it when a request is served by an Anthropic connection, so the reply is ordinary text. For JSON from Claude models, describe the shape in the prompt, or use [tool calling](https://infro.io/docs/features/tool-calling) with a single tool whose parameters are your schema, and validate the result.

| Mode | Guarantee | Model support |
| --- | --- | --- |
| `json_object` | Syntactically valid JSON; no guarantee about keys or shape | Widely supported |
| `json_schema` with `strict: true` | JSON conforming to your schema | Models whose vendor documents strict schema support |

## JSON object mode

Set `response_format` to `{"type": "json_object"}` and the model emits a single JSON value instead of prose. Nothing enforces which keys appear, though — describe the shape you want in the prompt and validate the result.

```json
{
  "model": "openai/gpt-5.6-terra",
  "messages": [
    {"role": "system", "content": "Reply in JSON with keys sentiment and confidence."},
    {"role": "user", "content": "The checkout flow keeps timing out and support has not replied in two days."}
  ],
  "response_format": {"type": "json_object"}
}
```

> **Note:** Most providers reject `json_object` requests unless the word "JSON" appears in your messages. Include an instruction like "Reply in JSON" in the system prompt.

## JSON schema mode

`json_schema` mode constrains generation to a schema, so the response contains exactly the fields you declared. Pass a `json_schema` object inside `response_format`:

- `name` (string, required) — An identifier for the schema, such as `invoice_extraction`. Letters, digits, underscores, and dashes.
- `strict` (boolean) — Set `true` to enforce the schema during decoding. Without it, the schema is advisory and output can drift.
- `schema` (object, required) — A standard JSON Schema object. For strict mode, set `additionalProperties: false` on every object and list every property in `required` — mark optional fields with a nullable type like `["string", "null"]` instead of omitting them.

Strict mode works only on models whose vendor supports it. INFRO does not record this per model yet — the `capabilities` array in [GET /v1/models](https://infro.io/docs/models) is empty — so check the vendor's documentation and test before relying on it. If a provider rejects the schema, INFRO returns a 400 carrying the provider's own message.

If the request also carries [fallbacks](https://infro.io/docs/features/fallbacks), or names a route whose targets span providers, make sure every model in the chain supports the mode you use — and remember a fallback to an Anthropic connection drops `response_format`. The response's `model` and `provider` fields report what actually served.

## Example: extracting fields from text

The request below pulls four typed fields out of a free-text invoice line. With `strict: true` the response contains exactly those fields.

**curl**

```bash
curl https://api.infro.io/v1/chat/completions \
  -H "Authorization: Bearer $INFRO_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "openai/gpt-5.6-sol",
    "messages": [
      {"role": "system", "content": "Extract invoice fields from the text."},
      {"role": "user", "content": "Invoice #4821 from Acme Corp, due March 15, 2026, total $1,240.50."}
    ],
    "response_format": {
      "type": "json_schema",
      "json_schema": {
        "name": "invoice_extraction",
        "strict": true,
        "schema": {
          "type": "object",
          "properties": {
            "invoice_number": {"type": "string"},
            "vendor": {"type": "string"},
            "due_date": {"type": "string", "description": "ISO 8601 date"},
            "total": {"type": "number"}
          },
          "required": ["invoice_number", "vendor", "due_date", "total"],
          "additionalProperties": false
        }
      }
    }
  }'
```

**Python**

```python
import os
import json
from openai import OpenAI

client = OpenAI(
    base_url="https://api.infro.io/v1",
    api_key=os.environ["INFRO_API_KEY"],
)

response = client.chat.completions.create(
    model="openai/gpt-5.6-sol",
    messages=[
        {"role": "system", "content": "Extract invoice fields from the text."},
        {"role": "user", "content": "Invoice #4821 from Acme Corp, due March 15, 2026, total $1,240.50."},
    ],
    response_format={
        "type": "json_schema",
        "json_schema": {
            "name": "invoice_extraction",
            "strict": True,
            "schema": {
                "type": "object",
                "properties": {
                    "invoice_number": {"type": "string"},
                    "vendor": {"type": "string"},
                    "due_date": {"type": "string", "description": "ISO 8601 date"},
                    "total": {"type": "number"},
                },
                "required": ["invoice_number", "vendor", "due_date", "total"],
                "additionalProperties": False,
            },
        },
    },
)

invoice = json.loads(response.choices[0].message.content)
print(invoice["vendor"], invoice["total"])
```

**TypeScript**

```typescript
import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://api.infro.io/v1",
  apiKey: process.env.INFRO_API_KEY,
});

const response = await client.chat.completions.create({
  model: "openai/gpt-5.6-sol",
  messages: [
    { role: "system", content: "Extract invoice fields from the text." },
    { role: "user", content: "Invoice #4821 from Acme Corp, due March 15, 2026, total $1,240.50." },
  ],
  response_format: {
    type: "json_schema",
    json_schema: {
      name: "invoice_extraction",
      strict: true,
      schema: {
        type: "object",
        properties: {
          invoice_number: { type: "string" },
          vendor: { type: "string" },
          due_date: { type: "string", description: "ISO 8601 date" },
          total: { type: "number" },
        },
        required: ["invoice_number", "vendor", "due_date", "total"],
        additionalProperties: false,
      },
    },
  },
});

const invoice = JSON.parse(response.choices[0].message.content!);
console.log(invoice.vendor, invoice.total);
```

The structured object arrives as a JSON string in `choices[0].message.content`, the same place as any other completion — parse it with `json.loads` or `JSON.parse`. It also works with `stream: true`: deltas arrive as ordinary content chunks and the JSON parses once the stream completes. See [Streaming](https://infro.io/docs/api/streaming).

## Prompting tips

- In `json_object` mode, say "JSON" explicitly and sketch the shape you want — a one-line example object in the system prompt beats a paragraph of description.
- Use `description` strings inside the schema. They reach the model and steer field content like prompt text, without bloating the conversation.
- Keep extraction requests at low `temperature` (0 to 0.3). You want determinism, not creativity.
- Budget `max_tokens` for the whole object. Truncated JSON never parses; `finish_reason: "length"` means the output was cut off mid-object.
- If you send the same schema on every request, keep the system prompt stable and put the varying input last so provider-side [prompt caching](https://infro.io/docs/features/prompt-caching) can kick in.

## Handling validation failures

INFRO does not rewrite or repair model output. If a model emits invalid JSON in `json_object` mode, or a provider's strict decoder fails, the raw text comes back in a normal 200 response — there is no special error status for malformed output.

Treat parsing as part of the request lifecycle: parse, validate against your schema client-side (pydantic, zod, or a plain JSON Schema validator), and retry on failure. A cheap, effective retry appends the parse error and the bad output to the conversation and asks the model to correct it. Check `finish_reason` first — if it is `"length"`, the fix is a higher `max_tokens`, not a retry. Transport-level failures are a separate concern; see [Errors](https://infro.io/docs/api/errors).

> **Warning:** A 200 status means the request succeeded, not that the content parses. Always validate structured output client-side before acting on it.

## Combining with tool calling

Tools and `response_format` solve different problems. Use [tool calling](https://infro.io/docs/features/tool-calling) when the model should decide to take an action or fetch data mid-conversation; use `response_format` when you always want the final answer in a fixed shape. Don't define a dummy tool just to get structured output — `json_schema` mode is simpler and skips a round trip.

The two combine cleanly: run the tool loop as usual, returning each result as a `tool` role message with its `tool_call_id`, and once the model stops requesting tools its final message conforms to `response_format`. While tools are being called, `finish_reason` is `tool_calls` and `content` may be empty; the structured payload arrives on the final `stop` message.