Skip to content

DocsFeatures

Images & vision

Send images to vision-capable models through one API: image_url content parts with https URLs or base64 data URIs, in request bodies up to 28MB.

INFRO is in private early access. Running these examples requires an early-access account, an active trial or subscription entitlement, and your own provider connection. Self-serve purchasing is not generally open. See Early access.

Vision-capable models accept images alongside text in the same request. Images go in as content parts on a user message — a public https URL or a base64 data: URI — using the standard OpenAI-compatible shape. OpenAI-protocol connections receive the parts unchanged; on Anthropic connections INFRO converts each image_url part to Anthropic's image block (a data: URI becomes a base64 source, an https URL a URL source).

A chat completion request body is limited to 28MB, which leaves room for about 20MB of base64-encoded image data, and a single message can carry several images. Providers bill images as input tokens under their own conversion rules; the estimate comes back in usage.cost.

Vision on chat completions is input only — responses are text or tool calls. To generate images, use the separate images endpoint (POST /v1/images/generations) with a connection that serves an image model.

Image content parts

When a user message contains images, its content field is an array of parts instead of a plain string. Text and image parts can be interleaved in any order, and the model reads them in sequence. The full message format is covered in the chat completions reference.

typestringrequired
"image_url" for an image part, "text" for a text part.
image_urlobjectrequired
Wrapper for the image reference. Present only on image_url parts.
image_url.urlstringrequired
An https URL, or a data: URI with base64-encoded bytes, e.g. data:image/png;base64,iVBORw0.... The whole request body is limited to 28MB.
textstring
The text of a "text" part.

Prefer an https URL when the image is publicly reachable — the serving provider fetches it directly and your request body stays small. Use a data: URI when the file is local or behind authentication. A request body over 28MB (20MB of image data is about 26.7MB once base64-encoded) fails with a 413 request_too_large before it is read — see error handling.

Describe an image

A complete request against a vision model. The example uses anthropic/claude-sonnet-5; the same body works with any vision-capable model your connections serve.

curl https://api.infro.io/v1/chat/completions \
  -H "Authorization: Bearer $INFRO_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "anthropic/claude-sonnet-5",
    "messages": [
      {
        "role": "user",
        "content": [
          {"type": "text", "text": "Describe this image in two sentences."},
          {
            "type": "image_url",
            "image_url": {"url": "https://upload.example.com/photos/harbor.jpg"}
          }
        ]
      }
    ]
  }'

Send a local image

For local files, base64-encode the bytes and pass them as a data: URI with the correct MIME type. Resize before encoding — base64 adds roughly 33% overhead, and an oversized image costs more tokens without improving answers.

Python
import base64
import os

from openai import OpenAI

client = OpenAI(
    base_url="https://api.infro.io/v1",
    api_key=os.environ["INFRO_API_KEY"],
)

with open("receipt.png", "rb") as f:
    encoded = base64.b64encode(f.read()).decode()

response = client.chat.completions.create(
    model="openai/gpt-5.6-sol",
    messages=[
        {
            "role": "user",
            "content": [
                {"type": "text", "text": "Extract the merchant, date, and total from this receipt."},
                {
                    "type": "image_url",
                    "image_url": {"url": f"data:image/png;base64,{encoded}"},
                },
            ],
        }
    ],
)

print(response.choices[0].message.content)

Multiple images in one message

Send several image parts in one message to compare, cross-reference, or batch-describe. Interleave text parts to label each image — models handle before/after prompts better when every image is introduced. The 28MB body limit applies to the request as a whole, and every image adds input tokens.

Python
import os
from openai import OpenAI

client = OpenAI(
    base_url="https://api.infro.io/v1",
    api_key=os.environ["INFRO_API_KEY"],
)

response = client.chat.completions.create(
    model="google/gemini-3.1-pro",
    messages=[
        {
            "role": "user",
            "content": [
                {"type": "text", "text": "First screenshot, before the deploy:"},
                {"type": "image_url", "image_url": {"url": "https://cdn.example.com/before.png"}},
                {"type": "text", "text": "Second screenshot, after the deploy:"},
                {"type": "image_url", "image_url": {"url": "https://cdn.example.com/after.png"}},
                {"type": "text", "text": "List every visual difference."},
            ],
        }
    ],
)

print(response.choices[0].message.content)

Which models support vision

INFRO does not record vision support per model yet — the capabilities array in GET /v1/models is empty — so check the model vendor's documentation. Current frontier text models such as openai/gpt-5.6-sol, anthropic/claude-sonnet-5 and google/gemini-3.1-pro accept images.

If a provider rejects an image, INFRO tries your next connection or fallback; if every one rejects it, you get a 400 invalid_request_error carrying the provider's own message. If you use fallbacks, keep every model in the chain vision-capable.

Cost and sizing tips

  • Images are billed as input tokens. Each provider converts image dimensions to tokens by its own rules, at the model's prompt rate. The estimate appears in usage.cost.
  • Resize before you send. Most models downscale large images internally, so a 4000px photo usually answers no better than a 1024px resize — it just costs more tokens and upload time.
  • Keep repeated images in stable positions. When the same images ride along in chat history across turns, prompt caching lets your provider bill them at its cached-input rate; reordering or editing earlier parts breaks the cache.
  • Pair vision with structured outputs for extraction. For receipts, forms, and screenshots, combine an image part with a strict JSON schema via structured outputs to get typed fields back instead of prose.
↑ ↓ navigate↵ open/ anywhere in docs