# Tool calling

> Use the OpenAI tools schema with every tool-capable model on INFRO. Covers the full loop, tool_choice, parallel calls, and streaming deltas.

Source: https://infro.io/docs/features/tool-calling

> **Note:** INFRO is in private early access. Running these examples requires an early-access account, an active trial or subscription entitlement, and your own provider connection. Self-serve purchasing is not generally open. See [Early access](https://infro.io/early-access).

Every tool-capable model on INFRO uses the same OpenAI `tools` and `tool_choice` schema. Define tools once. Most provider kinds speak the OpenAI protocol, so INFRO forwards the schema to them as is; for Anthropic connections INFRO translates tools, `tool_choice`, tool calls and tool results to and from Anthropic's `tool_use` blocks. The shapes on this page are the same whichever of your connections serves the request.

Tool calling is a loop: send tool definitions with a [chat completion](https://infro.io/docs/api/chat-completions), the model responds with one or more `tool_calls` instead of text, you execute them and return the results, and the model produces its final answer. This page walks the full loop, then covers `tool_choice`, parallel calls, and streaming.

## Check model support

INFRO does not record tool support per model yet — the `capabilities` array in [GET /v1/models](https://infro.io/docs/models) is empty — so check the model vendor's documentation. Most current-generation text models support tools, but capability sets differ between a vendor's flagship and its small models, so test at integration time rather than assuming. If a provider rejects the `tools` parameter, INFRO returns a 400 carrying the provider's own message.

## Define a tool

Tools go in the top-level `tools` array. Each entry describes one function the model may call; arguments are specified as JSON Schema.

- `type` (string, required) — Always "function".
- `function.name` (string, required) — The name the model uses to call the tool. Letters, numbers, underscores, and dashes; 64 characters max.
- `function.description` (string) — What the tool does and when to use it. The model leans on this heavily when deciding whether to call — write it for the model, not for humans.
- `function.parameters` (object) — JSON Schema for the arguments. Keep it flat where you can, and use enum and per-property description fields — they measurably improve argument quality.

```json
{
  "type": "function",
  "function": {
    "name": "get_weather",
    "description": "Get the current weather for a city. Use when the user asks about weather conditions.",
    "parameters": {
      "type": "object",
      "properties": {
        "city": {
          "type": "string",
          "description": "City name, e.g. \"Lisbon\""
        },
        "unit": {
          "type": "string",
          "enum": ["celsius", "fahrenheit"],
          "description": "Temperature unit. Defaults to celsius."
        }
      },
      "required": ["city"]
    }
  }
}
```

Tool definitions are part of the prompt prefix, so keep the `tools` array byte-stable across requests — system prompt first, then tools, then history — and provider-side [prompt caching](https://infro.io/docs/features/prompt-caching) can keep hitting.

## The tool-calling loop

When the model decides to call a tool, the response has `finish_reason` `"tool_calls"` and the assistant message carries a `tool_calls` array instead of (or alongside) text content. Each entry:

- `id` (string) — Unique id for this call. Echo it back as tool_call_id on the result message.
- `type` (string) — Always "function".
- `function.name` (string) — Which tool to run.
- `function.arguments` (string) — JSON-encoded arguments — a string, not an object. It is model-generated: parse defensively and validate before executing anything with side effects.

Sending results back has three rules: append the assistant message exactly as received (it carries the `tool_calls` your results answer), append one `"tool"` role message per call with `tool_call_id` set to the call's `id` and `content` set to the result as a string, then request the next completion with the same `tools` array. The complete loop:

**Python**

```python
import os
import json
from openai import OpenAI

client = OpenAI(
    base_url="https://api.infro.io/v1",
    api_key=os.environ["INFRO_API_KEY"],
)

tools = [
    {
        "type": "function",
        "function": {
            "name": "get_weather",
            "description": "Get the current weather for a city.",
            "parameters": {
                "type": "object",
                "properties": {
                    "city": {"type": "string", "description": "City name"},
                    "unit": {"type": "string", "enum": ["celsius", "fahrenheit"]},
                },
                "required": ["city"],
            },
        },
    }
]

def get_weather(city: str, unit: str = "celsius") -> str:
    # Replace with a real lookup
    return json.dumps({"city": city, "temp": 21, "unit": unit, "conditions": "clear"})

messages = [{"role": "user", "content": "What's the weather in Lisbon right now?"}]

response = client.chat.completions.create(
    model="anthropic/claude-sonnet-5",
    messages=messages,
    tools=tools,
)
message = response.choices[0].message

while message.tool_calls:
    # 1. Append the assistant turn that requested the calls
    messages.append(message)

    # 2. Execute every call, one tool message per tool_call_id
    for tool_call in message.tool_calls:
        if tool_call.function.name == "get_weather":
            args = json.loads(tool_call.function.arguments)
            result = get_weather(**args)
        messages.append({
            "role": "tool",
            "tool_call_id": tool_call.id,
            "content": result,
        })

    # 3. Ask for the next turn with the same tools
    response = client.chat.completions.create(
        model="anthropic/claude-sonnet-5",
        messages=messages,
        tools=tools,
    )
    message = response.choices[0].message

print(message.content)
```

**TypeScript**

```typescript
import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://api.infro.io/v1",
  apiKey: process.env.INFRO_API_KEY,
});

const tools: OpenAI.ChatCompletionTool[] = [
  {
    type: "function",
    function: {
      name: "get_weather",
      description: "Get the current weather for a city.",
      parameters: {
        type: "object",
        properties: {
          city: { type: "string", description: "City name" },
          unit: { type: "string", enum: ["celsius", "fahrenheit"] },
        },
        required: ["city"],
      },
    },
  },
];

function getWeather(city: string, unit = "celsius"): string {
  // Replace with a real lookup
  return JSON.stringify({ city, temp: 21, unit, conditions: "clear" });
}

const messages: OpenAI.ChatCompletionMessageParam[] = [
  { role: "user", content: "What's the weather in Lisbon right now?" },
];

let response = await client.chat.completions.create({
  model: "anthropic/claude-sonnet-5",
  messages,
  tools,
});
let message = response.choices[0].message;

while (message.tool_calls?.length) {
  // 1. Append the assistant turn that requested the calls
  messages.push(message);

  // 2. Execute every call, one tool message per tool_call_id
  for (const toolCall of message.tool_calls) {
    if (toolCall.type !== "function") continue;
    const args = JSON.parse(toolCall.function.arguments);
    messages.push({
      role: "tool",
      tool_call_id: toolCall.id,
      content: getWeather(args.city, args.unit),
    });
  }

  // 3. Ask for the next turn with the same tools
  response = await client.chat.completions.create({
    model: "anthropic/claude-sonnet-5",
    messages,
    tools,
  });
  message = response.choices[0].message;
}

console.log(message.content);
```

The loop runs until the model answers in plain text (`finish_reason` `"stop"`); multi-step tasks commonly take two or three iterations — the model calls a tool, reads the result, and calls another. If you use [fallbacks](https://infro.io/docs/features/fallbacks), pick fallback models that also support tools — one that can't call tools answers in plain text instead — and read the response `model` field to see which model actually served each turn.

## tool_choice

`tool_choice` controls whether the model may, must, or must not call tools. OpenAI-protocol providers receive it unchanged; on Anthropic connections INFRO maps each value to Anthropic's equivalent (`auto`, `none`, `any`, a named `tool`). A `tool_choice` sent without `tools` is dropped on Anthropic connections.

| Value | Behavior |
| --- | --- |
| `"auto"` | The model decides whether to call a tool or answer directly. Default whenever `tools` is present. |
| `"none"` | Tools are never called. Definitions still count toward prompt tokens. |
| `"required"` | The model must call at least one tool before answering. |
| `{"type": "function", "function": {"name": "get_weather"}}` | Forces a call to the named tool. |

Forcing a tool guarantees a call, not good arguments — a model pushed into a tool it wouldn't have chosen will invent parameters it doesn't have. Validate arguments before executing, especially with `"required"` or a forced function.

## Parallel tool calls

Where the underlying model supports it, one assistant turn can carry several entries in `tool_calls` — independent lookups batched into a single round trip. Execute them all (concurrently if you like) and append one `"tool"` message per call. Order doesn't matter; results are matched by `tool_call_id`, not position. The loop above already handles this — its `for` loop over `message.tool_calls` returns every result before the next request.

> **Warning:** Return a `"tool"` message for every `tool_call_id` in the turn before requesting the next completion. A missing result makes the conversation invalid, and most providers reject it with a 400 — see [error handling](https://infro.io/docs/api/errors). If a tool fails, return the error text as the result rather than omitting it; models recover well from described failures.

## Streaming tool calls

With `stream: true`, tool calls arrive incrementally in `delta.tool_calls`. Fragments are keyed by `index` within the turn — the first fragment of each call carries the `id`, `type`, and `function.name`; subsequent fragments append pieces of the `function.arguments` string. Concatenate arguments per index until the chunk with `finish_reason` `"tool_calls"`, then parse.

```text
data: {"choices":[{"index":0,"delta":{"tool_calls":[{"index":0,"id":"call_9f2b","type":"function","function":{"name":"get_weather","arguments":""}}]},"finish_reason":null}]}

data: {"choices":[{"index":0,"delta":{"tool_calls":[{"index":0,"function":{"arguments":"{\"city\":\"Lis"}}]},"finish_reason":null}]}

data: {"choices":[{"index":0,"delta":{"tool_calls":[{"index":0,"function":{"arguments":"bon\"}"}}]},"finish_reason":null}]}

data: {"choices":[{"index":0,"delta":{},"finish_reason":"tool_calls"}]}

data: [DONE]
```

Parallel calls interleave in the same stream, distinguished by `index`. The SSE format, usage accounting via `stream_options`, and mid-stream failure behavior are covered in [streaming](https://infro.io/docs/api/streaming).

## Which models to use

Tool-use quality varies more across models than plain-text quality does, and it shifts between versions — benchmark on your own tools before committing. Frontier models such as `anthropic/claude-sonnet-5`, `anthropic/claude-opus-5` and `openai/gpt-5.6-sol` are the usual starting point for long multi-step chains; open-weight models such as `moonshot/kimi-k2.6` are worth including in the comparison.

For high-volume, single-call workloads — one lookup, one answer — a smaller model such as `openai/gpt-5.6-luna` or `google/gemini-3.7-flash` is often enough; compare the vendors' published rates and the estimated cost INFRO records per request. And if what you need is validated JSON output rather than executed actions, use [structured outputs](https://infro.io/docs/features/structured-outputs) instead: tool schemas describe actions, `response_format` describes shapes.