# Bounding AI agent loops: step, time, token and spend limits

URL: https://infro.io/blog/bound-ai-agent-loops
Author: INFRO Editorial (https://infro.io/blog/editorial)
Publisher: INFRO
Published: 2026-09-29
Last modified: 2026-09-29
Last reviewed: 2026-09-29 (next review due 2026-10-29)
Topic: Agents and multimodal systems
Format: Guide

> Bound every agent run in two places: a guard inside the loop that stops on step count, wall-clock deadline, token total, estimated spend and repeated tool calls, and hard ceilings outside the process — per-key, per-project and per-end-customer spend limits at the gateway, plus provider quotas — that hold even when the guard has a bug.

Key takeaways:
- An agent's input grows with every step because the history is re-sent, so cost per run grows faster than step count.
- A run guard checks steps, a wall-clock deadline, tokens, estimated spend and repeated tool calls before every model call, not after.
- Set max_tokens on each call from what the run has left; the model's maximum is not a budget.
- In-process limits fail with the process. Put hard ceilings where a bug cannot skip them: gateway keys, projects, end customers and provider quotas.
- A spend refusal is terminal. Treat a 402 as the end of the run, not as a transient error to retry.

A chat request is one model call. An agent run is a loop: the model asks for a tool, your code runs it, the result goes back, and the model decides what to do next. Nothing in that loop stops by itself. A model that misreads a tool error, a tool that returns the same page forever, or a planner that keeps "double-checking" will run until something outside it says no.

OWASP files this risk as *unbounded consumption*, and one of its named examples is usage that runs up a pay-per-use bill until it becomes unsustainable[1]. Its mitigations are the ones this guide implements — rate limits and quotas, timeouts and throttling for expensive operations, and monitoring for unusual consumption[1].

## Why agent cost grows faster than step count

Each model call in an agent run re-sends everything the model needs: the system prompt, the tool definitions, and the whole conversation so far, including every tool result. Tool definitions alone are billed as input tokens on every call; OpenAI says function definitions are injected into the system message and "billed as input tokens"[2], and Anthropic adds a tool-use system prompt on top of your tool schemas[3].

So step 20 does not cost what step 1 cost. It costs step 1 plus everything the run has accumulated. Total input over a run grows roughly with the square of the step count.

[Diagram: Illustrative: a 3,500-token fixed prefix and 1,200 tokens of new history per step. The prefix is paid on every call; the history is paid again on every later call.]

> **Example: Twenty steps versus five**
> Take a fixed prefix of 3,500 tokens (system prompt and tool definitions), 1,200 tokens of new history per step, and 400 output tokens per step, at illustrative rates of $2 per million input tokens and $10 per million output tokens.
> 
> A 5-step run sends 5 × 3,500 + 1,200 × (0 + 1 + 2 + 3 + 4) = 29,500 input tokens and generates 2,000 output tokens: about $0.08.
> 
> A 20-step run sends 20 × 3,500 + 1,200 × (0 + 1 + … + 19) = 298,000 input tokens and generates 8,000: about $0.68.
> 
> Four times the steps, roughly eight and a half times the cost. Prompt caching lowers the rate on the repeated prefix but does not change the shape; see [prompt caching](https://infro.io/docs/features/prompt-caching).

That shape is why a step limit alone is a weak control. A run with a modest step count and one tool that returns 40,000 tokens of HTML can cost more than a long run of small steps. Bound the resources, not only the iterations.

## The five limits a run needs

Agent frameworks already expose the first one. Anthropic's guidance on agents recommends stopping conditions "such as a maximum number of iterations" and warns that autonomy brings "higher costs, and the potential for compounding errors"[4]. The OpenAI Agents SDK raises `MaxTurnsExceeded` when a run passes the `max_turns` given to the runner[5]. Those are step limits. A production run needs four more.

| Limit | What it stops | Checked when |
| --- | --- | --- |
| Steps | Planners that never converge | Before each model call |
| Wall-clock deadline | Slow tools, retries and provider stalls holding a user or worker | Before each model call and each tool call |
| Tokens | Oversized tool results and history growth | After each response, from provider-reported usage |
| Estimated spend | Expensive models and long outputs | Before each call, as a worst case; after, as recorded |
| Repeated calls | The same tool with the same arguments, over and over | Before executing a tool call |

Table: Run limits and when each is checked. Check before the next call, so a run never starts work it is not allowed to finish.

[Diagram: The guard runs at the top of every iteration. The loop ends on a final answer, on any limit, on a repeated call, or on a refusal from the gateway.]

## A run guard in Python

The guard below is framework-neutral: call `before_model_call()` before each request, `after_model_call()` with the provider's reported usage, and `before_tool_call()` before executing anything the model asked for. The spend check uses a worst case — input tokens you are about to send plus the `max_tokens` you are about to allow — so a single call cannot jump past the limit.

```python
import hashlib
import json
import time
from dataclasses import dataclass, field


class RunLimitReached(Exception):
    def __init__(self, reason: str, detail: str):
        super().__init__(f"{reason}: {detail}")
        self.reason = reason


@dataclass
class RunLimits:
    max_steps: int = 25
    deadline_seconds: float = 120.0
    max_total_tokens: int = 400_000
    max_spend_usd: float = 1.00
    max_identical_tool_calls: int = 2
    min_output_tokens: int = 256  # stop rather than ask for a useless reply


@dataclass
class RunGuard:
    limits: RunLimits
    input_usd_per_token: float
    output_usd_per_token: float
    started: float = field(default_factory=time.monotonic)
    steps: int = 0
    tokens: int = 0
    spend_usd: float = 0.0
    seen_calls: dict[str, int] = field(default_factory=dict)

    def remaining_seconds(self) -> float:
        return self.limits.deadline_seconds - (time.monotonic() - self.started)

    def before_model_call(self, input_tokens_estimate: int, wanted_output: int) -> int:
        """Return the max_tokens this call may use, or raise."""
        if self.steps >= self.limits.max_steps:
            raise RunLimitReached("steps", f"{self.steps} steps used")
        if self.remaining_seconds() <= 0:
            raise RunLimitReached("deadline", "wall-clock deadline passed")
        token_room = self.limits.max_total_tokens - self.tokens - input_tokens_estimate
        spend_room = self.limits.max_spend_usd - self.spend_usd
        spend_room -= input_tokens_estimate * self.input_usd_per_token
        output_by_spend = int(spend_room / self.output_usd_per_token)
        max_tokens = min(wanted_output, token_room, output_by_spend)
        if max_tokens < self.limits.min_output_tokens:
            raise RunLimitReached("budget", f"only {max(max_tokens, 0)} output tokens left")
        self.steps += 1
        return max_tokens

    def after_model_call(self, prompt_tokens: int, completion_tokens: int, cost_usd: float | None):
        self.tokens += prompt_tokens + completion_tokens
        if cost_usd is None:  # unpriced: fall back to your own rates, never to zero
            cost_usd = (prompt_tokens * self.input_usd_per_token
                        + completion_tokens * self.output_usd_per_token)
        self.spend_usd += cost_usd

    def before_tool_call(self, name: str, arguments: str) -> float:
        """Return the timeout for this tool call, or raise on a repeat."""
        try:
            canonical = json.dumps(json.loads(arguments), sort_keys=True)
        except json.JSONDecodeError:
            canonical = arguments
        key = hashlib.sha256(f"{name}:{canonical}".encode()).hexdigest()
        self.seen_calls[key] = self.seen_calls.get(key, 0) + 1
        if self.seen_calls[key] > self.limits.max_identical_tool_calls:
            raise RunLimitReached("repeat", f"{name} called again with identical arguments")
        remaining = self.remaining_seconds()
        if remaining <= 1:
            raise RunLimitReached("deadline", "no time left for a tool call")
        return min(30.0, remaining)
```

Three details matter more than they look:

- **`max_tokens` comes from the guard.** Asking for the model's maximum output on every step reserves, and can spend, far more than the run has left. The guard hands each call only what fits.
- **Unpriced is not free.** When a response carries no cost estimate, the guard charges your own rate rather than zero, so an unpriced model cannot run unbounded.
- **Repeat detection canonicalises arguments.** `{"id": 7, "page": 1}` and `{"page": 1, "id": 7}` are the same call. Two identical calls can be legitimate (a retry after a transient error); a third is a loop.

## The loop that uses it

The loop is ordinary tool calling; the only additions are the guard calls, a timeout around each tool, and treating limit errors as a normal way for a run to end. It uses the OpenAI Python SDK against any OpenAI-compatible endpoint.

```python
import concurrent.futures
import json

from openai import OpenAI, APIStatusError

from run_guard import RunGuard, RunLimitReached, RunLimits

client = OpenAI()  # base_url and api_key from the environment
pool = concurrent.futures.ThreadPoolExecutor(max_workers=4)
TERMINAL_STATUSES = {400, 401, 402, 403, 404}


def run_agent(task: str, tools: list[dict], handlers: dict, model: str, run_id: str) -> dict:
    guard = RunGuard(RunLimits(), input_usd_per_token=2e-6, output_usd_per_token=10e-6)
    messages = [{"role": "system", "content": "You are a support agent."},
                {"role": "user", "content": task}]
    last_prompt_tokens = 2_000  # first-call estimate; later calls use reported usage
    try:
        while True:
            estimate = last_prompt_tokens + 1_500  # plus what the last tool result added
            max_tokens = guard.before_model_call(estimate, wanted_output=2_048)
            try:
                response = client.chat.completions.create(
                    model=model, messages=messages, tools=tools, max_tokens=max_tokens,
                    timeout=max(5.0, guard.remaining_seconds()),
                    extra_body={"metadata": {"trace_id": run_id, "workflow": "support-agent",
                                             "step": f"turn_{guard.steps}"}},
                )
            except APIStatusError as err:
                if err.status_code in TERMINAL_STATUSES:
                    return {"status": "refused", "http_status": err.status_code, "run_id": run_id}
                raise
            usage = response.usage
            cost = getattr(usage, "cost", None)  # present when the endpoint reports it
            guard.after_model_call(usage.prompt_tokens, usage.completion_tokens, cost)
            last_prompt_tokens = usage.prompt_tokens + usage.completion_tokens

            message = response.choices[0].message
            messages.append(message.model_dump(exclude_none=True))
            if not message.tool_calls:
                return {"status": "done", "answer": message.content, "run_id": run_id}

            for call in message.tool_calls:
                timeout = guard.before_tool_call(call.function.name, call.function.arguments)
                handler = handlers.get(call.function.name)
                if handler is None:
                    result = {"error": f"unknown tool {call.function.name}"}
                else:
                    future = pool.submit(handler, json.loads(call.function.arguments))
                    try:
                        result = future.result(timeout=timeout)
                    except concurrent.futures.TimeoutError:
                        result = {"error": f"{call.function.name} timed out after {timeout:.0f}s"}
                messages.append({"role": "tool", "tool_call_id": call.id,
                                 "content": json.dumps(result)[:20_000]})
    except RunLimitReached as stop:
        return {"status": "stopped", "reason": stop.reason, "steps": guard.steps,
                "spend_usd": round(guard.spend_usd, 4), "run_id": run_id}
```

A timed-out tool thread keeps running in the pool; for tools with side effects, pass a cancellation token or make the tool itself honour a deadline. The `[:20_000]` cap on tool results is crude, and deliberately so: a result that large should be summarised or paginated by the tool, and the cap stops one bad page from dominating every later step.

## In-process limits are not enough

A run guard lives in the same process as the bug it protects against. A code path that forgets to call it, a retry wrapper that restarts the loop with fresh counters, a sub-agent spawned with its own guard, or a deploy that sets the limits from a missing environment variable — each silently removes the bound. The guard is the precise control; it needs a blunt one behind it that sits outside your code.

[Diagram: Each layer catches what the one above misses. Only the lower two hold when your own code is wrong.]

The outer layers answer different questions. A per-key ceiling bounds what one deployment of one agent can spend in total. A per-end-customer cap bounds what any single user of your product can cause. Provider-side quotas are the last backstop, and you set those in each provider account.

## Setting hard ceilings for agents in INFRO

INFRO sits between your agents and the provider accounts you connect, so its limits apply to every model call regardless of which code path made it. Your providers still bill you directly; these limits act on INFRO's estimate of that provider cost.

**A key per agent, with a spend limit.** A gateway key can carry a spend limit that resets daily, weekly or monthly, or a lifetime limit that never resets; key limits always block, and every applicable blocking limit must allow the request ([spend controls](https://infro.io/docs/org/spend-controls)). A lifetime limit on a key used only by one agent worker bounds what that worker can ever spend, whatever its loop does. `GET /v1/key` reports `limit`, `usage`, `limit_remaining` and `limit_resets_at` for the calling key ([limits](https://infro.io/docs/api/limits)), so a guard can read the outer ceiling before it starts a run.

**Reservation, not after-the-fact accounting.** A blocking budget admits a request only if its worst-case estimated cost fits the room left, counting the worst case of every request still in flight. When a chat request sets no `max_tokens`, the worst case assumes the model's maximum output and INFRO sends that ceiling to the provider. Setting `max_tokens` from your guard, as above, reserves less per call, so more concurrent agent steps fit under the same budget ([spend controls](https://infro.io/docs/org/spend-controls)).

**Per end customer.** Send your user's opaque id as `metadata.user_id` and an end-customer cap limits what each of your customers can spend per day, week or month; a request over the cap returns `402` with code `end_user_budget`, plus `end_user_id`, `period` and `resets_at` so the application can tell that user when they can continue ([end-customer caps](https://infro.io/docs/org/end-customer-caps)).

**Refusals are terminal.** A budget refusal is `402 budget_exceeded` with a code naming the ceiling, such as `key_budget`, `key_lifetime_limit`, `project_budget` or `end_user_budget`. It does not clear on retry, which is why the loop above ends the run on a 402 instead of backing off.

```json
{
  "error": {
    "type": "budget_exceeded",
    "code": "key_lifetime_limit",
    "message": "Budget exceeded: … (names the key and what to raise)"
  }
}
```

**See what a run cost.** The loop tags each call with `metadata.trace_id`, `workflow` and `step`. `GET /v1/traces/{trace_id}` then returns every request in the run with estimated cost by step, and flags the total as a lower bound with `cost_is_partial` when any call was unpriced ([request traces](https://infro.io/docs/observability/traces)). Tracing a run end to end is covered in [tracing an AI agent run](https://infro.io/blog/trace-ai-agent-runs).

These estimates are not the provider's invoice. Provider-side charges INFRO cannot observe — a built-in tool surcharge, or an attempt the provider billed after a timeout — sit outside them, so keep provider quotas and billing alerts on as well.

## Checklist

- [ ] Every agent loop checks steps, a wall-clock deadline, tokens, estimated spend and repeated tool calls before each model call.
- [ ] `max_tokens` on each call is computed from the run's remaining budget.
- [ ] Every tool call has a timeout, and tool results are capped or summarised before they enter the history.
- [ ] Limit hits end the run with a labelled partial result and a run id, and they are counted and alerted on per workflow.
- [ ] 400, 401, 402, 403 and 404 responses end the run; they are never retried in a loop.
- [ ] Each agent deployment uses its own gateway key with a spend limit; user-facing agents send an end-customer id and have a cap.
- [ ] Provider account quotas and billing alerts are set as the last backstop.
- [ ] A test forces each limit — steps, deadline, tokens, spend, repeats, a 402 — and asserts the run stops cleanly.

## Questions

### How many steps should an AI agent be allowed to take?

There is no universal number. Measure the step distribution of successful runs for each workflow, set the limit a little above a high percentile of it, and alert when runs end on the limit. A run that routinely hits its step ceiling is a design problem, not a limit problem.

### Should a stopped agent return a partial answer?

Usually yes, if the partial result is labelled as incomplete. Return what the run established, the reason it stopped, and an identifier for the run so a person can inspect it. Never present a run that hit a limit as a finished task.

## Sources

1. LLM10:2025 Unbounded Consumption — OWASP GenAI Security Project. https://genai.owasp.org/llmrisk/llm102025-unbounded-consumption/ (accessed 2026-09-29)
2. Function calling — OpenAI. https://developers.openai.com/api/docs/guides/function-calling (accessed 2026-09-29)
3. Tool use with Claude — Anthropic. https://platform.claude.com/docs/en/agents-and-tools/tool-use/overview (accessed 2026-09-29)
4. Building effective agents — Anthropic. https://www.anthropic.com/engineering/building-effective-agents (accessed 2026-09-29)
5. Running agents — OpenAI Agents SDK — OpenAI. https://openai.github.io/openai-agents-python/running_agents/ (accessed 2026-09-29)