Skip to content

Agents and multimodal systemsComparison

Which agent work can wait for OpenAI's lower-rate background windows?

OpenAI's slower processing options trade time for a lower rate, but an agent job that fails halfway can cost more than it would have at the standard rate. This comparison covers Flex, Batch and the standard tier on latency, retries and cost per finished job, then shows how to route only eligible work into the slow lane.

By INFRO Editorial

Published

7 min read

A glowing autonomous agent core coordinating tools, memory, and multiple AI model routes

In short

Work that nobody waits on, that can be restarted or resumed, and whose steps are short enough to survive a 429 can go into a lower-rate window. Flex needs no rewrite and Batch needs one; cost per finished job, not the rate, decides which. 1

Key takeaways

  • Flex bills at Batch rates on ordinary Responses requests, so it needs no file-based rewrite, but it can return 429 Resource Unavailable and OpenAI suggests a 15-minute client timeout.
  • Batch holds a 24-hour window and cannot run a loop that calls a tool, reads the result and decides again, so multi-step agents fit Flex or a per-step batch, not one Batch request.
  • Compare options by expected cost per finished job, including wasted steps when a slow-tier request is refused midway, not by the per-token rate.
  • Make work eligible by checking four properties: nobody waits, each step is idempotent or resumable, the job has a deadline you can miss safely, and a fallback to standard is allowed.
  • OpenAI's newer background-window offer could not be read for this article, so its rates and window lengths are not stated here; plug them into the same formula once you have read the page yourself.

What this comparison covers, and what it does not

The question is which agent and multimodal work can wait for a lower rate without a rewrite. One caveat first: OpenAI's pricing page and its agents announcement could not be read when this was written (both returned HTTP 403). So this article does not state rates, window lengths or eligibility rules for any newer background completion window. Everything about OpenAI below is what is documented for Flex and Batch, as summarised in INFRO's reads of OpenAI's pages. Treat a new window as a third column in the table below, and fill it from OpenAI's own page before you route anything to it.

The method does not depend on that column. Any slower window has three properties you must measure: how long it can take, what happens when it cannot serve you, and what a job costs once failures are included.

Flex, Batch and standard side by side

Flex bills at Batch rates for Responses or Chat Completions requests, in exchange for slower responses and occasional resource unavailability. It is in beta with limited model availability and returns 429 Resource Unavailable, without charge, when there is no capacity.1 Batch is a file-based API with a 24-hour window and a separate rate-limit pool.2

StandardFlexBatch API
RateList rateBatch rate (half)Half the synchronous rate
How you send itNormal requestNormal request with service_tier: "flex"JSONL file, then poll
LatencySecondsSlower; OpenAI suggests a 15-minute client timeoutUp to 24 hours, often sooner
When capacity is shortNormal 429 handling429 Resource Unavailable, not chargedUnfinished requests expire at 24 hours; completed ones are billed
Tool-calling loopsYesYes, step by stepNot inside one request
Code changeNoneA field and a longer timeoutA new submit, poll and match pipeline
Rate and behaviour as documented for GPT-6 Sol class models on 2026-10-27 and 2026-11-16 reads.12

On GPT-6 Sol, Flex is $1.00 input and $5.00 output per million tokens against $2.00 and $10.00 standard, up to 272K input tokens.1 The rate card lists Batch and Flex at half the standard rate across models, for example GPT-6 Astra on Batch at $5 input and $25 output.3 These are OpenAI's list rates, which your own contract may change.

Latency: what "background" really means for an agent

An agent run is a chain of model calls. A window that adds minutes to each call adds minutes times the number of steps to the job. A 12-step run with a 2-minute worst-case step is a 24-minute job, and the tail is set by the slowest step, not the average one.

Batch is different in kind. Its window applies to a whole request, and OpenAI's Batch API completes within 24 hours, often sooner.2 That suits a stage that runs the same prompt over many items. It does not suit the loop, because each tool result has to come back before the next request exists. The practical split:

  • Flex for the loop itself: each step is an ordinary request, so state, tools and the Responses API shape stay as they are.
  • Batch for the fan-out stages around the loop: classify 20,000 documents, caption 5,000 images, grade last night's runs.
  • Standard for any step a person is waiting on, and for the final step if a deadline is attached to it.

OpenAI also advises against running large ETL or batch jobs in Fast mode, which is a sign that each tier has an intended traffic shape.1 The same applies in the other direction.

Retries: where a slow lane turns into a bill

A refused Flex request is not charged, but the steps before it were. OpenAI suggests either backing off or retrying with service_tier: "auto" when finishing matters more than price.1 Which one you pick decides whether a job costs more than standard would have.

Batch has a different failure shape. Nothing times out in front of a user; requests expire or error individually, results may come back in any order, and you match them on custom_id.2 Expiry is normal operation, so the resubmission path has to exist before the first large run.

Three rules keep retries from eating the rate gap:

  1. Make steps idempotent or resumable. If a run is restarted from scratch after a refusal at step 8 of 12, you pay for 8 steps twice.
  2. Cap Flex attempts per step, then fall back to standard for the remaining steps rather than the whole job.
  3. Back off, don't hammer. A gateway route that retries the same target retries immediately, with no backoff,4 which is the wrong behaviour for a capacity refusal. Set the failure rule for that kind of failure explicitly.

Cost per finished job

Compare options by what a completed job costs, failures included:

expected cost = (1 − p) × slow_cost + p × (slow_cost_wasted + fallback_cost)

where p is the share of jobs that hit a refusal and fall back.

The lesson is about design more than price: resumability matters more than the headline rate. Whatever a newer background window lists, put its rate and refusal behaviour into the same formula.

An eligibility rule you can encode

Work is eligible for a slow lane only if all four hold:

  • Nobody, human or live process, waits on the answer inside the window.
  • Each step is idempotent, or the run can resume from its last completed step.
  • A missed deadline is an acceptable outcome, or a standard-rate fallback is allowed by budget.
  • The model and features you use are available on the slow lane (Flex has limited model availability).1

Multimodal work needs one more check: INFRO's async jobs API today covers video generation, upscale and lip-sync, while images and audio are synchronous.5 A long media render already has its own job lifecycle, so decide per modality whether a slow text window adds anything.

eligible.py
from dataclasses import dataclass

@dataclass
class Job:
    user_waiting: bool
    resumable: bool
    deadline_minutes: int | None
    standard_fallback_allowed: bool
    model_on_flex: bool

def lane(job: Job) -> str:
    if job.user_waiting or not job.model_on_flex:
        return "standard"
    if not job.resumable and not job.standard_fallback_allowed:
        return "standard"
    if job.deadline_minutes is not None and job.deadline_minutes < 30:
        return "standard"
    return "background"

Routing only eligible work

Keep the decision in one place and let configuration carry it. In an INFRO route, a name your application sends as model resolves to your own provider connections, and up to 20 rules can send matching requests to their own targets, checked in order with the first match deciding.6 Rules can read metadata, the workflow and step attribution values, has_tools, input_tokens and the hour in UTC.6 So an application tags eligible work and a rule sends it to the background target:

JSON
{
  "name": "Background agent work",
  "match": "all",
  "conditions": [
    { "field": "metadata", "key": "lane", "op": "in", "value": ["background"] }
  ],
  "targets": [{ "connection_id": "conn_…", "model": "openai/gpt-6-sol" }],
  "fallthrough": true
}

The condition follows the shape in the routes docs; confirm exact operators there before you copy it. Two cautions. First, whether a request field such as service_tier reaches OpenAI is a question for the endpoint reference, so check it before relying on it; a design that works either way keeps background traffic on its own route and its own OpenAI project, where OpenAI lets you choose which service tiers each project may use.1 Second, any upstream failure before the response starts moves a request to the next target, and a route's failure rules can change that per kind of failure: retry, move on or stop.4 Set the rate-limit kind to move to the standard target, not to retry in place.

Traffic that goes through INFRO records the model, latency and estimated provider cost per request, so you can sum cost per finished job by trace. Costs are estimates against your own rates, not a bill; your providers invoice you directly. Batch files submitted straight to OpenAI do not pass through a gateway, so give them their own provider project to attribute them.2 OpenAI's Responses API through INFRO is covered by the Claude Code and Codex ingress, with routing and budget checks applied as on any other request.

A rollout order

  1. Pick one eligible workflow and tag it with lane: background in metadata.
  2. Run it on Flex for a week alongside a standard baseline, recording refusals, step latency and finished-job cost.
  3. Add resume-on-standard before widening the share.
  4. Move fan-out stages to Batch only when a pipeline that matches custom_id and resubmits expired items is in place.
  5. When OpenAI's newer background windows are readable and available for your models, add them as a third lane with the same measurements and the same fallback.

Questions readers ask

Can I put a tool-calling agent into the Batch API?
Not as one request. A loop that calls a tool, reads the result and decides again cannot live inside a single batch request; run it synchronously or on Flex, or restructure it so each step over many items is its own batch. 2
Does a refused Flex request cost anything?
OpenAI's Flex documentation says a 429 Resource Unavailable is not charged. The cost to watch is the earlier steps of the same job that already ran and were billed. 1

From the team that builds it

Where this shows up in INFRO

INFRO is a control plane in front of the provider accounts you already hold. The capabilities below are the parts of it this article touches, each with the documentation that defines its behaviour.

  • Health-aware routing

    Available with a plan

    Named routes use priority or weighted targets over your own connections, with health checks, retries and failover.

    Read the documentation for Health-aware routing
  • Routing rules

    Available with a plan

    If-rules on a route send matching requests to their own targets, by metadata, feature, end user, key, project, prompt size, streaming, images, tools or time of day. Failure rules decide per failure kind whether to retry, move on or stop.

    Read the documentation for Routing rules
  • Automatic failover

    Available with a plan

    A route that times out or 5xxs before the response starts is retried elsewhere without your client seeing it.

    Read the documentation for Automatic failover
  • Usage and cost per request

    Available with a plan

    Every request records its model, latency, token or unit counts, and estimated provider cost, queryable by project and key.

    Read the documentation for Usage and cost per request
  • Request traces and payload logs

    Available with a plan

    Per-request timing detail, with prompt and completion logging off by default and a retention window you choose.

    Read the documentation for Request traces and payload logs
  • Claude Code and Codex through INFRO

    Available with a plan

    Point Claude Code, the Claude Agent SDK, Codex or the OpenAI Agents SDK at INFRO by changing the base URL and the key. Anthropic's Messages API and OpenAI's Responses API, with routing, budgets refused before the call, guardrails and the request log applied as on every other request.

  • Negotiated provider rates

    Available with a plan

    Enter the rates your contracts actually carry, per model and optionally per provider, so estimated cost uses them before the vendor's list rate. Included on every plan.

    Read the documentation for Negotiated provider rates

INFRO accounts are self-serve. Provider usage is billed to you by your providers; INFRO charges a software subscription and takes 0% of model spend. See plans.

Sources

Numbered in order of first citation. Each was opened and checked against the claim on the date shown.

  1. 1.LLM service tiers compared: Flex, Fast, Ultrafast and Priority (opens in a new tab) — INFRO. Accessed Nov 19, 2026.
  2. 2.Batch or synchronous: which LLM workloads should move (opens in a new tab) — INFRO. Accessed Nov 19, 2026.
  3. 3.OpenAI API pricing in 2026: GPT-6 tiers, Batch, Flex and Fast (opens in a new tab) — INFRO. Accessed Nov 19, 2026.
  4. 4.Smart routing (opens in a new tab) — INFRO Docs. Accessed Nov 19, 2026.
  5. 5.Async jobs (opens in a new tab) — INFRO Docs. Accessed Nov 19, 2026.
  6. 6.Routes (opens in a new tab) — INFRO Docs. Accessed Nov 19, 2026.

About this article

Read on 2026-11-19: INFRO's OpenAI pricing, service-tier and Batch articles (which cite OpenAI's own pages) and INFRO's routes, routing and async-jobs docs. OpenAI's pricing page and agents announcement returned HTTP 403, so the newer reduced-rate background windows, their rates and window lengths are left out as unconfirmed.

Last checked against its sources on . The next review is due by . No separate named reviewer is recorded for this article.

How articles are written, sourced and corrected is set out in the editorial standards.