# Which agent work can wait for OpenAI's lower-rate background windows?

URL: https://infro.io/blog/which-agent-work-can-wait-for-openai-s-lower-rate-background-windows
Author: INFRO Editorial (https://infro.io/blog/editorial)
Publisher: INFRO
Published: 2026-11-19
Last modified: 2026-11-19
Last reviewed: 2026-11-19 (next review due 2026-12-19)
Topic: Agents and multimodal systems
Format: Comparison

> Work that nobody waits on, that can be restarted or resumed, and whose steps are short enough to survive a 429 can go into a lower-rate window. Flex needs no rewrite and Batch needs one; cost per finished job, not the rate, decides which. [^tiers]

Key takeaways:
- Flex bills at Batch rates on ordinary Responses requests, so it needs no file-based rewrite, but it can return 429 Resource Unavailable and OpenAI suggests a 15-minute client timeout.
- Batch holds a 24-hour window and cannot run a loop that calls a tool, reads the result and decides again, so multi-step agents fit Flex or a per-step batch, not one Batch request.
- Compare options by expected cost per finished job, including wasted steps when a slow-tier request is refused midway, not by the per-token rate.
- Make work eligible by checking four properties: nobody waits, each step is idempotent or resumable, the job has a deadline you can miss safely, and a fallback to standard is allowed.
- OpenAI's newer background-window offer could not be read for this article, so its rates and window lengths are not stated here; plug them into the same formula once you have read the page yourself.

## What this comparison covers, and what it does not

The question is which agent and multimodal work can wait for a lower rate without a rewrite. One caveat first: OpenAI's pricing page and its agents announcement could not be read when this was written (both returned HTTP 403). So this article does **not** state rates, window lengths or eligibility rules for any newer background completion window. Everything about OpenAI below is what is documented for Flex and Batch, as summarised in INFRO's reads of OpenAI's pages. Treat a new window as a third column in the table below, and fill it from OpenAI's own page before you route anything to it.

The method does not depend on that column. Any slower window has three properties you must measure: how long it can take, what happens when it cannot serve you, and what a job costs once failures are included.

## Flex, Batch and standard side by side

Flex bills at Batch rates for Responses or Chat Completions requests, in exchange for slower responses and occasional resource unavailability. It is in beta with limited model availability and returns 429 Resource Unavailable, without charge, when there is no capacity.[1] Batch is a file-based API with a 24-hour window and a separate rate-limit pool.[2]

|  | Standard | Flex | Batch API |
| --- | --- | --- | --- |
| Rate | List rate | Batch rate (half) | Half the synchronous rate |
| How you send it | Normal request | Normal request with `service_tier: "flex"` | JSONL file, then poll |
| Latency | Seconds | Slower; OpenAI suggests a 15-minute client timeout | Up to 24 hours, often sooner |
| When capacity is short | Normal 429 handling | 429 Resource Unavailable, not charged | Unfinished requests expire at 24 hours; completed ones are billed |
| Tool-calling loops | Yes | Yes, step by step | Not inside one request |
| Code change | None | A field and a longer timeout | A new submit, poll and match pipeline |

Table: Rate and behaviour as documented for GPT-6 Sol class models on 2026-10-27 and 2026-11-16 reads.[1][2]

On GPT-6 Sol, Flex is $1.00 input and $5.00 output per million tokens against $2.00 and $10.00 standard, up to 272K input tokens.[1] The rate card lists Batch and Flex at half the standard rate across models, for example GPT-6 Astra on Batch at $5 input and $25 output.[3] These are OpenAI's list rates, which your own contract may change.

## Latency: what "background" really means for an agent

An agent run is a chain of model calls. A window that adds minutes to each call adds minutes times the number of steps to the job. A 12-step run with a 2-minute worst-case step is a 24-minute job, and the tail is set by the slowest step, not the average one.

Batch is different in kind. Its window applies to a whole request, and OpenAI's Batch API completes within 24 hours, often sooner.[2] That suits a stage that runs the same prompt over many items. It does not suit the loop, because each tool result has to come back before the next request exists. The practical split:

- **Flex** for the loop itself: each step is an ordinary request, so state, tools and the Responses API shape stay as they are.
- **Batch** for the fan-out stages around the loop: classify 20,000 documents, caption 5,000 images, grade last night's runs.
- **Standard** for any step a person is waiting on, and for the final step if a deadline is attached to it.

OpenAI also advises against running large ETL or batch jobs in Fast mode, which is a sign that each tier has an intended traffic shape.[1] The same applies in the other direction.

## Retries: where a slow lane turns into a bill

A refused Flex request is not charged, but the steps before it were. OpenAI suggests either backing off or retrying with `service_tier: "auto"` when finishing matters more than price.[1] Which one you pick decides whether a job costs more than standard would have.

Batch has a different failure shape. Nothing times out in front of a user; requests expire or error individually, results may come back in any order, and you match them on `custom_id`.[2] Expiry is normal operation, so the resubmission path has to exist before the first large run.

Three rules keep retries from eating the rate gap:

1. **Make steps idempotent or resumable.** If a run is restarted from scratch after a refusal at step 8 of 12, you pay for 8 steps twice.
2. **Cap Flex attempts per step**, then fall back to standard for the remaining steps rather than the whole job.
3. **Back off, don't hammer.** A gateway route that retries the same target retries immediately, with no backoff,[4] which is the wrong behaviour for a capacity refusal. Set the failure rule for that kind of failure explicitly.

## Cost per finished job

Compare options by what a completed job costs, failures included:

`expected cost = (1 − p) × slow_cost + p × (slow_cost_wasted + fallback_cost)`

where `p` is the share of jobs that hit a refusal and fall back.

> **Example: Illustrative figures, not measured**
> A job uses 200,000 input tokens and 20,000 output tokens across its steps on a model listed at $2 input and $10 output per million at standard, and $1 and $5 on Flex.
> 
> - Standard: 0.2 × $2 + 0.02 × $10 = **$0.60**
> - Flex, no failure: 0.2 × $1 + 0.02 × $5 = **$0.30**
> 
> Suppose 20% of jobs are refused after two thirds of their steps (the Flex part already spent: $0.20).
> 
> - Restart from scratch on standard: $0.20 + $0.60 = $0.80, so expected cost = 0.8 × $0.30 + 0.2 × $0.80 = **$0.40**
> - Resume the last third on standard: $0.20 + $0.20 = $0.40, so expected cost = 0.8 × $0.30 + 0.2 × $0.40 = **$0.32**
> 
> With restart-from-scratch, the Flex lane stops paying at a 60% refusal rate ($0.30 + 0.5p = $0.60). With resume, the break-even is far higher.

The lesson is about design more than price: resumability matters more than the headline rate. Whatever a newer background window lists, put its rate and refusal behaviour into the same formula.

## An eligibility rule you can encode

Work is eligible for a slow lane only if all four hold:

- [ ] Nobody, human or live process, waits on the answer inside the window.
- [ ] Each step is idempotent, or the run can resume from its last completed step.
- [ ] A missed deadline is an acceptable outcome, or a standard-rate fallback is allowed by budget.
- [ ] The model and features you use are available on the slow lane (Flex has limited model availability).[1]

Multimodal work needs one more check: INFRO's async jobs API today covers video generation, upscale and lip-sync, while images and audio are synchronous.[5] A long media render already has its own job lifecycle, so decide per modality whether a slow text window adds anything.

```python
from dataclasses import dataclass

@dataclass
class Job:
    user_waiting: bool
    resumable: bool
    deadline_minutes: int | None
    standard_fallback_allowed: bool
    model_on_flex: bool

def lane(job: Job) -> str:
    if job.user_waiting or not job.model_on_flex:
        return "standard"
    if not job.resumable and not job.standard_fallback_allowed:
        return "standard"
    if job.deadline_minutes is not None and job.deadline_minutes < 30:
        return "standard"
    return "background"
```

## Routing only eligible work

Keep the decision in one place and let configuration carry it. In an INFRO route, a name your application sends as `model` resolves to your own provider connections, and up to 20 rules can send matching requests to their own targets, checked in order with the first match deciding.[6] Rules can read metadata, the `workflow` and `step` attribution values, `has_tools`, `input_tokens` and the hour in UTC.[6] So an application tags eligible work and a rule sends it to the background target:

```json
{
  "name": "Background agent work",
  "match": "all",
  "conditions": [
    { "field": "metadata", "key": "lane", "op": "in", "value": ["background"] }
  ],
  "targets": [{ "connection_id": "conn_…", "model": "openai/gpt-6-sol" }],
  "fallthrough": true
}
```

The condition follows the shape in the routes docs; confirm exact operators there before you copy it. Two cautions. First, whether a request field such as `service_tier` reaches OpenAI is a question for the endpoint reference, so check it before relying on it; a design that works either way keeps background traffic on its own route and its own OpenAI project, where OpenAI lets you choose which service tiers each project may use.[1] Second, any upstream failure before the response starts moves a request to the next target, and a route's failure rules can change that per kind of failure: retry, move on or stop.[4] Set the rate-limit kind to move to the standard target, not to retry in place.

Traffic that goes through INFRO records the model, latency and estimated provider cost per request, so you can sum cost per finished job by trace. Costs are estimates against your own rates, not a bill; your providers invoice you directly. Batch files submitted straight to OpenAI do not pass through a gateway, so give them their own provider project to attribute them.[2] OpenAI's Responses API through INFRO is covered by the Claude Code and Codex ingress, with routing and budget checks applied as on any other request.

## A rollout order

1. Pick one eligible workflow and tag it with `lane: background` in metadata.
2. Run it on Flex for a week alongside a standard baseline, recording refusals, step latency and finished-job cost.
3. Add resume-on-standard before widening the share.
4. Move fan-out stages to Batch only when a pipeline that matches `custom_id` and resubmits expired items is in place.
5. When OpenAI's newer background windows are readable and available for your models, add them as a third lane with the same measurements and the same fallback.

## Questions

### Can I put a tool-calling agent into the Batch API?

Not as one request. A loop that calls a tool, reads the result and decides again cannot live inside a single batch request; run it synchronously or on Flex, or restructure it so each step over many items is its own batch. [2]

### Does a refused Flex request cost anything?

OpenAI's Flex documentation says a 429 Resource Unavailable is not charged. The cost to watch is the earlier steps of the same job that already ran and were billed. [1]

## Sources

1. LLM service tiers compared: Flex, Fast, Ultrafast and Priority — INFRO. https://infro.io/blog/llm-service-tiers-compared-2026 (accessed 2026-11-19)
2. Batch or synchronous: which LLM workloads should move — INFRO. https://infro.io/blog/batch-vs-synchronous-llm (accessed 2026-11-19)
3. OpenAI API pricing in 2026: GPT-6 tiers, Batch, Flex and Fast — INFRO. https://infro.io/blog/openai-api-pricing-2026 (accessed 2026-11-19)
4. Smart routing — INFRO Docs. https://infro.io/docs/features/routing (accessed 2026-11-19)
5. Async jobs — INFRO Docs. https://infro.io/docs/api/jobs (accessed 2026-11-19)
6. Routes — INFRO Docs. https://infro.io/docs/routes (accessed 2026-11-19)
