Skip to content

Architecture and migrationAnalysis

Provider-held threads or your own history: what a gateway owns

Anthropic's message threads, in beta since November 2, let the API hold a conversation so each turn sends only new messages. For an architect, the real question is which copy of the history is the source of truth, and what that means for failover, retention, cost records and exit.

By INFRO Editorial

Published

8 min read

One application request entering a central AI gateway and branching to many model routes

In short

Treat provider-held threads as a short-lived cache, not as storage. Anthropic's own documentation says your application stays the source of truth, so keep the full history in a store you control, ideally at the gateway, and use the thread only to shrink requests.2

Key takeaways

  • Message threads are a best-effort server-side cache: they expire 5 minutes after the last request and are deleted within 1 hour, and a continue can return 404.2
  • Anthropic tells you to keep your own copy of every message and build the replay path from the start, so the choice is where that copy lives, not whether to have one.
  • A thread pins the model and several other fields, so a cross-model fallback has to start a fresh thread from the full history.
  • Threads are not ZDR eligible and not HIPAA eligible, which can rule them out for some workloads before cost or latency come into it.
  • A gateway that sees only the new messages must read token counts from the response, because the request body no longer shows the conversation.

What changed on November 2

Anthropic added message threads to the Messages API, in beta behind the message-threads-2026-08-12 header. The API keeps the conversation on the server, so each later request sends only the new messages.1 You send the full conversation, system prompt and tools once with "thread": {"type": "create"}. After that you send the new messages and the id of the previous response with "thread": {"type": "continue", ...}.2

The benefit is smaller requests and a steadier time to first token in long agent loops. Without a thread, every turn uploads the whole conversation again and the API validates and tokenizes all of it again.2 That is a real gain for a coding agent with a long tool trace.

The architectural question is different. If the provider holds conversation state, where does the authoritative copy live, and what does that do to failover, retention, cost records and your ability to leave? The rest of this piece works through those four, then gives a recommendation.

The thread is a cache, by Anthropic's own description

Read the lifetime rules before the feature list. A thread expires 5 minutes after its last request, and the API deletes it no more than 1 hour after last use. If the API can no longer continue a thread, the continue request returns a 404, and you must send the full conversation again in a new create request.2 No response tells you whether a turn was stored; you learn that on the next continue.2

The documentation is direct about the consequence: your application stays the source of truth, you should keep your own copy of every message, and you should build the replay path from the start.2 The SDK helper keeps that copy in memory only, so if the conversation has to outlive the process, you need your own store.2

So "adopt provider-held state or keep history in your own store" is a false choice. Under Anthropic's own guidance you always keep your own store. What you decide is whether to add the thread as an optimization on top, and which component owns the store: each application, or a shared layer such as a gateway.

The four designs compared

Three designs are realistic. In the table, "app store" means each service keeps its own history, and "gateway store" means a shared layer in the request path holds it.

ConcernThread only (not advisable)App store + threadGateway store + thread
Failover to another providerNothing to replay from; the id is meaningless elsewhereEach app must rebuild and resend historyGateway rebuilds the full request for the next target
RetentionProvider memory, up to 1 hour; not ZDR eligible2Your database, your policy, per serviceOne policy, one deletion path
Cost visibilityRequest body shows only the deltaSame, per serviceGateway records usage from each response
Lock-inRequest shape tied to threadStored history in provider shapeStored history in one place, convertible once
How the options differ on the four concerns. "Thread only" is included to show why it fails; the docs advise against it.

The sections below explain each row.

Failover: the thread does not travel

A thread fixes model, tool_choice, thinking, context_management, output_config and the set of anthropic-beta values. Sending a different value on a later turn returns a 400 with the code thread_fingerprint_mismatch.2 system and tools are create-only, and sending either on a continue request is a 400.2

That has direct consequences for a gateway:

  • A model fallback cannot be a continue request with a different model. It must be a fresh create, or a plain request, carrying the full history.
  • A move to another provider has no thread at all. The history has to come from somewhere other than the provider.
  • Even a retry on the same model after a timeout is uncertain, since you do not know whether the turn was stored until the next continue.

INFRO's failover works before the first token: connection failover tries the same model on another of your connections, and model fallbacks try an ordered list of other models.3 Either way the next target receives a complete request. With a gateway-held history, building that request is a lookup. Without one, the gateway holds only the delta and cannot build it.

The pattern is simple to state:

continue_or_replay.py
def send_turn(conv, new_messages):
    conv.history.extend(new_messages)          # your store is authoritative
    try:
        if conv.last_response_id:
            resp = call_continue(conv, new_messages)
        else:
            resp = call_create(conv, conv.history)
    except NotFound:                            # thread expired or deleted
        resp = call_create(conv, conv.history)  # replay the full history
    except TargetUnavailable:
        resp = call_create(next_target(conv), conv.history)  # fresh start elsewhere
    conv.history.append(resp.message)
    conv.last_response_id = resp.id
    return resp

The point is the order: append to your store first, then treat the thread as a shortcut that may fail.

Retention: a short window, with an eligibility catch

On retention, the docs say a thread's contents (conversation, system prompt, tools) are held in memory, encrypted under a server-managed key, never written to durable storage, and deleted within an hour of last use.2 The same page lists the feature as not ZDR eligible and not HIPAA eligible.2 Using threads does not affect the ZDR eligibility of other features, but a workload that must stay inside a ZDR agreement should not enable the beta without confirming it with the provider. The existing article on zero data retention across providers covers what to verify.

The other half of the retention story is your own store, which now holds the full conversation for as long as you decide. That is a data-protection obligation you carry, but also a control you have: one deletion path, one retention clock, one place to answer an access request. A gateway store concentrates that. A per-app store multiplies it by the number of services.

Separately, a gateway that logs payloads will see less than before. With payload logging on, a continue request records only the new messages, so the log no longer shows what the model read. INFRO keeps prompt and response capture off by default and lets the organization choose a retention window within its plan's limit.4 If you rely on those logs for audit, decide whether the history store, not the request log, is the record of what the model saw.

Cost visibility: read usage from the response

A thread shrinks the request, not necessarily the model's input. The API rebuilds the conversation before it calls the model.2 The threads page I read does not state how input tokens on continue turns are billed, so I am not asserting it here. Verify it against the usage object and your provider's own cost report before you forecast.

What does change is how a gateway can see cost. Any estimator that sizes the request body, or counts tokens on the request, will undercount a continue turn, because the body holds only the delta. The reliable source is the usage the provider returns with each response. INFRO's documentation describes recording what each request used as the provider reports it, and estimating the provider bill from that.5 Two checks are worth adding:

  1. Compare estimated spend on thread traffic against the provider's cost report for the same period, and investigate a persistent gap.
  2. Track cache read and write tokens separately if you set the top-level cache_control, since the page notes that prompt caching on a thread is a separate option and nothing is cached on later turns unless you ask for it or add breakpoints.2

Failover adds a cost wrinkle: the first turn on a new target is a full-history create, which pays for the whole input again. See what failover does to your bill.

Lock-in: small in code, larger in stored data

The thread parameter is one field on a request, so the code-level coupling is small, and the SDK helper hides it. The beta header must be sent on every request of a thread, and the helper adds it for you.2 The risk is a design that lets application code depend on the response id as a conversation id.

The larger coupling is what you store. Anthropic-shaped history includes thinking blocks with signatures, cache breakpoints and server-tool blocks. INFRO's agent documentation notes that on an Anthropic connection these reach the provider as written.6 Store them verbatim, then convert at the edge when you fail over to a model reached through another API shape. Converting at write time loses data you may need to continue on Anthropic later.

A practical rule: use your own conversation id as the key, keep the provider response id as a disposable pointer next to it, and never expose the pointer to clients.

Where a gateway should draw the line

Put these in the gateway, because they need the full history and a view across providers:

  • The authoritative history, keyed by your conversation id, written before the provider call.
  • The decision to continue, replay or start fresh on another target.
  • Usage records read from each response, attributed to the conversation.
  • The retention and deletion policy for stored history.

Leave these to the provider or the application: the thread id itself, the 5-minute timer, and anything that depends on a model's own fields.

INFRO's /v1/messages endpoint forwards Anthropic's anthropic-beta header on an Anthropic connection.6 The pages I read do not mention the thread field or a gateway-held conversation store, so test thread requests through your gateway end to end before depending on them, and do not assume a history store exists in any product unless its documentation says so.

A decision checklist

  • Is the workload long-running with large histories, where request size and time to first token matter? If not, skip threads.
  • Is it covered by a zero-data-retention or HIPAA requirement? Threads are not eligible for either.2
  • Do you already store full history somewhere you control? If not, build that first.
  • Does your fallback plan cross models or providers? Then every fallback is a full-history create.
  • Can your cost tooling read usage from responses rather than sizing requests?
  • Is the feature acceptable as a beta, with the header and behavior open to change?

If you answer yes to the first and the last, and no to the second, adopt threads behind a gateway-held history and treat every 404 as routine.

Questions readers ask

Does a message thread survive a failover to another provider?
No. A thread is held by the API that created it, so a different provider has nothing to continue. Send the full history in a new request to the fallback target and start a new thread there if it supports one.
Can I use message threads with zero data retention?
Anthropic lists threads as not ZDR eligible and not HIPAA eligible. Its docs say the contents are held in memory, encrypted, and deleted within an hour, but eligibility is stated as not eligible, so check with your own agreement before enabling it.
Do I still need to keep conversation history if I use threads?
Yes. Anthropic says your application stays the source of truth and that you should build the replay path from the start, because a thread is best-effort and a continue request can return 404.

From the team that builds it

Where this shows up in INFRO

INFRO is a control plane in front of the provider accounts you already hold. The capabilities below are the parts of it this article touches, each with the documentation that defines its behaviour.

  • Automatic failover

    Available with a plan

    A route that times out or 5xxs before the response starts is retried elsewhere without your client seeing it.

    Read the documentation for Automatic failover
  • Usage and cost per request

    Available with a plan

    Every request records its model, latency, token or unit counts, and estimated provider cost, queryable by project and key.

    Read the documentation for Usage and cost per request
  • Request traces and payload logs

    Available with a plan

    Per-request timing detail, with prompt and completion logging off by default and a retention window you choose.

    Read the documentation for Request traces and payload logs
  • Claude Code and Codex through INFRO

    Available with a plan

    Point Claude Code, the Claude Agent SDK, Codex or the OpenAI Agents SDK at INFRO by changing the base URL and the key. Anthropic's Messages API and OpenAI's Responses API, with routing, budgets refused before the call, guardrails and the request log applied as on every other request.

INFRO accounts are self-serve. Provider usage is billed to you by your providers; INFRO charges a software subscription and takes 0% of model spend. See plans.

Sources

Numbered in order of first citation. Each was opened and checked against the claim on the date shown.

  1. 1.Claude Platform release notes (opens in a new tab) — Anthropic (Claude Platform Docs). Accessed Nov 7, 2026.
  2. 2.Message threads (opens in a new tab) — Anthropic (Claude Platform Docs). Accessed Nov 7, 2026.
  3. 3.Failover & fallbacks (opens in a new tab) — INFRO Docs. Accessed Nov 7, 2026.
  4. 4.Privacy & data (opens in a new tab) — INFRO Docs. Accessed Nov 7, 2026.
  5. 5.INFRO documentation (opens in a new tab) — INFRO Docs. Accessed Nov 7, 2026.
  6. 6.Claude Code & Codex (opens in a new tab) — INFRO Docs. Accessed Nov 7, 2026.

About this article

Read on 2026-11-07: Anthropic's message threads page, the Claude Platform release notes, and INFRO's docs overview, failover, privacy and coding-agents pages. The threads page was truncated before its fallback and recovery sections, and billing treatment of continue turns was not visible, so both are left as open questions rather than stated.

Last checked against its sources on . The next review is due by . No separate named reviewer is recorded for this article.

How articles are written, sourced and corrected is set out in the editorial standards.