What changed on November 2
Anthropic added message threads to the Messages API, in beta behind the message-threads-2026-08-12 header. The API keeps the conversation on the server, so each later request sends only the new messages.1 You send the full conversation, system prompt and tools once with "thread": {"type": "create"}. After that you send the new messages and the id of the previous response with "thread": {"type": "continue", ...}.2
The benefit is smaller requests and a steadier time to first token in long agent loops. Without a thread, every turn uploads the whole conversation again and the API validates and tokenizes all of it again.2 That is a real gain for a coding agent with a long tool trace.
The architectural question is different. If the provider holds conversation state, where does the authoritative copy live, and what does that do to failover, retention, cost records and your ability to leave? The rest of this piece works through those four, then gives a recommendation.
The thread is a cache, by Anthropic's own description
Read the lifetime rules before the feature list. A thread expires 5 minutes after its last request, and the API deletes it no more than 1 hour after last use. If the API can no longer continue a thread, the continue request returns a 404, and you must send the full conversation again in a new create request.2 No response tells you whether a turn was stored; you learn that on the next continue.2
The documentation is direct about the consequence: your application stays the source of truth, you should keep your own copy of every message, and you should build the replay path from the start.2 The SDK helper keeps that copy in memory only, so if the conversation has to outlive the process, you need your own store.2
So "adopt provider-held state or keep history in your own store" is a false choice. Under Anthropic's own guidance you always keep your own store. What you decide is whether to add the thread as an optimization on top, and which component owns the store: each application, or a shared layer such as a gateway.
The four designs compared
Three designs are realistic. In the table, "app store" means each service keeps its own history, and "gateway store" means a shared layer in the request path holds it.
| Concern | Thread only (not advisable) | App store + thread | Gateway store + thread |
|---|---|---|---|
| Failover to another provider | Nothing to replay from; the id is meaningless elsewhere | Each app must rebuild and resend history | Gateway rebuilds the full request for the next target |
| Retention | Provider memory, up to 1 hour; not ZDR eligible2 | Your database, your policy, per service | One policy, one deletion path |
| Cost visibility | Request body shows only the delta | Same, per service | Gateway records usage from each response |
| Lock-in | Request shape tied to thread | Stored history in provider shape | Stored history in one place, convertible once |
The sections below explain each row.
Failover: the thread does not travel
A thread fixes model, tool_choice, thinking, context_management, output_config and the set of anthropic-beta values. Sending a different value on a later turn returns a 400 with the code thread_fingerprint_mismatch.2 system and tools are create-only, and sending either on a continue request is a 400.2
That has direct consequences for a gateway:
- A model fallback cannot be a continue request with a different
model. It must be a fresh create, or a plain request, carrying the full history. - A move to another provider has no thread at all. The history has to come from somewhere other than the provider.
- Even a retry on the same model after a timeout is uncertain, since you do not know whether the turn was stored until the next continue.
INFRO's failover works before the first token: connection failover tries the same model on another of your connections, and model fallbacks try an ordered list of other models.3 Either way the next target receives a complete request. With a gateway-held history, building that request is a lookup. Without one, the gateway holds only the delta and cannot build it.
The pattern is simple to state:
def send_turn(conv, new_messages):
conv.history.extend(new_messages) # your store is authoritative
try:
if conv.last_response_id:
resp = call_continue(conv, new_messages)
else:
resp = call_create(conv, conv.history)
except NotFound: # thread expired or deleted
resp = call_create(conv, conv.history) # replay the full history
except TargetUnavailable:
resp = call_create(next_target(conv), conv.history) # fresh start elsewhere
conv.history.append(resp.message)
conv.last_response_id = resp.id
return respThe point is the order: append to your store first, then treat the thread as a shortcut that may fail.
Retention: a short window, with an eligibility catch
On retention, the docs say a thread's contents (conversation, system prompt, tools) are held in memory, encrypted under a server-managed key, never written to durable storage, and deleted within an hour of last use.2 The same page lists the feature as not ZDR eligible and not HIPAA eligible.2 Using threads does not affect the ZDR eligibility of other features, but a workload that must stay inside a ZDR agreement should not enable the beta without confirming it with the provider. The existing article on zero data retention across providers covers what to verify.
The other half of the retention story is your own store, which now holds the full conversation for as long as you decide. That is a data-protection obligation you carry, but also a control you have: one deletion path, one retention clock, one place to answer an access request. A gateway store concentrates that. A per-app store multiplies it by the number of services.
Separately, a gateway that logs payloads will see less than before. With payload logging on, a continue request records only the new messages, so the log no longer shows what the model read. INFRO keeps prompt and response capture off by default and lets the organization choose a retention window within its plan's limit.4 If you rely on those logs for audit, decide whether the history store, not the request log, is the record of what the model saw.
Cost visibility: read usage from the response
A thread shrinks the request, not necessarily the model's input. The API rebuilds the conversation before it calls the model.2 The threads page I read does not state how input tokens on continue turns are billed, so I am not asserting it here. Verify it against the usage object and your provider's own cost report before you forecast.
What does change is how a gateway can see cost. Any estimator that sizes the request body, or counts tokens on the request, will undercount a continue turn, because the body holds only the delta. The reliable source is the usage the provider returns with each response. INFRO's documentation describes recording what each request used as the provider reports it, and estimating the provider bill from that.5 Two checks are worth adding:
- Compare estimated spend on thread traffic against the provider's cost report for the same period, and investigate a persistent gap.
- Track cache read and write tokens separately if you set the top-level
cache_control, since the page notes that prompt caching on a thread is a separate option and nothing is cached on later turns unless you ask for it or add breakpoints.2
Failover adds a cost wrinkle: the first turn on a new target is a full-history create, which pays for the whole input again. See what failover does to your bill.
Lock-in: small in code, larger in stored data
The thread parameter is one field on a request, so the code-level coupling is small, and the SDK helper hides it. The beta header must be sent on every request of a thread, and the helper adds it for you.2 The risk is a design that lets application code depend on the response id as a conversation id.
The larger coupling is what you store. Anthropic-shaped history includes thinking blocks with signatures, cache breakpoints and server-tool blocks. INFRO's agent documentation notes that on an Anthropic connection these reach the provider as written.6 Store them verbatim, then convert at the edge when you fail over to a model reached through another API shape. Converting at write time loses data you may need to continue on Anthropic later.
A practical rule: use your own conversation id as the key, keep the provider response id as a disposable pointer next to it, and never expose the pointer to clients.
Where a gateway should draw the line
Put these in the gateway, because they need the full history and a view across providers:
- The authoritative history, keyed by your conversation id, written before the provider call.
- The decision to continue, replay or start fresh on another target.
- Usage records read from each response, attributed to the conversation.
- The retention and deletion policy for stored history.
Leave these to the provider or the application: the thread id itself, the 5-minute timer, and anything that depends on a model's own fields.
INFRO's /v1/messages endpoint forwards Anthropic's anthropic-beta header on an Anthropic connection.6 The pages I read do not mention the thread field or a gateway-held conversation store, so test thread requests through your gateway end to end before depending on them, and do not assume a history store exists in any product unless its documentation says so.
A decision checklist
- Is the workload long-running with large histories, where request size and time to first token matter? If not, skip threads.
- Is it covered by a zero-data-retention or HIPAA requirement? Threads are not eligible for either.2
- Do you already store full history somewhere you control? If not, build that first.
- Does your fallback plan cross models or providers? Then every fallback is a full-history create.
- Can your cost tooling read usage from responses rather than sizing requests?
- Is the feature acceptable as a beta, with the header and behavior open to change?
If you answer yes to the first and the last, and no to the second, adopt threads behind a gateway-held history and treat every 404 as routine.
