# Provider-held threads or your own history: what a gateway owns

URL: https://infro.io/blog/provider-held-threads-or-your-own-history-what-a-gateway-owns
Author: INFRO Editorial (https://infro.io/blog/editorial)
Publisher: INFRO
Published: 2026-11-07
Last modified: 2026-11-07
Last reviewed: 2026-11-07 (next review due 2026-12-07)
Topic: Architecture and migration
Format: Analysis

> Treat provider-held threads as a short-lived cache, not as storage. Anthropic's own documentation says your application stays the source of truth, so keep the full history in a store you control, ideally at the gateway, and use the thread only to shrink requests.[^threads]

Key takeaways:
- Message threads are a best-effort server-side cache: they expire 5 minutes after the last request and are deleted within 1 hour, and a continue can return 404.[2]
- Anthropic tells you to keep your own copy of every message and build the replay path from the start, so the choice is where that copy lives, not whether to have one.
- A thread pins the model and several other fields, so a cross-model fallback has to start a fresh thread from the full history.
- Threads are not ZDR eligible and not HIPAA eligible, which can rule them out for some workloads before cost or latency come into it.
- A gateway that sees only the new messages must read token counts from the response, because the request body no longer shows the conversation.

## What changed on November 2

Anthropic added message threads to the Messages API, in beta behind the `message-threads-2026-08-12` header. The API keeps the conversation on the server, so each later request sends only the new messages.[1] You send the full conversation, system prompt and tools once with `"thread": {"type": "create"}`. After that you send the new messages and the id of the previous response with `"thread": {"type": "continue", ...}`.[2]

The benefit is smaller requests and a steadier time to first token in long agent loops. Without a thread, every turn uploads the whole conversation again and the API validates and tokenizes all of it again.[2] That is a real gain for a coding agent with a long tool trace.

The architectural question is different. If the provider holds conversation state, where does the authoritative copy live, and what does that do to failover, retention, cost records and your ability to leave? The rest of this piece works through those four, then gives a recommendation.

## The thread is a cache, by Anthropic's own description

Read the lifetime rules before the feature list. A thread expires 5 minutes after its last request, and the API deletes it no more than 1 hour after last use. If the API can no longer continue a thread, the continue request returns a 404, and you must send the full conversation again in a new create request.[2] No response tells you whether a turn was stored; you learn that on the next continue.[2]

The documentation is direct about the consequence: your application stays the source of truth, you should keep your own copy of every message, and you should build the replay path from the start.[2] The SDK helper keeps that copy in memory only, so if the conversation has to outlive the process, you need your own store.[2]

So "adopt provider-held state or keep history in your own store" is a false choice. Under Anthropic's own guidance you always keep your own store. What you decide is whether to add the thread as an optimization on top, and which component owns the store: each application, or a shared layer such as a gateway.

## The four designs compared

Three designs are realistic. In the table, "app store" means each service keeps its own history, and "gateway store" means a shared layer in the request path holds it.

| Concern | Thread only (not advisable) | App store + thread | Gateway store + thread |
| --- | --- | --- | --- |
| Failover to another provider | Nothing to replay from; the id is meaningless elsewhere | Each app must rebuild and resend history | Gateway rebuilds the full request for the next target |
| Retention | Provider memory, up to 1 hour; not ZDR eligible[2] | Your database, your policy, per service | One policy, one deletion path |
| Cost visibility | Request body shows only the delta | Same, per service | Gateway records usage from each response |
| Lock-in | Request shape tied to `thread` | Stored history in provider shape | Stored history in one place, convertible once |

Table: How the options differ on the four concerns. "Thread only" is included to show why it fails; the docs advise against it.

The sections below explain each row.

## Failover: the thread does not travel

A thread fixes `model`, `tool_choice`, `thinking`, `context_management`, `output_config` and the set of `anthropic-beta` values. Sending a different value on a later turn returns a 400 with the code `thread_fingerprint_mismatch`.[2] `system` and `tools` are create-only, and sending either on a continue request is a 400.[2]

That has direct consequences for a gateway:

- A model fallback cannot be a continue request with a different `model`. It must be a fresh create, or a plain request, carrying the full history.
- A move to another provider has no thread at all. The history has to come from somewhere other than the provider.
- Even a retry on the same model after a timeout is uncertain, since you do not know whether the turn was stored until the next continue.

INFRO's failover works before the first token: connection failover tries the same model on another of your connections, and model fallbacks try an ordered list of other models.[3] Either way the next target receives a complete request. With a gateway-held history, building that request is a lookup. Without one, the gateway holds only the delta and cannot build it.

The pattern is simple to state:

```python
def send_turn(conv, new_messages):
    conv.history.extend(new_messages)          # your store is authoritative
    try:
        if conv.last_response_id:
            resp = call_continue(conv, new_messages)
        else:
            resp = call_create(conv, conv.history)
    except NotFound:                            # thread expired or deleted
        resp = call_create(conv, conv.history)  # replay the full history
    except TargetUnavailable:
        resp = call_create(next_target(conv), conv.history)  # fresh start elsewhere
    conv.history.append(resp.message)
    conv.last_response_id = resp.id
    return resp
```

The point is the order: append to your store first, then treat the thread as a shortcut that may fail.

## Retention: a short window, with an eligibility catch

On retention, the docs say a thread's contents (conversation, system prompt, tools) are held in memory, encrypted under a server-managed key, never written to durable storage, and deleted within an hour of last use.[2] The same page lists the feature as not ZDR eligible and not HIPAA eligible.[2] Using threads does not affect the ZDR eligibility of other features, but a workload that must stay inside a ZDR agreement should not enable the beta without confirming it with the provider. The existing article on [zero data retention across providers](https://infro.io/blog/zero-data-retention-across-ai-providers-what-to-verify) covers what to verify.

The other half of the retention story is your own store, which now holds the full conversation for as long as you decide. That is a data-protection obligation you carry, but also a control you have: one deletion path, one retention clock, one place to answer an access request. A gateway store concentrates that. A per-app store multiplies it by the number of services.

Separately, a gateway that logs payloads will see less than before. With payload logging on, a continue request records only the new messages, so the log no longer shows what the model read. INFRO keeps prompt and response capture off by default and lets the organization choose a retention window within its plan's limit.[4] If you rely on those logs for audit, decide whether the history store, not the request log, is the record of what the model saw.

## Cost visibility: read usage from the response

A thread shrinks the request, not necessarily the model's input. The API rebuilds the conversation before it calls the model.[2] The threads page I read does not state how input tokens on continue turns are billed, so I am not asserting it here. Verify it against the `usage` object and your provider's own cost report before you forecast.

What does change is how a gateway can see cost. Any estimator that sizes the request body, or counts tokens on the request, will undercount a continue turn, because the body holds only the delta. The reliable source is the usage the provider returns with each response. INFRO's documentation describes recording what each request used as the provider reports it, and estimating the provider bill from that.[5] Two checks are worth adding:

1. Compare estimated spend on thread traffic against the provider's cost report for the same period, and investigate a persistent gap.
2. Track cache read and write tokens separately if you set the top-level `cache_control`, since the page notes that prompt caching on a thread is a separate option and nothing is cached on later turns unless you ask for it or add breakpoints.[2]

Failover adds a cost wrinkle: the first turn on a new target is a full-history create, which pays for the whole input again. See [what failover does to your bill](https://infro.io/blog/what-does-failover-do-to-your-bill-price-gaps-and-cold-caches).

## Lock-in: small in code, larger in stored data

The `thread` parameter is one field on a request, so the code-level coupling is small, and the SDK helper hides it. The beta header must be sent on every request of a thread, and the helper adds it for you.[2] The risk is a design that lets application code depend on the response id as a conversation id.

The larger coupling is what you store. Anthropic-shaped history includes thinking blocks with signatures, cache breakpoints and server-tool blocks. INFRO's agent documentation notes that on an Anthropic connection these reach the provider as written.[6] Store them verbatim, then convert at the edge when you fail over to a model reached through another API shape. Converting at write time loses data you may need to continue on Anthropic later.

A practical rule: use your own conversation id as the key, keep the provider response id as a disposable pointer next to it, and never expose the pointer to clients.

## Where a gateway should draw the line

Put these in the gateway, because they need the full history and a view across providers:

- The authoritative history, keyed by your conversation id, written before the provider call.
- The decision to continue, replay or start fresh on another target.
- Usage records read from each response, attributed to the conversation.
- The retention and deletion policy for stored history.

Leave these to the provider or the application: the thread id itself, the 5-minute timer, and anything that depends on a model's own fields.

INFRO's `/v1/messages` endpoint forwards Anthropic's `anthropic-beta` header on an Anthropic connection.[6] The pages I read do not mention the `thread` field or a gateway-held conversation store, so test thread requests through your gateway end to end before depending on them, and do not assume a history store exists in any product unless its documentation says so.

## A decision checklist

- [ ] Is the workload long-running with large histories, where request size and time to first token matter? If not, skip threads.
- [ ] Is it covered by a zero-data-retention or HIPAA requirement? Threads are not eligible for either.[2]
- [ ] Do you already store full history somewhere you control? If not, build that first.
- [ ] Does your fallback plan cross models or providers? Then every fallback is a full-history create.
- [ ] Can your cost tooling read usage from responses rather than sizing requests?
- [ ] Is the feature acceptable as a beta, with the header and behavior open to change?

If you answer yes to the first and the last, and no to the second, adopt threads behind a gateway-held history and treat every 404 as routine.

## Questions

### Does a message thread survive a failover to another provider?

No. A thread is held by the API that created it, so a different provider has nothing to continue. Send the full history in a new request to the fallback target and start a new thread there if it supports one.

### Can I use message threads with zero data retention?

Anthropic lists threads as not ZDR eligible and not HIPAA eligible. Its docs say the contents are held in memory, encrypted, and deleted within an hour, but eligibility is stated as not eligible, so check with your own agreement before enabling it.

### Do I still need to keep conversation history if I use threads?

Yes. Anthropic says your application stays the source of truth and that you should build the replay path from the start, because a thread is best-effort and a continue request can return 404.

## Sources

1. Claude Platform release notes — Anthropic (Claude Platform Docs). https://platform.claude.com/docs/en/release-notes/overview (accessed 2026-11-07)
2. Message threads — Anthropic (Claude Platform Docs). https://platform.claude.com/docs/en/build-with-claude/message-threads (accessed 2026-11-07)
3. Failover & fallbacks — INFRO Docs. https://infro.io/docs/features/fallbacks (accessed 2026-11-07)
4. Privacy & data — INFRO Docs. https://infro.io/docs/privacy (accessed 2026-11-07)
5. INFRO documentation — INFRO Docs. https://infro.io/docs (accessed 2026-11-07)
6. Claude Code & Codex — INFRO Docs. https://infro.io/docs/coding-agents (accessed 2026-11-07)
