What is confirmed and what is not
The premise of this article is that GPT-6.2 discards reasoning state when the start of a prompt is edited, or when a value in the prefix changes per request. I could not confirm that. OpenAI's GPT-6.2 announcement and its pricing page both returned HTTP 403 when I tried to read them on 2026-11-19, so I have no OpenAI statement on what GPT-6.2 does with reasoning, and no GPT-6.2 rate. Check both pages yourself before you act on the claim.
What I can document is the part you control. A templated prompt has a start, and anything that changes the start of a request changes what the provider sees as its prefix. Provider caches match on that prefix, so an edit to the template or a per-request value rendered into the first lines will break caching on any model. If GPT-6.2 also ties reasoning to an unchanged prefix, the same edits hurt twice. The method below works whether or not the claim holds: it finds the calls whose prefix moved and measures what changed.
How a prefix changes without anyone editing the model call
Prefix changes in gateway traffic come from four places, and the first two are easy to miss.
- A new prompt version. In INFRO, a managed prompt is a slug with numbered versions. A chat completion names the prompt, and the gateway puts the version's messages in front of the request's own messages before routing, guardrails and the spend check.1 Every request therefore starts with the version's text, and a new version changes the start of every request that uses it.
- A header that selects a version. A client that cannot add a body field can name the prompt in the
X-INFRO-Promptheader, assupport-reply@3or a bare slug for production.1 If different clients or tenants send different values, they are sending different prefixes, and moving a label such as production changes the prefix for everyone on it. - Per-request values in the template.
{{variable}}placeholders are replaced once with the values sent, up to 50 variables of up to 100,000 characters each.1 A customer name, a request id or a timestamp in the first lines of the template makes every request's prefix unique. - A different upstream. Caches live with the upstream that served the request, so hits depend on consecutive requests reaching the same connection.2 A failover to another connection starts from a cold cache.
For OpenAI, INFRO's documentation says caching is automatic once the prompt passes a minimum prefix length of roughly 1,024 tokens, with reads at roughly 0.1x the input rate.2 The same page calls these ballpark figures that providers change; your provider's invoice is authoritative.
Find the calls whose prefix moved
You need two things per request: which prompt version it used, and how much of the prompt was served from cache. The version is recorded on every request.1 Cached token counts come from what the provider reports and feed the estimated cost in usage.cost.2
INFRO's traffic-changes findings cover part of this. The cached-share signal fires when the cached share of prompt tokens falls by at least 10 percentage points, and the docs say the usual cause is that the start of the prompt changed.3 The output-tokens signal fires when output per request is up by at least 15% and at least 20 tokens.3 Two limits matter here:
- The output signal fires only on increases. If a model that keeps its reasoning produces fewer output tokens than one that recomputes it, a drop will not raise a finding, so you have to query for it.
- Traffic changes read request metadata, never prompts or answers.3 They cannot tell you that answers got worse. That needs an evaluation.
So run your own comparison. Export the request log, group by prompt version, and compare the periods either side of each edit. The column names below are placeholders; map them to the fields your export or provider usage object actually carries.
import pandas as pd
# One row per request. Rename columns to match your export.
df = pd.read_csv("requests.csv", parse_dates=["created_at"])
df["cached_share"] = df["cached_input_tokens"] / df["input_tokens"].clip(lower=1)
by_version = (
df.groupby(["prompt_slug", "prompt_version", "model"])
.agg(
requests=("created_at", "size"),
cached_share=("cached_share", "mean"),
out_tokens_p50=("output_tokens", "median"),
out_tokens_p90=("output_tokens", lambda s: s.quantile(0.9)),
cost=("estimated_cost", "sum"),
)
.reset_index()
.sort_values(["prompt_slug", "prompt_version"])
)
# Flag a version whose cached share fell 10+ points against the one before it,
# or whose median output moved 15%+ in either direction.
by_version["prev_share"] = by_version.groupby(["prompt_slug", "model"])["cached_share"].shift()
by_version["prev_out"] = by_version.groupby(["prompt_slug", "model"])["out_tokens_p50"].shift()
flag = (
(by_version["prev_share"] - by_version["cached_share"] >= 0.10)
| ((by_version["out_tokens_p50"] / by_version["prev_out"] - 1).abs() >= 0.15)
)
print(by_version[flag].to_string(index=False))A flagged row is a lead, not a verdict. Cached share also falls when traffic is too sparse to keep an entry alive, which is the separate problem INFRO's article on the 12-hour cache covers.4 To separate the two causes, compare requests that arrived close together on the same version. If they still miss, the prefix is varying.
Test whether reasoning is actually dropped
Cache misses are visible in usage. A reasoning discard is not, so test it directly rather than inferring it from the bill.
- Pick one prompt and one fixed set of inputs, ideally from logged requests turned into a dataset.
- Define two variants that differ only in the prefix: the current template, and the same template with one edited sentence in the first lines.
- Run both on GPT-6.2 at the same reasoning effort, with a multi-step input where earlier reasoning would matter, and record output tokens, latency and pass rate for each.
- Compare. If the edited prefix produces more output tokens, higher latency or a lower pass rate on identical inputs, the edit is costing you something beyond the cache.
INFRO's experiments run a dataset across up to six variants, a prompt version among them, judged by your evaluators and shown side by side, with the request count and estimated provider cost before it starts and a spend cap it stops at. That fits this test. Record the effort setting too: the GPT-6 tiers differ in the effort levels they accept, and a comparison at mismatched effort answers a narrower question.5
Put stable text first, except when the input is shared
The default layout is static text first and per-request text last: system instructions, policy, tool definitions and examples, then the customer's message. Caching matches from the start, so everything before the first changing token can be cached. OpenAI's cache guide, as summarised in INFRO's article, also says the cached prefix must be byte-identical and that timestamps, request ids and per-user data belong after the last breakpoint.4
Moving instructions after the input reverses that, and it pays in one case: many calls over the same large input with different instructions. Think of ten questions over one contract. If the instruction comes first and differs per call, no two calls share a prefix. Put the document first and the instruction last, and the document becomes the shared prefix.
Two cautions apply. Whether a model follows instructions as well when they come after a long input is a quality question, not a billing one; test it before you move a prompt, as in the experiment above. And the saving depends on the calls landing close together on the same connection, which failover can break.
Where per-model layout belongs
If GPT-6.2 prefers one layout and your fallback prefers another, the layout is a property of the model, and it should not live in application string concatenation where nobody can see it. Three placement facts decide the design.
First, the prompt version is applied before routing.1 The version is chosen before the gateway knows which target will serve the request, so a request that fails over, or falls back through the fallbacks array of up to five catalog models,6 reaches the second model with the first model's layout. Fallback models should therefore get a layout that works acceptably on both, and your fallback tests should include it.
Second, a version never changes once saved: a change is a new version, and going back is moving a label.1 That gives you a clean unit for layout. Keep one version per layout, record which model it was tested on, and send each model family to its own prompt slug or label. If a layout turns out to regress, rolling it back is a label move, and a rollback rule on the label can do it when scores fall below a threshold you set.
Third, routing rules can send requests to their own targets by metadata, feature, key, project, prompt size and other conditions. Use them to pair a model with the prompt that was tested on it, rather than letting callers pick both independently.
Keep layout out of the call sites. The goal is one place where "GPT-6.2 gets this layout" is written, with a version history and a recorded result, so that when OpenAI's page settles the reasoning question you change one prompt and not forty services.
A checklist before you ship a prefix edit
- Read OpenAI's GPT-6.2 page for what it says about reasoning and prefix changes; do not rely on this article's premise.
- Make every prefix edit a new prompt version, never an in-place change to a shared string.
- Move request ids, timestamps, tenant names and other per-request values below the stable text.
- Confirm the stable prefix is over roughly 1,024 tokens, or it will not cache at all on OpenAI.2
- Run the before-and-after experiment on identical inputs at one reasoning effort, and record output tokens as well as pass rate.
- Watch cached share and median output tokens for the first day on the new version, including decreases.
- Test the fallback model with the primary's layout, since the version is applied before routing.
- Reconcile the effect against your provider's invoice, not only the estimate.
