Skip to content

Architecture and migrationGuide

GPT-6.2 and prefix edits: audit your prompt layout

The report that GPT-6.2 discards reasoning after a prefix edit is unconfirmed here, because OpenAI's pages could not be read. The layout problem is real either way: a templated prefix that changes per request or per edit breaks caching. This guide shows how to find those calls, test the alternative layout and decide where per-model layout belongs.

By INFRO Editorial

Published

8 min read

One application request entering a central AI gateway and branching to many model routes

In short

Treat the GPT-6.2 reasoning claim as a hypothesis until you have checked OpenAI's own page, and test it on your traffic by comparing calls before and after each prompt edit. Put stable text first and per-request text last, except when many calls share one large input, where input first and instructions after cache better.

Key takeaways

  • I could not read OpenAI's GPT-6.2 announcement or pricing page (both returned 403), so the claim that a prefix edit discards reasoning is unconfirmed here and should be checked against OpenAI's page.
  • A prompt edit or a per-request value rendered into the start of the prompt changes the prefix, so the provider's cache stops matching whatever happens to reasoning.
  • Find affected calls by grouping requests by prompt version and comparing cached share and output tokens before and after each edit; INFRO's output-token signal only fires on increases, so check decreases yourself.
  • Stable text first and per-request text last is the default; input first and instructions after wins only when many calls reuse one large input.
  • INFRO applies a prompt version before routing, so a failover model receives the primary's layout; keep per-model layout in versions you can test, not in application code.

What is confirmed and what is not

The premise of this article is that GPT-6.2 discards reasoning state when the start of a prompt is edited, or when a value in the prefix changes per request. I could not confirm that. OpenAI's GPT-6.2 announcement and its pricing page both returned HTTP 403 when I tried to read them on 2026-11-19, so I have no OpenAI statement on what GPT-6.2 does with reasoning, and no GPT-6.2 rate. Check both pages yourself before you act on the claim.

What I can document is the part you control. A templated prompt has a start, and anything that changes the start of a request changes what the provider sees as its prefix. Provider caches match on that prefix, so an edit to the template or a per-request value rendered into the first lines will break caching on any model. If GPT-6.2 also ties reasoning to an unchanged prefix, the same edits hurt twice. The method below works whether or not the claim holds: it finds the calls whose prefix moved and measures what changed.

How a prefix changes without anyone editing the model call

Prefix changes in gateway traffic come from four places, and the first two are easy to miss.

  • A new prompt version. In INFRO, a managed prompt is a slug with numbered versions. A chat completion names the prompt, and the gateway puts the version's messages in front of the request's own messages before routing, guardrails and the spend check.1 Every request therefore starts with the version's text, and a new version changes the start of every request that uses it.
  • A header that selects a version. A client that cannot add a body field can name the prompt in the X-INFRO-Prompt header, as support-reply@3 or a bare slug for production.1 If different clients or tenants send different values, they are sending different prefixes, and moving a label such as production changes the prefix for everyone on it.
  • Per-request values in the template. {{variable}} placeholders are replaced once with the values sent, up to 50 variables of up to 100,000 characters each.1 A customer name, a request id or a timestamp in the first lines of the template makes every request's prefix unique.
  • A different upstream. Caches live with the upstream that served the request, so hits depend on consecutive requests reaching the same connection.2 A failover to another connection starts from a cold cache.

For OpenAI, INFRO's documentation says caching is automatic once the prompt passes a minimum prefix length of roughly 1,024 tokens, with reads at roughly 0.1x the input rate.2 The same page calls these ballpark figures that providers change; your provider's invoice is authoritative.

Find the calls whose prefix moved

You need two things per request: which prompt version it used, and how much of the prompt was served from cache. The version is recorded on every request.1 Cached token counts come from what the provider reports and feed the estimated cost in usage.cost.2

INFRO's traffic-changes findings cover part of this. The cached-share signal fires when the cached share of prompt tokens falls by at least 10 percentage points, and the docs say the usual cause is that the start of the prompt changed.3 The output-tokens signal fires when output per request is up by at least 15% and at least 20 tokens.3 Two limits matter here:

  • The output signal fires only on increases. If a model that keeps its reasoning produces fewer output tokens than one that recomputes it, a drop will not raise a finding, so you have to query for it.
  • Traffic changes read request metadata, never prompts or answers.3 They cannot tell you that answers got worse. That needs an evaluation.

So run your own comparison. Export the request log, group by prompt version, and compare the periods either side of each edit. The column names below are placeholders; map them to the fields your export or provider usage object actually carries.

prefix_audit.py
import pandas as pd

# One row per request. Rename columns to match your export.
df = pd.read_csv("requests.csv", parse_dates=["created_at"])

df["cached_share"] = df["cached_input_tokens"] / df["input_tokens"].clip(lower=1)

by_version = (
    df.groupby(["prompt_slug", "prompt_version", "model"])
      .agg(
          requests=("created_at", "size"),
          cached_share=("cached_share", "mean"),
          out_tokens_p50=("output_tokens", "median"),
          out_tokens_p90=("output_tokens", lambda s: s.quantile(0.9)),
          cost=("estimated_cost", "sum"),
      )
      .reset_index()
      .sort_values(["prompt_slug", "prompt_version"])
)

# Flag a version whose cached share fell 10+ points against the one before it,
# or whose median output moved 15%+ in either direction.
by_version["prev_share"] = by_version.groupby(["prompt_slug", "model"])["cached_share"].shift()
by_version["prev_out"] = by_version.groupby(["prompt_slug", "model"])["out_tokens_p50"].shift()
flag = (
    (by_version["prev_share"] - by_version["cached_share"] >= 0.10)
    | ((by_version["out_tokens_p50"] / by_version["prev_out"] - 1).abs() >= 0.15)
)
print(by_version[flag].to_string(index=False))

A flagged row is a lead, not a verdict. Cached share also falls when traffic is too sparse to keep an entry alive, which is the separate problem INFRO's article on the 12-hour cache covers.4 To separate the two causes, compare requests that arrived close together on the same version. If they still miss, the prefix is varying.

Test whether reasoning is actually dropped

Cache misses are visible in usage. A reasoning discard is not, so test it directly rather than inferring it from the bill.

  1. Pick one prompt and one fixed set of inputs, ideally from logged requests turned into a dataset.
  2. Define two variants that differ only in the prefix: the current template, and the same template with one edited sentence in the first lines.
  3. Run both on GPT-6.2 at the same reasoning effort, with a multi-step input where earlier reasoning would matter, and record output tokens, latency and pass rate for each.
  4. Compare. If the edited prefix produces more output tokens, higher latency or a lower pass rate on identical inputs, the edit is costing you something beyond the cache.

INFRO's experiments run a dataset across up to six variants, a prompt version among them, judged by your evaluators and shown side by side, with the request count and estimated provider cost before it starts and a spend cap it stops at. That fits this test. Record the effort setting too: the GPT-6 tiers differ in the effort levels they accept, and a comparison at mismatched effort answers a narrower question.5

Put stable text first, except when the input is shared

The default layout is static text first and per-request text last: system instructions, policy, tool definitions and examples, then the customer's message. Caching matches from the start, so everything before the first changing token can be cached. OpenAI's cache guide, as summarised in INFRO's article, also says the cached prefix must be byte-identical and that timestamps, request ids and per-user data belong after the last breakpoint.4

Moving instructions after the input reverses that, and it pays in one case: many calls over the same large input with different instructions. Think of ten questions over one contract. If the instruction comes first and differs per call, no two calls share a prefix. Put the document first and the instruction last, and the document becomes the shared prefix.

Two cautions apply. Whether a model follows instructions as well when they come after a long input is a quality question, not a billing one; test it before you move a prompt, as in the experiment above. And the saving depends on the calls landing close together on the same connection, which failover can break.

Where per-model layout belongs

If GPT-6.2 prefers one layout and your fallback prefers another, the layout is a property of the model, and it should not live in application string concatenation where nobody can see it. Three placement facts decide the design.

First, the prompt version is applied before routing.1 The version is chosen before the gateway knows which target will serve the request, so a request that fails over, or falls back through the fallbacks array of up to five catalog models,6 reaches the second model with the first model's layout. Fallback models should therefore get a layout that works acceptably on both, and your fallback tests should include it.

Second, a version never changes once saved: a change is a new version, and going back is moving a label.1 That gives you a clean unit for layout. Keep one version per layout, record which model it was tested on, and send each model family to its own prompt slug or label. If a layout turns out to regress, rolling it back is a label move, and a rollback rule on the label can do it when scores fall below a threshold you set.

Third, routing rules can send requests to their own targets by metadata, feature, key, project, prompt size and other conditions. Use them to pair a model with the prompt that was tested on it, rather than letting callers pick both independently.

Keep layout out of the call sites. The goal is one place where "GPT-6.2 gets this layout" is written, with a version history and a recorded result, so that when OpenAI's page settles the reasoning question you change one prompt and not forty services.

A checklist before you ship a prefix edit

  • Read OpenAI's GPT-6.2 page for what it says about reasoning and prefix changes; do not rely on this article's premise.
  • Make every prefix edit a new prompt version, never an in-place change to a shared string.
  • Move request ids, timestamps, tenant names and other per-request values below the stable text.
  • Confirm the stable prefix is over roughly 1,024 tokens, or it will not cache at all on OpenAI.2
  • Run the before-and-after experiment on identical inputs at one reasoning effort, and record output tokens as well as pass rate.
  • Watch cached share and median output tokens for the first day on the new version, including decreases.
  • Test the fallback model with the primary's layout, since the version is applied before routing.
  • Reconcile the effect against your provider's invoice, not only the estimate.

Questions readers ask

Should I stop editing prompts until the GPT-6.2 claim is confirmed?
No. Edit as needed, but make each edit a new prompt version, so the version recorded on every request lets you compare cached share and output tokens before and after it.
Does a response cache change any of this?
It is a different mechanism. INFRO's response cache answers an identical request without calling the provider, and cache hits are left out of the traffic-change signals, so measure the provider cache on requests that reached the provider.

From the team that builds it

Where this shows up in INFRO

INFRO is a control plane in front of the provider accounts you already hold. The capabilities below are the parts of it this article touches, each with the documentation that defines its behaviour.

  • Prompt management

    Available with a plan

    Versioned prompts with labels such as production, filled and applied by the gateway on chat completions, Messages and Responses requests before routing, guardrails and spend checks, with the version recorded on every request. A protected label is moved only by a person in the console. On every plan.

  • Traffic changes

    Available with a plan

    Changes in your traffic that come with no error: a model answered by a new snapshot, longer answers, a lost prompt cache, more fallbacks, answers cut off by the length limit, slower responses or a new error mix. Each finding names the likely cause and the estimated weekly change in provider cost. INFRO sees request metadata, not answers, so it does not judge answer quality.

    Read the documentation for Traffic changes
  • Usage and cost per request

    Available with a plan

    Every request records its model, latency, token or unit counts, and estimated provider cost, queryable by project and key.

    Read the documentation for Usage and cost per request
  • Health-aware routing

    Available with a plan

    Named routes use priority or weighted targets over your own connections, with health checks, retries and failover.

    Read the documentation for Health-aware routing
  • Automatic failover

    Available with a plan

    A route that times out or 5xxs before the response starts is retried elsewhere without your client seeing it.

    Read the documentation for Automatic failover
  • Experiments

    Available with a plan

    A dataset run through the gateway across up to six variants (a model or route, a prompt version, settings), judged by your evaluators and compared side by side, with the request count and estimated provider cost shown before it starts and a spend cap it stops at. On every plan.

  • Runs

    Available with a plan

    Requests that share a trace id listed as one run each: requests, errors, duration, models, estimated provider cost, scores and prompt versions, filtered and sorted over a window of up to 31 days. In the API, the console, the management API and MCP, on every plan.

  • Prompt rollback rules

    Available with a plan

    A rule on a prompt label moves it back to the previous version, or only alerts, when the version it points at scores below a threshold: a mean score, a pass rate or the thumbs-down share of your end users' feedback, over a window and a minimum number of answers. Checked every five minutes; each move is audited and announced to your alert channels. On every plan.

INFRO accounts are self-serve. Provider usage is billed to you by your providers; INFRO charges a software subscription and takes 0% of model spend. See plans.

Sources

Numbered in order of first citation. Each was opened and checked against the claim on the date shown.

  1. 1.Prompt management — Features (opens in a new tab) — INFRO Docs. Accessed Nov 19, 2026.
  2. 2.Prompt caching — Features (opens in a new tab) — INFRO Docs. Accessed Nov 19, 2026.
  3. 3.Traffic changes — Observability (opens in a new tab) — INFRO Docs. Accessed Nov 19, 2026.
  4. 4.OpenAI's 12-hour prompt cache: when the 2.5x write pays off (opens in a new tab) — INFRO Blog. Accessed Nov 19, 2026.
  5. 5.GPT-6 Astra, Sol or Luna: choosing a tier per task (opens in a new tab) — INFRO Blog. Accessed Nov 19, 2026.
  6. 6.Failover & fallbacks (opens in a new tab) — INFRO Docs. Accessed Nov 19, 2026.

About this article

Read on 2026-11-19: INFRO's docs pages on prompt caching, prompt management, traffic changes and failover, and INFRO's articles on OpenAI's 12-hour cache and the GPT-6 tiers. OpenAI's GPT-6.2 announcement and pricing page both returned HTTP 403, so the GPT-6.2 reasoning behaviour and any GPT-6.2 rate are left out as unconfirmed.

Last checked against its sources on . The next review is due by . No separate named reviewer is recorded for this article.

How articles are written, sourced and corrected is set out in the editorial standards.