A model retired with one day's notice: build a fallback plan

What the OCR 4.0 retirement looked like

Mistral's changelog records the sequence in four entries. On 23 June 2026 it released OCR 4 (mistral-ocr-4-0) and pointed mistral-ocr-latest at it. On 16 July it released OCR 4.1 (mistral-ocr-4-1) and moved mistral-ocr-latest and mistral-ocr-4 to it. OCR 4.1 became generally available on 31 August. Then, on 29 September, the changelog says OCR 4.0 is deprecated and retires on 30 September 2026, with OCR 4.1 as the replacement at the same price ¹.

That is a deprecation notice and a shutdown date one day apart. The announcement that OCR 4 was available is on Mistral's news page, dated 23 June ². Nothing in the changelog says what a retired model returns, so this guide does not guess at the error.

Two kinds of caller sat in this window:

• Alias callers using mistral-ocr-latest had been moved to 4.1 on 16 July without an error. Their risk was a silent behaviour change, not an outage.
• Pinned callers using mistral-ocr-4-0 had one day. For them the replacement was a one-line change, but only if someone had already checked that 4.1's output still fed their parser.

The second group is the case this guide is about. The fix was small. The time to verify it was not there. A same-price successor tells you the invoice will not change; it tells you nothing about whether your downstream code still works.

Why a long-notice playbook does not cover this

Our model deprecation playbook is written for the months between a vendor's email and the date. It lays the work out as inventory on day 0, a candidate evaluation in week 1, a canary by weight in week 2, and blocking the old id before the date ³. That works when notice is weeks long. With one day, every step that assumes calendar time falls away.

What survives is the idea of moving traffic by configuration. What changes is when the work happens. Under short notice the only steps available on the day are:

1. Decide which tier to serve from.
2. Flip to it.
3. Verify it served correctly.

Everything else, including finding the candidate, writing the mapping, running the evaluation and getting sign-off, has to exist beforehand. A fallback plan is the set of things you did before the notice arrived. The sections below cover the three that matter: a register of approved fallbacks, a parity test that keeps them honest, and a switch.

Pre-approve fallbacks in a register

Treat each single-vendor model as a dependency with a written exit. For every model your production path calls, keep a register entry with ordered tiers. Each tier carries four facts: what it is, who approved it, what it cannot do, and when its parity test last passed.

Three rules make the register useful at 02:00:

• Tier 1 is the vendor's own successor, because it is the quickest to approve and the least likely to change your contract with the vendor.
• Every tier lists its gaps. The tier 2 entry above is not decoration. Mistral's OCR documentation says block extraction through include_blocks is available only for OCR 4 or newer, and that older models accept the parameter but return an empty array ⁴. A fallback that accepts your request and quietly returns less is worse than one that errors, because nothing alerts. The same page says table formatting and header and footer extraction need OCR 2512 or newer ⁴, which tells you which older models are even eligible.
• The last tier is a queue. If no model is acceptable, holding documents for a bounded time often beats failing them, provided the caller can tolerate it.

For speech, expect the cross-vendor tier to be thin. INFRO's audio documentation notes that voice IDs are vendor-specific and not portable, which is why fallbacks across text-to-speech vendors rarely work ⁵. For a speech dependency, your realistic fallbacks may be a second model from the same vendor, with the voice difference approved in advance by whoever owns the product's sound.

Prove parity before you need it

A fallback you have never run is a hope. A parity test replays recorded inputs through each tier and compares the output to a golden result against thresholds you have written down. Run it on a schedule, such as nightly or weekly, and on every change to the register, so parity_last_passed is a fact rather than a memory.

Compare what your code consumes, not what the vendor documents. For OCR that usually means:

• the page count and the presence of every field your parser reads;
• text similarity per page against the golden output;
• table count and table markup, since Mistral's table_format can return Markdown or HTML tables ⁴;
• block counts and labels, if you read them;
• the confidence scores your thresholds depend on, which the API returns at page, block or word granularity through confidence_scores_granularity ⁴.

That last one has a catch. The changelog shows the "block" granularity was added on 16 July, alongside OCR 4.1 ¹. If your review queue is driven by confidence, check whether a tier returns the field you threshold on at all.

import difflib
import json
from pathlib import Path

MIN_TEXT_SIMILARITY = 0.97   # illustrative; set from your own tolerance
REQUIRED_PAGE_FIELDS = {"index", "markdown", "tables"}

def check(candidate: dict, golden: dict, needs_blocks: bool) -> list[str]:
    problems = []
    if len(candidate["pages"]) != len(golden["pages"]):
        problems.append("page count differs")
        return problems
    for cand, gold in zip(candidate["pages"], golden["pages"]):
        missing = REQUIRED_PAGE_FIELDS - cand.keys()
        if missing:
            problems.append(f"page {gold['index']}: missing {sorted(missing)}")
        ratio = difflib.SequenceMatcher(
            None, cand["markdown"], gold["markdown"]).ratio()
        if ratio < MIN_TEXT_SIMILARITY:
            problems.append(f"page {gold['index']}: text similarity {ratio:.3f}")
        if len(cand.get("tables", [])) != len(gold.get("tables", [])):
            problems.append(f"page {gold['index']}: table count differs")
        if needs_blocks and not cand.get("blocks"):
            problems.append(f"page {gold['index']}: no blocks returned")
    return problems

def run(tier_outputs_dir: str, golden_dir: str, needs_blocks: bool) -> int:
    failures = 0
    for gold_path in sorted(Path(golden_dir).glob("*.json")):
        cand_path = Path(tier_outputs_dir) / gold_path.name
        found = check(json.loads(cand_path.read_text()),
                      json.loads(gold_path.read_text()), needs_blocks)
        for line in found:
            print(f"{gold_path.name}: {line}")
        failures += bool(found)
    return failures

Keep the golden set from real documents, including the ugly ones: scans, rotated pages, tables that span pages, and the language mix you actually receive. A parity test run on clean samples passes everything. Record what differs and decide in advance which differences are acceptable. If tier 3 returns different table markup, either write the mapping now or mark the gap in the register so the on-call engineer knows what they are accepting.

Build the emergency switch

The switch moves traffic from one tier to another without a deploy. Four properties matter more than how it is implemented:

• Read at request time or with a short cache. A flag read only at process start needs a restart, which is a deploy by another name.
• One key per capability. Flipping ocr.tier should not require edits in five services. Services call a capability name you own, the same idea as the route name in the playbook.
• A recorded reason. Whoever flips it writes why, so the next shift knows whether to flip it back.
• A way to drain. Work already in flight on the old tier must finish or be retried on the new one, and idempotency keys must stop a retry creating duplicates.

import time

_CACHE = {"tier": None, "at": 0.0}
TTL_SECONDS = 5

def current_tier(read_config) -> str:
    now = time.monotonic()
    if _CACHE["tier"] is None or now - _CACHE["at"] > TTL_SECONDS:
        _CACHE["tier"] = read_config("ocr.tier")   # e.g. "tier-1"
        _CACHE["at"] = now
    return _CACHE["tier"]

def ocr(document, read_config, register, call_model, record):
    tier = register[current_tier(read_config)]
    if tier.get("mode") == "queue":
        return record.queue(document, max_age_hours=tier["max_age_hours"])
    result = call_model(tier["model"], document)
    record.served(model=tier["model"], tier=tier_id_of(tier))   # served model is data
    return result

Log the model that actually served every request. After the flip, the first thing the on-call engineer needs is evidence that traffic really moved. The second is a comparison of output length, error rate and downstream rejection rate against the hour before. If you only look at HTTP status, a fallback that returns empty block arrays looks healthy.

What a gateway can and cannot do here

Some of this you can put in a gateway. INFRO's routes give your application a stable name and resolve it to ordered or weighted targets over your own provider connections; a target at weight 0 is a standby that never takes a first attempt ⁶. Changing a route's targets is a configuration edit, which is the switch above. For text calls, the fallbacks array lets a request name up to five catalog models to try, in order, once every connection for the primary model has failed before the first token ⁷. A route's failure rules can decide per failure kind whether to retry, move to the next target or stop ⁸.

Two limits matter for this case. First, those fallbacks fire on upstream failures before the response starts ⁷. A model that returns a 200 with thinner output will not trigger one, so the parity test and served-model logging still carry that risk. Second, INFRO's documented endpoints do not include OCR ⁹, so an OCR dependency needs the register, test and switch in your own code. Transcription and speech are covered endpoints; Mistral connections serve transcription ⁵.

INFRO also records vendors' retirement dates and, from the shutdown date, refuses a retired model before any provider call, naming the replacement. That turns a mystery failure into an explicit message, but it is a refusal, not a fallback: the fallback still has to exist beforehand.

The first hour when the notice lands

Use this when a one-day notice arrives, or when a model simply stops answering.

• Confirm the date and the named replacement on the vendor's own changelog, not a forwarded email.
• List every service and key that sends the retiring model id, from request data rather than code search.
• Open the register entry for the capability and read tier 1's gaps and last parity date.
• Re-run the parity test against tier 1 now, even if it passed last week.
• If it passes, flip the switch for a small share first if your traffic allows, then for all of it; write the reason.
• Check served model, output length and downstream rejection rate for the first hour.
• If tier 1 fails parity, move to tier 2 or 3 and record the gap you accepted, with the owner who approved it.
• Remove the retiring model from every fallback list so it protects nothing it cannot serve.
• After the date, write down what the register lacked and fix it that week.

The case study's lesson is not that vendors should give more notice, though they might. It is that notice periods are outside your control and the register, the test and the switch are inside it. Build them while the dependency is healthy, and a one-day notice becomes a one-line configuration change.