OpenRouter now routes by speed and reliability: what to re-check

What changed on Oct 28

Until now the default router sent most traffic to the provider with the lowest input-token price, and a failed request went to the next cheapest.¹ The new default scores each provider endpoint on a "Routing Cost": effective price, time spent waiting (throughput and time to first token), and the expected cost of retries when a provider errors, rate limits or returns unusable output.¹ If 1 in 5 tries fails, OpenRouter counts 1.25 tries per good answer.¹

Effective price is computed from recent measured traffic, including the provider's real cache hit rate and the model's usual input/output mix, so it can differ a lot from list price.¹ Traffic is shared among providers inside a competitive edge, every provider keeps a trickle, a failing provider gets less, and a cap spills load once a provider starts returning 429s.¹

What OpenRouter reports, and what it does not tell you

In a switchback experiment over one week, 24 million requests and a sample of open-source models, OpenRouter reported these results for new versus legacy routing:¹

Treat these as direction, not a forecast. Your model, prompt size, cache pattern and streaming mix decide where you land. The cost line is the one to watch: OpenRouter says the new router can pick a faster or more reliable provider over the cheapest one.¹

Which settings bypass the new router

The rule is per request, so audit what your clients actually send.

An ongoing conversation is pinned to its first provider so the prompt cache stays warm, and falls back automatically if that provider becomes unavailable.¹ The post does not say how a conversation is identified, so check with your own client whether your multi-turn traffic really sticks.

Pin it, and see what served

A deterministic request names the order, turns off fallbacks if you want a hard failure, and asks for routing metadata:

import requests

r = requests.post(
    "https://openrouter.ai/api/v1/chat/completions",
    headers={
        "Authorization": "Bearer " + KEY,
        "X-OpenRouter-Metadata": "enabled",
    },
    json={
        "model": "vendor/model",
        "messages": [{"role": "user", "content": "ping"}],
        "provider": {
            "order": ["provider-a", "provider-b"],
            "allow_fallbacks": True,
            "require_parameters": True,
        },
    },
    timeout=60,
)
meta = r.json().get("openrouter_metadata", {})
print(meta.get("attempt"), meta.get("attempts"), meta.get("endpoints"))

The header surfaces openrouter_metadata with the selected endpoint, candidates considered, and attempt (above 1 means earlier attempts failed and fell back).² For streams it arrives on the final chunk; cache hits never carry it.² The fields order, allow_fallbacks, require_parameters, zdr, only and ignore are documented provider-object fields.³ Provider slugs above are placeholders.

What to re-check

• Serving provider. Sample openrouter_metadata and count requests per provider before and after Oct 28.
• Cost per request. Compare your own spend per request against the week before; effective price follows cache hits and output mix, not list input price.¹
• Tool calls and structured output. The router's quality check today is mainly whether tool calls come back well formed.¹ require_parameters is false by default, so set it where a provider ignoring a parameter would break you.³
• Data rules. Confirm zdr, data_collection and only are set on requests that need them rather than assumed.³
• Timeouts. A faster median does not change your tail; re-measure p95 with streaming on.
• Double retries. OpenRouter retries a failed try with the next ranked provider.¹ If your own gateway also retries, a single outage can multiply attempts; cap one layer.

Test it against your own rules

Replay a fixed prompt set in three arms: default, sort: "price", and your order. Run each long enough that the trickle traffic to weaker providers shows up, compare latency, cost per request and tool-call validity, and keep the arm that meets your SLO. Repeat after a provider incident, since the router reacts in real time.¹

If part of your traffic goes through INFRO routes, those routes use priority or weighted targets over your own connections, retry before the response starts, and never restart a stream that has begun.⁴ INFRO's request log records latency and estimated provider cost per request, which gives you the before/after series on your side; the upstream provider behind OpenRouter still comes from OpenRouter's metadata.