How do I know if the OpenAI API is down or it's my code?

Look at your own error codes by status. Rising 500s, 503 server_is_overloaded and timeouts across many requests point to OpenAI. A 429 with a spend or quota code, or a 401, is on your account, not an outage.

Should I retry when the OpenAI API is down?

Once at most, inside the caller's deadline, and honouring retry-after. Failed requests count toward your rate limit, and the SDKs already retry twice by default, so more retries prolong the problem.

What is the best fallback when OpenAI is down?

The same model on another account (Azure OpenAI, or a second OpenAI project), then a different model that has passed your own evaluation for the task. Never an untested model, and never on a spend-limit error.

OpenAI API down? A runbook for the first 30 minutes

Minutes 0–5: is it OpenAI, or is it you?

The fastest signal is your own error rate by status code, per provider and model. OpenAI's status page, status.openai.com, runs on incident.io and lists API components separately from ChatGPT, including Chat Completions, Responses, Embeddings, Images, Batch, Audio and Realtime. Check the API components, not the product most people talk about.

Then read the codes on your own failed requests:

You see	Meaning	Retry?
`500`	An error on OpenAI's side	Yes, once, with backoff
`503` `server_is_overloaded`	The model is temporarily overloaded	Yes, after `retry-after`
Timeouts, connection resets	Unreachable or stuck	Once, if the deadline allows
`429` with no code, or `slow_down`	Rate limit, or traffic rose too fast	After `retry-after`; ramp up gradually
`429` `project_spend_limit_exceeded`, `organization_spend_limit_exceeded`	Your own enforced spend limit	No: raise the limit
`429` `credit_balance_exhausted`, `organization_usage_limit_exceeded`	Balance or OpenAI-assigned limit	No: billing change needed
`401`, `403`	Key, organization or permission problem	No: fix the key

From OpenAI's error-codes guide.

The misdiagnosis that wastes the most incident time: a 429 that is not a rate limit. If your dashboards group every 429 as "rate limited", you will spend the incident tuning backoff for a problem that needs someone to raise a spend limit. And a deploy that doubled traffic produces slow_down errors that look exactly like an outage.

Minutes 5–10: stop making it worse

Cap retries now. Failed requests count toward OpenAI's per-minute limit, so resending in a loop prolongs a rate limit. The official SDKs already retry twice by default; with your own loop and a gateway on top, one failure can become nine requests.
Shorten timeouts. The SDKs' default request timeout is 10 minutes. During an incident, every user request held that long is a user who has given up.
Open the circuit. Stop sending traffic to a provider whose error rate is above your threshold, and probe it on a timer, instead of letting every request discover the outage.
Pause background work. Backfills, evaluation runs and re-indexing compete with user traffic for the same limits. Stop them first.

import OpenAI, { APIError } from "openai";
import Anthropic from "@anthropic-ai/sdk";

type State = "closed" | "open" | "half-open";

/**
 * Stops sending traffic to a provider that keeps failing, then lets one probe
 * through after a cool-down. Counts only failures that say the provider is
 * unhealthy: a 429 means "busy", not "broken", and must not open the breaker.
 */
export class CircuitBreaker {
  state: State = "closed";
  private failures: number[] = [];
  private openedAt = 0;
  private probing = false;
  private readonly threshold: number;
  private readonly windowMs: number;
  private readonly cooldownMs: number;

  constructor(opts: { threshold: number; windowMs: number; cooldownMs: number }) {
    this.threshold = opts.threshold;
    this.windowMs = opts.windowMs;
    this.cooldownMs = opts.cooldownMs;
  }

  allowRequest(now = Date.now()): boolean {
    if (this.state === "open" && now - this.openedAt >= this.cooldownMs) {
      this.state = "half-open";
      this.probing = false;
    }
    if (this.state === "closed") return true;
    if (this.state === "half-open" && !this.probing) {
      this.probing = true; // exactly one probe at a time
      return true;
    }
    return false;
  }

  recordSuccess() {
    this.state = "closed";
    this.failures = [];
    this.probing = false;
  }

  recordFailure(now = Date.now()) {
    this.failures = this.failures.filter((t) => now - t < this.windowMs);
    this.failures.push(now);
    if (this.state === "half-open" || this.failures.length >= this.threshold) {
      this.state = "open";
      this.openedAt = now;
      this.probing = false;
    }
  }
}

// Unhealthy: server errors, timeouts, broken connections. Not 4xx, not 429.
export function isProviderFailure(err: unknown): boolean {
  if (!(err instanceof APIError)) return false;
  return err.status === undefined || err.status >= 500;
}

const openai = new OpenAI({ maxRetries: 0, timeout: 20_000 });
const anthropic = new Anthropic({ maxRetries: 1, timeout: 20_000 });
// Production values are seconds to minutes; 100 ms keeps this demo fast.
const openaiBreaker = new CircuitBreaker({ threshold: 3, windowMs: 30_000, cooldownMs: 100 });

export async function answer(prompt: string): Promise<string> {
  if (openaiBreaker.allowRequest()) {
    try {
      const res = await openai.chat.completions.create({
        model: "gpt-5.4-mini",
        messages: [{ role: "user", content: prompt }],
      });
      openaiBreaker.recordSuccess();
      return `[openai] ${res.choices[0]?.message.content ?? ""}`;
    } catch (err) {
      if (isProviderFailure(err)) openaiBreaker.recordFailure();
      else if (!(err instanceof APIError && err.status === 429)) throw err;
    }
  }
  const res = await anthropic.messages.create({
    model: "claude-haiku-5-5",
    max_tokens: 1024,
    messages: [{ role: "user", content: prompt }],
  });
  return `[anthropic] ${res.content.map((b) => (b.type === "text" ? b.text : "")).join("")}`;
}

for (let i = 1; i <= 5; i++) {
  console.log(`request ${i}: ${await answer("Tag this ticket.")} (breaker ${openaiBreaker.state})`);
}
await new Promise((r) => setTimeout(r, 150)); // the cool-down passes
console.log(`request 6: ${await answer("Tag this ticket.")} (breaker ${openaiBreaker.state})`);

import threading
import time

import anthropic
import openai


class CircuitBreaker:
    """Stops sending traffic to a provider that keeps failing, then lets one
    probe through after a cool-down. Counts only failures that say the provider
    is unhealthy: a 429 means "busy", not "broken", and must not open it."""

    def __init__(self, threshold: int, window: float, cooldown: float):
        self.threshold, self.window, self.cooldown = threshold, window, cooldown
        self.state = "closed"
        self.failures: list[float] = []
        self.opened_at = 0.0
        self.probing = False
        self.lock = threading.Lock()

    def allow_request(self) -> bool:
        with self.lock:
            now = time.monotonic()
            if self.state == "open" and now - self.opened_at >= self.cooldown:
                self.state, self.probing = "half-open", False
            if self.state == "closed":
                return True
            if self.state == "half-open" and not self.probing:
                self.probing = True  # exactly one probe at a time
                return True
            return False

    def record_success(self) -> None:
        with self.lock:
            self.state, self.failures, self.probing = "closed", [], False

    def record_failure(self) -> None:
        with self.lock:
            now = time.monotonic()
            self.failures = [t for t in self.failures if now - t < self.window] + [now]
            if self.state == "half-open" or len(self.failures) >= self.threshold:
                self.state, self.opened_at, self.probing = "open", now, False


def is_provider_failure(err: Exception) -> bool:
    """Unhealthy: server errors, timeouts, broken connections. Not 4xx, not 429."""
    if isinstance(err, openai.APIConnectionError):
        return True
    return isinstance(err, openai.APIStatusError) and err.status_code >= 500


gpt = openai.OpenAI(max_retries=0, timeout=20.0)
claude = anthropic.Anthropic(max_retries=1, timeout=20.0)
# Production values are seconds to minutes; 0.1 s keeps this demo fast.
openai_breaker = CircuitBreaker(threshold=3, window=30.0, cooldown=0.1)


def answer(prompt: str) -> str:
    if openai_breaker.allow_request():
        try:
            res = gpt.chat.completions.create(
                model="gpt-5.4-mini",
                messages=[{"role": "user", "content": prompt}],
            )
            openai_breaker.record_success()
            return f"[openai] {res.choices[0].message.content}"
        except Exception as err:
            if is_provider_failure(err):
                openai_breaker.record_failure()
            elif not (isinstance(err, openai.APIStatusError) and err.status_code == 429):
                raise
    res = claude.messages.create(
        model="claude-haiku-5-5",
        max_tokens=1024,
        messages=[{"role": "user", "content": prompt}],
    )
    return "[anthropic] " + "".join(b.text for b in res.content if b.type == "text")


if __name__ == "__main__":
    for i in range(1, 6):
        print(f"request {i}: {answer('Tag this ticket.')} (breaker {openai_breaker.state})")
    time.sleep(0.15)  # the cool-down passes
    print(f"request 6: {answer('Tag this ticket.')} (breaker {openai_breaker.state})")

Tested: three 5xx open the breaker, requests then go straight to the fallback, and one successful probe closes it again. 429s never open it.

Minutes 10–20: fail over, or degrade

Failover is only safe to a target you have tested for this workload: prompts tuned for one model behave differently on another, and structured output, tool calling and context limits differ. Choose per feature:

The same model on another account. GPT models on OpenAI and Azure OpenAI; a second OpenAI project with its own rate limit. Behaviour is closest.
Another tested model. One that has passed your evaluation for this task.
Degrade. Cached or partial results, a simpler mode, or a queue that finishes later and notifies the user.
Fail clearly. Say the AI feature is unavailable and the rest of the product works.

Never fail over on a spend or quota error: your own limit said stop, and another provider would spend around it.

import OpenAI, { APIError } from "openai";
import Anthropic from "@anthropic-ai/sdk";

// No retries on OpenAI: a rate-limited provider stays limited for seconds, so
// the fallback IS the retry. Both clients read their keys from the environment.
const openai = new OpenAI({ maxRetries: 0, timeout: 20_000 });
const anthropic = new Anthropic({ maxRetries: 1, timeout: 20_000 });

// Never fail over on these: your own spend or quota limit said stop, and
// moving the traffic to another provider would spend around it.
const QUOTA_CODES = new Set([
  "credit_balance_exhausted",
  "organization_spend_limit_exceeded",
  "project_spend_limit_exceeded",
  "organization_usage_limit_exceeded",
  "insufficient_quota",
]);

function shouldFailOver(err: unknown): boolean {
  if (!(err instanceof APIError)) return false;
  if (err.status === undefined) return true; // connection error or timeout
  if (err.status === 429) return !QUOTA_CODES.has(String(err.code)) && !QUOTA_CODES.has(String(err.type));
  return err.status === 408 || err.status >= 500;
}

export async function answer(prompt: string): Promise<{ text: string; servedBy: string }> {
  try {
    const res = await openai.chat.completions.create({
      model: "gpt-5.4-mini",
      messages: [{ role: "user", content: prompt }],
    });
    return { text: res.choices[0]?.message.content ?? "", servedBy: "openai" };
  } catch (err) {
    if (!shouldFailOver(err)) throw err;
    const res = await anthropic.messages.create({
      model: "claude-haiku-5-5",
      max_tokens: 1024,
      messages: [{ role: "user", content: prompt }],
    });
    const text = res.content.map((block) => (block.type === "text" ? block.text : "")).join("");
    return { text, servedBy: "anthropic" };
  }
}

const { text, servedBy } = await answer("Draft a two-line status update for a delayed order.");
// Log who served it: a fallback that nobody can see becomes a silent quality change.
console.log(`[${servedBy}] ${text}`);

import anthropic
import openai

# No retries on OpenAI: a rate-limited provider stays limited for seconds, so
# the fallback IS the retry. Both clients read their keys from the environment.
openai_client = openai.OpenAI(max_retries=0, timeout=20.0)
anthropic_client = anthropic.Anthropic(max_retries=1, timeout=20.0)

# Never fail over on these: your own spend or quota limit said stop, and
# moving the traffic to another provider would spend around it.
QUOTA_CODES = {
    "credit_balance_exhausted",
    "organization_spend_limit_exceeded",
    "project_spend_limit_exceeded",
    "organization_usage_limit_exceeded",
    "insufficient_quota",
}


def should_fail_over(err: Exception) -> bool:
    if isinstance(err, openai.APIConnectionError):  # includes timeouts
        return True
    if not isinstance(err, openai.APIStatusError):
        return False
    if err.status_code == 429:
        return err.code not in QUOTA_CODES and err.type not in QUOTA_CODES
    return err.status_code == 408 or err.status_code >= 500


def answer(prompt: str) -> tuple[str, str]:
    try:
        res = openai_client.chat.completions.create(
            model="gpt-5.4-mini",
            messages=[{"role": "user", "content": prompt}],
        )
        return res.choices[0].message.content or "", "openai"
    except Exception as err:
        if not should_fail_over(err):
            raise
        res = anthropic_client.messages.create(
            model="claude-haiku-5-5",
            max_tokens=1024,
            messages=[{"role": "user", "content": prompt}],
        )
        text = "".join(block.text for block in res.content if block.type == "text")
        return text, "anthropic"


if __name__ == "__main__":
    text, served_by = answer("Draft a two-line status update for a delayed order.")
    # Log who served it: a fallback that nobody can see becomes a silent quality change.
    print(f"[{served_by}] {text}")

Tested: a rate-limited or failing OpenAI request is answered by Anthropic; a spend-limit 429 is not failed over.

Streaming adds one rule: once a response has started reaching the user, a failure ends that response. Fail over before the first token, and let a broken stream end with a clear message. The 529 guide shows how to take back a partial answer.

Minutes 20–30: communicate and watch

Tell users what works. "Summaries are delayed; everything else is normal" beats a generic error.
Watch cost as well as errors. Failover to a larger model, or a retry storm on requests that started and were billed, makes an outage expensive.
Plan the return. When OpenAI recovers, ramp traffic back gradually; a sudden return to full traffic can trigger slow_down.
Check the other provider too. Anthropic's status page, status.claude.com, lists the Claude API (api.anthropic.com) as its own component and offers Atom and RSS feeds plus email, Slack and webhook subscriptions. Its overload signal is 529 overloaded_error.

After the incident

Then turn every manual decision into configuration: breaker thresholds, fallback chains, which features degrade and how. Rehearse it: force a 5xx on the primary in staging and check that the fallback serves within the caller's deadline.

The decisions, in order

Read your own error codes: if anything looks wrong; before opening the status page.
Stop retries and open the breaker: if 5xx, 503 or timeouts rise for one provider.
Fail over to a tested target: if the outage outlasts one request and a tested alternative exists.
Raise the limit, do not retry: if the 429 carries a spend, quota or balance code.
Degrade and say so: if no tested target can serve the feature.

The INFRO way

What INFRO does during a provider outage

INFRO routes over provider accounts you connected and applies the runbook before the first token: a provider 5xx, timeout or connection error moves the request to your next connection for that model, then to the fallbacks you named; a connection that fails 5 times in a row is taken out of rotation for 30 seconds, growing to 15 minutes; and a provider 429 moves the request on without counting as a failure. Your own budgets and allowlists are decided before any provider is called and never fail over.

Every attempt is on the request's trace with the connection that made it and the error that ended it, so the post-incident timeline is already written. INFRO holds no provider capacity of its own: if none of your connections can serve, the request fails with an error naming each attempt.

import OpenAI from "openai";

// The same SDK, pointed at INFRO. Requests run on your own provider accounts.
const client = new OpenAI({
  baseURL: process.env.INFRO_BASE_URL ?? "https://api.infro.io/v1",
  apiKey: process.env.INFRO_API_KEY,
});

const completion = await client.chat.completions.create({
  model: "openai/gpt-5.4-mini",
  messages: [{ role: "user", content: "Summarize this ticket in one line." }],
  // INFRO extension: tried in order once every connection for the model has failed.
  // @ts-expect-error not in the OpenAI SDK's types
  fallbacks: ["anthropic/claude-haiku-5.5"],
});

// `model` is the model that actually served; compare it to detect a fallback.
console.log(completion.model, completion.choices[0]?.message.content);

import os

from openai import OpenAI

# The same SDK, pointed at INFRO. Requests run on your own provider accounts.
client = OpenAI(
    base_url=os.environ.get("INFRO_BASE_URL", "https://api.infro.io/v1"),
    api_key=os.environ["INFRO_API_KEY"],
)

completion = client.chat.completions.create(
    model="openai/gpt-5.4-mini",
    messages=[{"role": "user", "content": "Summarize this ticket in one line."}],
    # INFRO extension: tried in order once every connection for the model has failed.
    extra_body={"fallbacks": ["anthropic/claude-haiku-5.5"]},
)

# `model` is the model that actually served; compare it to detect a fallback.
print(completion.model, completion.choices[0].message.content)

Questions

How do I know if the OpenAI API is down or it's my code?: Look at your own error codes by status. Rising 500s, 503 server_is_overloaded and timeouts across many requests point to OpenAI. A 429 with a spend or quota code, or a 401, is on your account, not an outage.
Should I retry when the OpenAI API is down?: Once at most, inside the caller's deadline, and honouring retry-after. Failed requests count toward your rate limit, and the SDKs already retry twice by default, so more retries prolong the problem.
What is the best fallback when OpenAI is down?: The same model on another account (Azure OpenAI, or a second OpenAI project), then a different model that has passed your own evaluation for the task. Never an untested model, and never on a spend-limit error.

The team building INFRO, a BYOK-first control plane for production AI. Articles are written from the engineering work on the gateway and cite their external sources.

What to do when the OpenAI API is down: a runbook

Minutes 0–5: is it OpenAI, or is it you?

Minutes 5–10: stop making it worse

Minutes 10–20: fail over, or degrade

Minutes 20–30: communicate and watch

After the incident

The decisions, in order

Questions

Sources