How to change the base URL in OpenAI SDKs, LangChain and AI SDK

How do you change the base URL of the OpenAI SDK? In Python, pass base_url to the client (OpenAI(base_url="https://api.example.com/v1", api_key=...)) or set the OPENAI_BASE_URL environment variable¹. In Node, pass baseURL to the constructor². Frameworks have their own setting: base_url on LangChain's ChatOpenAI, and createOpenAICompatible({ baseURL }) in the Vercel AI SDK. The new URL must include the version path the endpoint expects, usually /v1, and the key must be the new endpoint's.

That is the one-line migration every gateway and OpenAI-compatible API advertises. This guide gives the exact setting for each client, then the four things that commonly break after the switch.

Why change the base URL

• A gateway in front of your providers. Routing, failover, budgets and request logs without changing application code.
• Another provider's OpenAI-compatible API. Anthropic and Google both publish compatibility endpoints, with limits; see calling Claude and Gemini with the OpenAI SDK.
• A self-hosted model behind vLLM or a similar server; see self-hosted models behind an OpenAI-compatible endpoint.
• A test double in CI, so tests never call a paid API.

Python: the OpenAI SDK

import os
from openai import OpenAI

client = OpenAI(
    base_url="https://api.infro.io/v1",      # the endpoint you are switching to
    api_key=os.environ["INFRO_API_KEY"],     # that endpoint's key, not your OpenAI key
)

reply = client.chat.completions.create(
    model="openai/gpt-6-sol",                # the endpoint's model id
    messages=[{"role": "user", "content": "Say hello."}],
)
print(reply.choices[0].message.content)

Instead of the argument you can set OPENAI_BASE_URL; the SDK reads it when no base_url is passed¹. That is convenient for a staged rollout and dangerous in a shared environment: every client in the process that does not set base_url explicitly follows the variable. Prefer the explicit argument in application code and keep the variable for scripts and tests.

The SDK retries connection errors, 408, 409, 429 and 5xx responses twice by default with exponential backoff, and times out after ten minutes¹. Both are worth revisiting behind a gateway; see "Retries" below.

Node and TypeScript: the OpenAI SDK

import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://api.infro.io/v1",
  apiKey: process.env.INFRO_API_KEY,
});

const reply = await client.chat.completions.create({
  model: "openai/gpt-6-sol",
  messages: [{ role: "user", content: "Say hello." }],
});

The Node SDK's option is baseURL in camel case, and it also retries certain errors twice by default². Set it explicitly in the constructor; do not rely on an environment variable you have not confirmed the version you run reads.

LangChain: ChatOpenAI

import os
from langchain_openai import ChatOpenAI

llm = ChatOpenAI(
    model="openai/gpt-6-sol",
    base_url="https://api.infro.io/v1",
    api_key=os.environ["INFRO_API_KEY"],
)

LangChain reads OPENAI_API_BASE at initialisation, while the openai SDK underneath reads OPENAI_BASE_URL; an explicit base_url takes precedence over both³. Two variables for one setting is a common source of "it works on my machine": set the argument and stop guessing.

One limit matters when the endpoint is not OpenAI. LangChain's page says ChatOpenAI targets only the official OpenAI API specification, and provider-specific response fields such as reasoning_content are not extracted or preserved; for providers like OpenRouter, LiteLLM, vLLM or DeepSeek it recommends a provider-specific package³. A gateway that returns OpenAI's own shape works with ChatOpenAI; a provider that adds its own fields needs its own integration if you want those fields.

Vercel AI SDK

import { createOpenAICompatible } from "@ai-sdk/openai-compatible";
import { generateText } from "ai";

const infro = createOpenAICompatible({
  name: "infro",
  baseURL: "https://api.infro.io/v1",
  apiKey: process.env.INFRO_API_KEY,
  headers: { "X-INFRO-Feature": "support-chat" },
});

const { text } = await generateText({
  model: infro("openai/gpt-6-sol"),
  prompt: "Say hello.",
});

The @ai-sdk/openai-compatible provider takes a name, baseURL, apiKey (sent as a Bearer token) and optional headers, added after the authorization header⁴. Custom headers are a clean place to put attribution — feature, team, end customer — if the endpoint reads them.

curl

curl https://api.infro.io/v1/chat/completions \
  -H "Authorization: Bearer $INFRO_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model": "openai/gpt-6-sol", "messages": [{"role": "user", "content": "Say hello."}]}'

Always test the new endpoint with curl before touching application code. If curl works and the SDK does not, the problem is configuration; if curl fails, it is the endpoint or the key.

Four things to check after switching

1. The path and the key

The base URL replaces everything before the resource path, so it must include the version prefix: https://api.infro.io/v1, not https://api.infro.io. A missing /v1 gives a 404 that looks like an endpoint problem. The key must belong to the endpoint you are calling: a gateway that holds your provider keys server-side issues you its own key, and your OpenAI key should not be in the application at all any more.

2. Model ids

Many gateways name models vendor/model (openai/gpt-6-sol, anthropic/claude-sonnet-5.5) so one endpoint can serve several vendors. Others accept the provider's bare id. Check GET /v1/models on the new endpoint and update the ids in one place.

3. Retries

The SDK retries; so does almost every gateway. Two layers of two retries can turn one slow provider call into nine attempts and a long wait. Decide which layer owns retries for each failure: usually the gateway, which can move to another provider, while the SDK keeps a single retry for connection errors. In the OpenAI Python SDK that is max_retries=1 on the client, or client.with_options(max_retries=1) for one call¹.

4. Streaming usage and extra fields

Streamed responses report token usage only if the client asks for it (stream_options={"include_usage": True} on Chat Completions). If your cost tracking read usage from the provider directly before, confirm it still arrives through the new endpoint. And check any provider-specific fields your code reads — reasoning summaries, citations, safety ratings — because a compatibility layer may drop them.

Reading the first errors

The first requests after a base URL change fail in recognisable ways:

Keeping the switch reversible

Put the base URL, key and model ids in configuration, not code, and read them in one place your whole service uses. Then the change is a configuration deploy, and so is the rollback. Keep the old provider key valid until the new path has carried full traffic for a week; a rollback that needs a new key is not a rollback. And log the endpoint each request went to, so that when a metric moves after the switch you can tell which path it came from.

Rolling it out

Change the base URL behind a flag, send a small share of traffic, compare error rates, latency and cost per request with the old path, then widen. From direct OpenAI calls to a gateway is a step-by-step canary runbook, and migrating to an OpenAI-compatible multi-model API covers the wider move.

Using INFRO's endpoint

INFRO's gateway is OpenAI-compatible at https://api.infro.io/v1: set it as the base URL, use an INFRO key, and name models as vendor/model. Requests run on provider accounts you connected, with your routes and failover, model allowlists and budgets applied before the provider is called. Each response carries usage.cost, an estimate of what your provider will bill for that request, beside provider and connection. INFRO-specific fields such as fallbacks sit at the top level of the body and are ignored by other OpenAI-compatible backends, so the same code still runs against them. Claude Code and Codex use their own base-URL variables; see routing Claude Code and Codex through a gateway.

Summary

Set the base URL and key explicitly in each client, include /v1, update model ids from the endpoint's own list, give retries to one layer, and verify usage and provider-specific fields before you widen the rollout.