# How to change the base URL in OpenAI SDKs, LangChain and AI SDK

URL: https://infro.io/blog/openai-sdk-base-url
Author: INFRO Editorial (https://infro.io/blog/editorial)
Publisher: INFRO
Published: 2026-10-20
Last modified: 2026-10-20
Last reviewed: 2026-10-20 (next review due 2026-11-19)
Topic: Architecture and migration
Format: Guide

> To send OpenAI SDK traffic to another OpenAI-compatible endpoint, set the base URL and the key: base_url (or OPENAI_BASE_URL) in the Python SDK, baseURL in the Node SDK, base_url in LangChain's ChatOpenAI (or OPENAI_API_BASE), and createOpenAICompatible with baseURL in the Vercel AI SDK. Include the /v1 path, keep the provider's key out of the client when a gateway holds it, and check retries, streaming usage and provider-specific fields after the switch.

Key takeaways:
- Python: OpenAI(base_url=..., api_key=...) or the OPENAI_BASE_URL environment variable. Node: new OpenAI({ baseURL, apiKey }).
- LangChain's ChatOpenAI takes base_url or reads OPENAI_API_BASE; the openai SDK underneath reads OPENAI_BASE_URL.
- The Vercel AI SDK's @ai-sdk/openai-compatible package takes a name, baseURL, apiKey and headers.
- Include the version path the endpoint expects (usually /v1) and use the key of the endpoint you are calling.
- After switching, check retries (SDK plus gateway can double them), streamed usage, and fields the framework drops for non-OpenAI providers.

**How do you change the base URL of the OpenAI SDK?** In Python, pass `base_url` to the client (`OpenAI(base_url="https://api.example.com/v1", api_key=...)`) or set the `OPENAI_BASE_URL` environment variable[1]. In Node, pass `baseURL` to the constructor[2]. Frameworks have their own setting: `base_url` on LangChain's `ChatOpenAI`, and `createOpenAICompatible({ baseURL })` in the Vercel AI SDK. The new URL must include the version path the endpoint expects, usually `/v1`, and the key must be the new endpoint's.

That is the one-line migration every gateway and OpenAI-compatible API advertises. This guide gives the exact setting for each client, then the four things that commonly break after the switch.

## Why change the base URL

- **A gateway in front of your providers.** Routing, failover, budgets and request logs without changing application code.
- **Another provider's OpenAI-compatible API.** Anthropic and Google both publish compatibility endpoints, with limits; see [calling Claude and Gemini with the OpenAI SDK](https://infro.io/blog/openai-sdk-claude-gemini).
- **A self-hosted model** behind vLLM or a similar server; see [self-hosted models behind an OpenAI-compatible endpoint](https://infro.io/blog/self-hosted-vllm-openai-compatible).
- **A test double** in CI, so tests never call a paid API.

## Python: the OpenAI SDK

```python
import os
from openai import OpenAI

client = OpenAI(
    base_url="https://api.infro.io/v1",      # the endpoint you are switching to
    api_key=os.environ["INFRO_API_KEY"],     # that endpoint's key, not your OpenAI key
)

reply = client.chat.completions.create(
    model="openai/gpt-6-sol",                # the endpoint's model id
    messages=[{"role": "user", "content": "Say hello."}],
)
print(reply.choices[0].message.content)
```

Instead of the argument you can set `OPENAI_BASE_URL`; the SDK reads it when no `base_url` is passed[1]. That is convenient for a staged rollout and dangerous in a shared environment: every client in the process that does not set `base_url` explicitly follows the variable. Prefer the explicit argument in application code and keep the variable for scripts and tests.

The SDK retries connection errors, 408, 409, 429 and 5xx responses twice by default with exponential backoff, and times out after ten minutes[1]. Both are worth revisiting behind a gateway; see "Retries" below.

## Node and TypeScript: the OpenAI SDK

```typescript
import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://api.infro.io/v1",
  apiKey: process.env.INFRO_API_KEY,
});

const reply = await client.chat.completions.create({
  model: "openai/gpt-6-sol",
  messages: [{ role: "user", content: "Say hello." }],
});
```

The Node SDK's option is `baseURL` in camel case, and it also retries certain errors twice by default[2]. Set it explicitly in the constructor; do not rely on an environment variable you have not confirmed the version you run reads.

## LangChain: ChatOpenAI

```python
import os
from langchain_openai import ChatOpenAI

llm = ChatOpenAI(
    model="openai/gpt-6-sol",
    base_url="https://api.infro.io/v1",
    api_key=os.environ["INFRO_API_KEY"],
)
```

LangChain reads `OPENAI_API_BASE` at initialisation, while the `openai` SDK underneath reads `OPENAI_BASE_URL`; an explicit `base_url` takes precedence over both[3]. Two variables for one setting is a common source of "it works on my machine": set the argument and stop guessing.

One limit matters when the endpoint is not OpenAI. LangChain's page says `ChatOpenAI` targets only the official OpenAI API specification, and provider-specific response fields such as `reasoning_content` are not extracted or preserved; for providers like OpenRouter, LiteLLM, vLLM or DeepSeek it recommends a provider-specific package[3]. A gateway that returns OpenAI's own shape works with `ChatOpenAI`; a provider that adds its own fields needs its own integration if you want those fields.

## Vercel AI SDK

```typescript
import { createOpenAICompatible } from "@ai-sdk/openai-compatible";
import { generateText } from "ai";

const infro = createOpenAICompatible({
  name: "infro",
  baseURL: "https://api.infro.io/v1",
  apiKey: process.env.INFRO_API_KEY,
  headers: { "X-INFRO-Feature": "support-chat" },
});

const { text } = await generateText({
  model: infro("openai/gpt-6-sol"),
  prompt: "Say hello.",
});
```

The `@ai-sdk/openai-compatible` provider takes a `name`, `baseURL`, `apiKey` (sent as a Bearer token) and optional `headers`, added after the authorization header[4]. Custom headers are a clean place to put attribution — feature, team, end customer — if the endpoint reads them.

## curl

```bash
curl https://api.infro.io/v1/chat/completions \
  -H "Authorization: Bearer $INFRO_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model": "openai/gpt-6-sol", "messages": [{"role": "user", "content": "Say hello."}]}'
```

Always test the new endpoint with curl before touching application code. If curl works and the SDK does not, the problem is configuration; if curl fails, it is the endpoint or the key.

## Four things to check after switching

### 1. The path and the key

The base URL replaces everything before the resource path, so it must include the version prefix: `https://api.infro.io/v1`, not `https://api.infro.io`. A missing `/v1` gives a 404 that looks like an endpoint problem. The key must belong to the endpoint you are calling: a gateway that holds your provider keys server-side issues you its own key, and your OpenAI key should not be in the application at all any more.

### 2. Model ids

Many gateways name models `vendor/model` (`openai/gpt-6-sol`, `anthropic/claude-sonnet-5.5`) so one endpoint can serve several vendors. Others accept the provider's bare id. Check `GET /v1/models` on the new endpoint and update the ids in one place.

### 3. Retries

The SDK retries; so does almost every gateway. Two layers of two retries can turn one slow provider call into nine attempts and a long wait. Decide which layer owns retries for each failure: usually the gateway, which can move to another provider, while the SDK keeps a single retry for connection errors. In the OpenAI Python SDK that is `max_retries=1` on the client, or `client.with_options(max_retries=1)` for one call[1].

### 4. Streaming usage and extra fields

Streamed responses report token usage only if the client asks for it (`stream_options={"include_usage": True}` on Chat Completions). If your cost tracking read usage from the provider directly before, confirm it still arrives through the new endpoint. And check any provider-specific fields your code reads — reasoning summaries, citations, safety ratings — because a compatibility layer may drop them.

## Reading the first errors

The first requests after a base URL change fail in recognisable ways:

| Error | Usual cause | Fix |
| --- | --- | --- |
| 404 on every request | The base URL lacks the version path, or has it twice | Use exactly the documented base, such as `https://api.example.com/v1` |
| 401 | The old provider key is still configured, or the new key is in the wrong variable | Print which key source the client resolved; set `api_key` explicitly |
| 400 or 404 naming the model | The endpoint uses different model ids | Copy ids from the endpoint's `GET /v1/models` |
| 400 naming a parameter | The target model does not accept a parameter you send | Check the endpoint's docs for per-model parameters, such as `max_completion_tokens` on newer OpenAI models |
| Timeouts on long requests | A proxy between you and the endpoint closes idle connections | Stream long responses, or raise the proxy's idle timeout |

## Keeping the switch reversible

Put the base URL, key and model ids in configuration, not code, and read them in one place your whole service uses. Then the change is a configuration deploy, and so is the rollback. Keep the old provider key valid until the new path has carried full traffic for a week; a rollback that needs a new key is not a rollback. And log the endpoint each request went to, so that when a metric moves after the switch you can tell which path it came from.

## Rolling it out

Change the base URL behind a flag, send a small share of traffic, compare error rates, latency and cost per request with the old path, then widen. [From direct OpenAI calls to a gateway](https://infro.io/blog/migrate-direct-openai-to-gateway-canary) is a step-by-step canary runbook, and [migrating to an OpenAI-compatible multi-model API](https://infro.io/blog/openai-compatible-api-migration) covers the wider move.

## Using INFRO's endpoint

INFRO's gateway is OpenAI-compatible at `https://api.infro.io/v1`: set it as the base URL, use an INFRO key, and name models as `vendor/model`. Requests run on provider accounts you connected, with your routes and failover, model allowlists and budgets applied before the provider is called. Each response carries `usage.cost`, an estimate of what your provider will bill for that request, beside `provider` and `connection`. INFRO-specific fields such as `fallbacks` sit at the top level of the body and are ignored by other OpenAI-compatible backends, so the same code still runs against them. Claude Code and Codex use their own base-URL variables; see [routing Claude Code and Codex through a gateway](https://infro.io/blog/claude-code-codex-through-gateway).

## Summary

Set the base URL and key explicitly in each client, include `/v1`, update model ids from the endpoint's own list, give retries to one layer, and verify usage and provider-specific fields before you widen the rollout.

## Questions

### What is the OpenAI base URL?

OpenAI's API is served at https://api.openai.com/v1. A gateway or another OpenAI-compatible API gives you its own base URL to use instead, for example https://api.infro.io/v1 for INFRO. Set it in the SDK client and use that endpoint's key.

### How do I set the OpenAI base URL with an environment variable?

The OpenAI Python SDK reads OPENAI_BASE_URL. LangChain's ChatOpenAI also reads OPENAI_API_BASE at initialization, and an explicit base_url argument takes precedence over both. Setting the variable changes every client that does not pass base_url explicitly.

### Do I need to change my code to use a gateway?

For an OpenAI-compatible gateway, usually only the base URL and the API key change. Model names may change too, if the gateway uses its own ids such as vendor/model, and anything the gateway does not support for a given provider has to be checked against its documentation.

## Sources

1. openai/openai-python — GitHub. https://github.com/openai/openai-python (accessed 2026-10-20)
2. openai/openai-node — GitHub. https://github.com/openai/openai-node (accessed 2026-10-20)
3. ChatOpenAI integration — LangChain. https://docs.langchain.com/oss/python/integrations/chat/openai (accessed 2026-10-20)
4. OpenAI Compatible Providers — Vercel AI SDK. https://ai-sdk.dev/providers/openai-compatible-providers (accessed 2026-10-20)