# Embeddings

> POST /v1/embeddings in OpenAI's shape on your own provider connection: one model per request, failover to the same model, and the estimated provider cost.

Source: https://infro.io/docs/api/embeddings

Send text, get vectors back. The request and response follow OpenAI's embeddings API, so the OpenAI SDKs, LangChain and LlamaIndex work with only the base URL and the key changed. The call runs on **your own provider connection**, which bills you for it; INFRO adds routing, budgets checked before the call, the request log and the estimated provider cost of each request. INFRO stores no documents and no vectors and runs no retrieval: your vector database stays yours.

```bash
curl https://api.infro.io/v1/embeddings \
  -H "Authorization: Bearer $INFRO_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "openai/text-embedding-3-small",
    "input": ["How do refunds work?", "Refunds are issued within 5 days."],
    "dimensions": 512,
    "metadata": { "workflow": "support-answer", "step": "embed_query" }
  }'
```

- `model` (string, required) — An embedding model (`openai/text-embedding-3-small`, `openai/text-embedding-3-large`, `mistral/mistral-embed`, `mistral/codestral-embed`), a [route](https://infro.io/docs/routes) that serves one, or `openai-compatible/<model id>` on your own server. A chat model is refused with `400`.
- `input` (string | string[] | number[] | number[][], required) — A string, up to 2,048 strings, or token arrays. Empty strings are refused. Token arrays (what LangChain's `OpenAIEmbeddings` sends by default) are served only by OpenAI and Azure OpenAI connections; see `unsupported_input` below. Per-request token totals are your provider's to enforce (OpenAI: 300,000 summed).
- `dimensions` (integer) — A positive integer, forwarded. Sent to Mistral as `output_dimension`.
- `encoding_format` ("float" | "base64") — Forwarded when set. The OpenAI Python SDK sends `base64` by default and decodes it.
- `user` (string) — Forwarded to the provider, not stored.
- `metadata` (object) — Attribution, as on every endpoint: `workflow`, `step`, `trace_id`, `user_id`, `session_id`, `tags`. See [cost attribution](https://infro.io/docs/features/attribution).
- `fallbacks` (string[]) — Only the same model: an embeddings request that could be served by two models is refused (see below).

Any other top-level parameter is forwarded to the provider as you sent it, which is how a provider-specific field such as Mistral's `output_dtype` reaches it.

```json
{
  "object": "list",
  "data": [
    { "object": "embedding", "index": 0, "embedding": [0.0123, -0.0456, …] },
    { "object": "embedding", "index": 1, "embedding": [0.0789, 0.0012, …] }
  ],
  "model": "openai/text-embedding-3-small",
  "usage": {
    "prompt_tokens": 2600,
    "total_tokens": 2600,
    "cost": 0.000052,
    "price_source": "list"
  },
  "id": "req_…",
  "provider": "openai",
  "connection": "conn_…",
  "route": "primary",
  "route_name": null
}
```

The body is your provider's, passed through as it arrives. INFRO adds `usage.cost` (the estimated provider cost, USD), `usage.price_source`, `id`, `provider`, `connection`, `route` and `route_name` after the vectors, and sets `model` to the model that served. The headers carry `X-INFRO-Request-Id`, `X-INFRO-Model`, `X-INFRO-Provider`, `X-INFRO-Connection` and, on a route, `X-INFRO-Route`.

```python
import os
from openai import OpenAI

client = OpenAI(base_url="https://api.infro.io/v1", api_key=os.environ["INFRO_API_KEY"])
result = client.embeddings.create(
    model="openai/text-embedding-3-small",
    input=["How do refunds work?"],
)
```

## One model per request

Vectors from two embedding models live in unrelated spaces: a query embedded by one model and searched against an index built by another returns confident, wrong neighbours, and nothing reports an error. So an embeddings request never fails over to a different model. Failover means **the same model on another connection** — OpenAI and an Azure OpenAI deployment of the same model, or two keys of one provider. A route whose targets name more than one embedding model is refused when you save it, and an embeddings request whose route or `fallbacks` reach more than one model is refused with `400 embedding_route_mixed_models` before anything is called. There is no override.

## Failover

| Your provider answers | What INFRO does |
| --- | --- |
| 400, 404, 413, 422 | Returns the error (as `400` with the provider's own words) without trying another connection: the same model elsewhere refuses the same request. |
| 401, 403 | Tries the next connection for the same model. |
| 408, 429, 5xx, a timeout, a network failure | Tries the next connection for the same model. |
| 2xx | Committed: the body streams to you and is never re-routed. |

## Cost and recording

Embedding models are priced per input token at the vendor's list rate (`$0.02 / 1M tokens` for `text-embedding-3-small`), or at the rate you entered for your contract. The cost is an estimate of what your provider bills you; INFRO charges nothing per token. Azure OpenAI is estimated at OpenAI's list rate unless you enter your Azure rate. A provider that gives a free token allowance bills less than the estimate while the allowance lasts.

Budgets reserve the worst case before the call — two characters a token for text, the exact count for token arrays — and refuse a request they cannot cover with `402` before your provider is called. Each request is one row in the request log with its tokens and its estimated cost, and counts as one request against your plan's monthly allowance whatever the batch size. When a provider reports no usage, the row records an estimate at four characters a token and is marked as estimated.

## What is stored

Embedding inputs follow your payload-logging setting: stored only when your organization records prompts and the request did not opt out, under your plan's retention. **Vectors are never stored**, nor sent to telemetry exports.

## Providers

OpenAI and Azure OpenAI (strings and token arrays), Mistral (strings), and your own OpenAI-compatible server (strings; it answers for its own `/embeddings`). Other providers' embedding models are not offered yet: their endpoints, prices or usage reporting have not been verified.