Embeddings

Send text, get vectors back. The request and response follow OpenAI's embeddings API, so the OpenAI SDKs, LangChain and LlamaIndex work with only the base URL and the key changed. The call runs on your own provider connection, which bills you for it; INFRO adds routing, budgets checked before the call, the request log and the estimated provider cost of each request. INFRO stores no documents and no vectors and runs no retrieval: your vector database stays yours.

curl https://api.infro.io/v1/embeddings \
  -H "Authorization: Bearer $INFRO_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "openai/text-embedding-3-small",
    "input": ["How do refunds work?", "Refunds are issued within 5 days."],
    "dimensions": 512,
    "metadata": { "workflow": "support-answer", "step": "embed_query" }
  }'

modelstringrequired

An embedding model (openai/text-embedding-3-small, openai/text-embedding-3-large, mistral/mistral-embed, mistral/codestral-embed), a route that serves one, or openai-compatible/<model id> on your own server. A chat model is refused with 400.

inputstring | string[] | number[] | number[][]required

A string, up to 2,048 strings, or token arrays. Empty strings are refused. Token arrays (what LangChain's OpenAIEmbeddings sends by default) are served only by OpenAI and Azure OpenAI connections; see unsupported_input below. Per-request token totals are your provider's to enforce (OpenAI: 300,000 summed).

dimensionsinteger

A positive integer, forwarded. Sent to Mistral as output_dimension.

encoding_format"float" | "base64"

Forwarded when set. The OpenAI Python SDK sends base64 by default and decodes it.

userstring

Forwarded to the provider, not stored.

metadataobject

Attribution, as on every endpoint: workflow, step, trace_id, user_id, session_id, tags. See cost attribution.

fallbacksstring[]

Only the same model: an embeddings request that could be served by two models is refused (see below).

Any other top-level parameter is forwarded to the provider as you sent it, which is how a provider-specific field such as Mistral's output_dtype reaches it.

{
  "object": "list",
  "data": [
    { "object": "embedding", "index": 0, "embedding": [0.0123, -0.0456, …] },
    { "object": "embedding", "index": 1, "embedding": [0.0789, 0.0012, …] }
  ],
  "model": "openai/text-embedding-3-small",
  "usage": {
    "prompt_tokens": 2600,
    "total_tokens": 2600,
    "cost": 0.000052,
    "price_source": "list"
  },
  "id": "req_…",
  "provider": "openai",
  "connection": "conn_…",
  "route": "primary",
  "route_name": null
}

The body is your provider's, passed through as it arrives. INFRO adds usage.cost (the estimated provider cost, USD), usage.price_source, id, provider, connection, route and route_name after the vectors, and sets model to the model that served. The headers carry X-INFRO-Request-Id, X-INFRO-Model, X-INFRO-Provider, X-INFRO-Connection and, on a route, X-INFRO-Route.

import os
from openai import OpenAI

client = OpenAI(base_url="https://api.infro.io/v1", api_key=os.environ["INFRO_API_KEY"])
result = client.embeddings.create(
    model="openai/text-embedding-3-small",
    input=["How do refunds work?"],
)

One model per request

Vectors from two embedding models live in unrelated spaces: a query embedded by one model and searched against an index built by another returns confident, wrong neighbours, and nothing reports an error. So an embeddings request never fails over to a different model. Failover means the same model on another connection — OpenAI and an Azure OpenAI deployment of the same model, or two keys of one provider. A route whose targets name more than one embedding model is refused when you save it, and an embeddings request whose route or fallbacks reach more than one model is refused with 400 embedding_route_mixed_models before anything is called. There is no override.

Failover

Cost and recording

Embedding models are priced per input token at the vendor's list rate ($0.02 / 1M tokens for text-embedding-3-small), or at the rate you entered for your contract. The cost is an estimate of what your provider bills you; INFRO charges nothing per token. Azure OpenAI is estimated at OpenAI's list rate unless you enter your Azure rate. A provider that gives a free token allowance bills less than the estimate while the allowance lasts.

Budgets reserve the worst case before the call — two characters a token for text, the exact count for token arrays — and refuse a request they cannot cover with 402 before your provider is called. Each request is one row in the request log with its tokens and its estimated cost, and counts as one request against your plan's monthly allowance whatever the batch size. When a provider reports no usage, the row records an estimate at four characters a token and is marked as estimated.

What is stored

Embedding inputs follow your payload-logging setting: stored only when your organization records prompts and the request did not opt out, under your plan's retention. Vectors are never stored, nor sent to telemetry exports.

Providers

OpenAI and Azure OpenAI (strings and token arrays), Mistral (strings), and your own OpenAI-compatible server (strings; it answers for its own /embeddings). Other providers' embedding models are not offered yet: their endpoints, prices or usage reporting have not been verified.