# Artificial intelligence APIs: a developer's map for 2026

URL: https://infro.io/blog/artificial-intelligence-apis
Author: INFRO Editorial (https://infro.io/blog/editorial)
Publisher: INFRO
Published: 2026-10-19
Last modified: 2026-10-19
Last reviewed: 2026-10-19 (next review due 2026-11-18)
Topic: Architecture and migration
Format: Guide

> Artificial intelligence APIs fall into about ten categories: chat and reasoning models, embeddings, rerank, image, video, speech-to-text, text-to-speech, music, moderation and decision or classifier models. Most chat APIs follow one of two wire formats, OpenAI's Chat Completions or Responses and Anthropic's Messages; media APIs are often asynchronous jobs; and each category bills in its own unit, such as tokens, images, seconds of video, minutes of audio or characters.

Key takeaways:
- Group AI APIs by what they return, not by vendor: text, vectors, scores, images, video, audio or a decision. Each group has its own request lifecycle and billing unit.
- Two wire formats dominate text: OpenAI's Chat Completions and Responses, and Anthropic's Messages. Compatibility layers exist, and they drop fields.
- Long media work is asynchronous: submit, get an id, then poll or receive a webhook. Design for that from the first line of code.
- Pricing units differ by modality: tokens, images, seconds, minutes, characters, songs or search units. Store the unit with every usage record.
- Rate limits belong to the provider account and its tier, so every API you add brings its own ceilings.

Artificial intelligence APIs are HTTP interfaces to models that generate or interpret text, images, video, audio and structured decisions. In practice they fall into about ten categories. Each category has its own request lifecycle, some answering in a second and some handing you a job id. Each also bills in its own unit, and most text APIs copy one of two wire formats. This map covers all three dimensions so you can choose, combine and budget for them without surprises.

It names providers only where their own documentation was checked on 2026-10-19. Model lineups change monthly, so treat the names as examples and the structure as the durable part.

## The categories of AI APIs

| Category | What it returns | Examples of providers and models | Usual lifecycle | Typical billing unit |
| --- | --- | --- | --- | --- |
| Chat and reasoning (LLM API) | Text, tool calls, structured JSON | OpenAI Responses and Chat Completions; Anthropic Messages; Google Gemini | Synchronous or streamed; background mode for long runs | Input and output tokens |
| Embeddings | Vectors | OpenAI `text-embedding-3-*`; Google Gemini Embedding 2; Cohere Embed | Synchronous, batchable | Tokens (Google also per image, second or frame) |
| Rerank | Relevance scores for documents | Cohere Rerank | Synchronous | Search units |
| Image | Generated or edited images | OpenAI `gpt-image-2`; Google Gemini image models | Synchronous, sometimes queued | Image and text tokens, or per image |
| Video | Video files | Google Veo; models hosted on fal's queue | Asynchronous job | Seconds of output |
| Speech-to-text | Transcripts | OpenAI transcription models; ElevenLabs Scribe v2 | Synchronous for short files, realtime streams | Minutes of audio, or tokens |
| Text-to-speech | Audio | OpenAI TTS; ElevenLabs; Gemini TTS | Synchronous or streamed | Characters, or tokens |
| Music and sound | Audio tracks | ElevenLabs Music; Google Lyria | Synchronous or job | Per song or clip |
| Moderation | Category flags and scores | OpenAI `omni-moderation-latest` | Synchronous | Free at OpenAI |
| Decision and classifier | A label, score or action band | Decision models such as TypeSafe's Jev; any classifier behind an API | Synchronous, often batched | Varies by provider |

Table: A map of AI API categories. Every name in it comes from the cited vendor documentation below; none is a ranking.

The sources behind each row:

- **Chat.** OpenAI documents both Chat Completions[1] and the Responses API, which accepts text, images and files, calls functions and built-in tools, and has a `background` option for long runs[2]. Anthropic's Claude API serves `POST /v1/messages`, a Message Batches API and a token-counting endpoint[3].
- **Embeddings.** OpenAI prices `text-embedding-3-small` and `-large` per million tokens[4]. Google's Gemini Embedding 2 bills text per token, images per image, audio per second and video per frame[5]. Cohere's `POST /v2/embed` takes a required `input_type` such as `search_document` or `search_query`[6].
- **Rerank.** Cohere's Rerank takes a query and a list of documents, returns them ordered with a relevance score between 0 and 1, and reports usage as billed search units[7].
- **Image.** OpenAI prices its image models, including `gpt-image-2`, in image tokens and text tokens rather than per picture[4]. Google exposes Gemini image generation through its OpenAI-compatible `images.generate`[8].
- **Video.** Google bills Veo models per second of generated video[5]. fal serves hosted models, video among them, through a persistent queue: you submit, get a `request_id`, then poll or take a webhook[9].
- **Speech.** OpenAI bills several transcription models per minute and others per token, `tts-1` per million characters and newer speech models per token[4]. ElevenLabs documents a per-character price difference between its speech models[10] and offers speech-to-text with Scribe v2[11]. Gemini TTS models bill per token[5].
- **Music.** ElevenLabs' Music API is available to paid subscribers[12]. Google bills Lyria per song or per 30-second clip[5].
- **Moderation.** OpenAI's moderation endpoint classifies text and images, does not classify audio, and is free to use[13].
- **Decisions.** Decision models return a structured choice instead of prose; [What is Jev](https://infro.io/blog/what-is-jev-typesafe) and [classifiers in the request path](https://infro.io/blog/classifiers-in-the-request-path) cover how they are used.

## The two wire formats behind most LLM APIs

Most text APIs either are, or offer a copy of, one of two formats.

**OpenAI's formats.** Chat Completions takes a `messages` array with roles, including the system prompt as a message, and returns `choices`[1]. The newer Responses API takes `input` as a string or a list of items, and adds built-in tools and background execution[2]. Because so many SDKs, frameworks and agents speak Chat Completions, it is the format most third parties imitate. Google's Gemini API, for example, documents an OpenAI-compatible base URL for chat completions, embeddings and images, and labels that support beta[8].

**Anthropic's Messages.** The system prompt is a top-level field, not a message. Content is a string or a list of typed blocks. Requests carry an `anthropic-version` header, with the key in `Authorization: Bearer` or the older `x-api-key` header[3]. Agent tools built on Claude speak this format natively.

Compatibility layers are useful for trying a model and risky to depend on blindly. Anthropic's own OpenAI SDK layer says it is primarily for testing and comparing models. It ignores `strict` on tools and `response_format`, does not support prompt caching or audio input, and concatenates every system message into one top-level system field. Anthropic recommends the native API for full features[14]. Before you migrate a workload, list the fields it depends on and check each against the layer's table. [Migrating to an OpenAI-compatible API](https://infro.io/blog/openai-compatible-api-migration) has the full checklist.

## Synchronous calls, streams and asynchronous jobs

Three lifecycles cover almost every AI API:

1. **Request and response.** Embeddings, rerank, moderation, most images and short transcriptions. Set a timeout and retry idempotent failures.
2. **Streaming.** Chat and speech. The response arrives as server-sent events or audio chunks. Once the first chunk reaches your user, a retry on another provider would duplicate output, so a stream that fails midway should surface as an error.
3. **Jobs.** Video, long media and long reasoning runs. OpenAI's background mode returns a response you poll at `GET /v1/responses/{id}` while it is `queued` or `in_progress`. Background responses that are not stored are deleted after about ten minutes, so collect results promptly[15]. fal's queue reports `IN_QUEUE`, `IN_PROGRESS` and `COMPLETED`, can POST the result to your webhook, and may retry delivery, so it tells you to use `request_id` for idempotency[9].

The job pattern is where most production bugs live: lost webhooks, duplicate deliveries, renders paid for twice. [Async media jobs and webhooks](https://infro.io/blog/async-media-jobs-webhooks) covers idempotent submits, signature checks and the reconciler that catches events you never received.

## Authentication, keys and rate limits

Each provider authenticates differently, even when the body looks the same. Anthropic takes `Authorization: Bearer` or `x-api-key` plus `anthropic-version`[3]. ElevenLabs uses an `xi-api-key` header[16]. fal expects `Authorization: Key …`[9]. Keep every provider key server-side, one per environment, and never ship one in a client app.

Rate limits are attached to the account, not to your code. OpenAI measures requests and tokens per minute and per day, images per minute and, for some audio models, audio minutes per minute. Limits apply per organization and project, and they rise as the organization moves through usage tiers[17]. Anthropic measures requests, input tokens and output tokens per minute per model class and tier. Its 429 responses may carry `retry-after`, and a monthly spend cap returns a 429 without one[18]. Every API you add brings its own ceilings, its own headers and its own way of saying "slow down".

## How pricing units differ by modality

There is no common unit. Text bills input and output tokens, often with separate cached-input rates. Images bill per image or in image tokens. Video bills per second. Transcription bills per minute or per token, and speech per character or per token. Music bills per song or clip. Rerank bills search units. Some endpoints, such as OpenAI moderation, cost nothing[4].

> **Example: One feature, four units**
> A support assistant transcribes a 3-minute voice note, embeds the transcript, reranks 50 help articles and answers with a chat model. With made-up rates of $0.01 per audio minute, $0.05 per million embedding tokens, $0.002 per search unit and $2 input / $8 output per million chat tokens:
> 
> - Transcription: 3 minutes × $0.01 = $0.03
> - Embedding: 600 tokens, about $0.00003
> - Rerank: 1 search unit = $0.002
> - Chat: 4,000 input tokens ($0.008) + 300 output tokens ($0.0024) = $0.0104
> 
> Total: about $0.042, and three quarters of it is the audio. These rates are illustrative, not any provider's.

Two habits make this manageable. Record the unit and quantity with every usage row, never only a dollar figure. And mark a request you cannot price as unpriced, not as $0. [AI API pricing explained](https://infro.io/blog/ai-api-pricing-explained) goes deeper on each unit, and [tokens are not the billing unit](https://infro.io/blog/tokens-are-not-the-billing-unit) explains why even token costs rarely equal tokens times a rate.

## Integrating many AI APIs behind one endpoint

Once a product uses three or four of these APIs, the code that talks to them starts to repeat: one client per provider, one retry loop per client, one spreadsheet per invoice. The common answer is to put one OpenAI-compatible endpoint in front of all of them. Applications keep a standard client; the endpoint translates to each provider's format, keeps media jobs asynchronous and records every call in one shape. [One multimodal AI API](https://infro.io/blog/multimodal-ai-api-guide) describes the design.

The choice that matters is whose accounts sit behind that endpoint. An aggregator runs requests on its own provider accounts; a gateway over your own accounts leaves your contracts, negotiated rates and data agreements in place. [AI aggregator platform vs AI gateway](https://infro.io/blog/ai-aggregator-platform) compares the shapes.

INFRO is that endpoint over your own accounts. You connect the providers you already use, and your applications call one base URL with an INFRO gateway key. That covers chat completions, responses and Anthropic's Messages format for agent tools, plus embeddings, images, speech, transcription, rerank, decisions and video. Video runs as an asynchronous job with webhooks or polling at [`/v1/jobs`](https://infro.io/docs/api/jobs). [Embeddings](https://infro.io/docs/api/embeddings) run on OpenAI, Azure OpenAI, Mistral or OpenAI-compatible connections, and fail over only to the same model, because vectors from different models are not comparable. Each request runs on one of your connections, and failover moves only between your own connections. Every record carries the provider, the unit and an estimate of what your provider will bill.

```python
import os
from openai import OpenAI

client = OpenAI(base_url="https://api.infro.io/v1", api_key=os.environ["INFRO_API_KEY"])

answer = client.chat.completions.create(
    model="anthropic/claude-sonnet-5",
    messages=[{"role": "user", "content": "Summarize this transcript in three bullets."}],
)

image = client.images.generate(model="openai/gpt-image-2", prompt="A product diagram on a dark background")
```

Your providers invoice you for the model usage; INFRO charges a subscription for the control plane. The [quickstart](https://infro.io/docs/quickstart) connects a first provider and makes the first call, and the [video docs](https://infro.io/docs/modalities/video) cover the job lifecycle.

## Questions

### What are the main types of AI APIs?

Text generation (chat and reasoning models), embeddings, rerank, image generation and editing, video generation, speech-to-text, text-to-speech, music and sound, content moderation, and decision or classifier models. Most products use two or three of them together, for example embeddings and rerank to find context and a chat model to answer.

### Which AI API should a developer start with?

Start with the category your feature needs and a provider whose documentation covers the parameters you depend on, such as streaming, tool calls or structured output. Build against an OpenAI-compatible client with a configurable base URL, so trying a second provider later is a configuration change.

### Are all LLM APIs OpenAI-compatible?

No. Many providers offer an OpenAI-compatible endpoint, including Google's Gemini API and Anthropic's Claude API, but these layers can ignore fields. Anthropic, for example, documents that its compatibility layer ignores the strict flag on tools and does not support prompt caching, and recommends its native Messages API for the full feature set.

### Why do some AI APIs return a job id instead of a result?

Because the work takes longer than an HTTP request should stay open. Video renders and long-running model responses are submitted, return an id at once, and finish later; you poll a status endpoint or receive a webhook. Treat delivery as at-least-once and deduplicate on the id.

## Sources

1. Create chat completion — API reference — OpenAI. https://developers.openai.com/api/reference/resources/chat/subresources/completions/methods/create (accessed 2026-10-19)
2. Create a model response — API reference — OpenAI. https://developers.openai.com/api/reference/resources/responses/methods/create (accessed 2026-10-19)
3. API overview — Anthropic. https://platform.claude.com/docs/en/api/overview (accessed 2026-10-19)
4. Pricing — OpenAI. https://developers.openai.com/api/docs/pricing (accessed 2026-10-19)
5. Gemini Developer API pricing — Google. https://ai.google.dev/gemini-api/docs/pricing (accessed 2026-10-19)
6. Embed API (v2) — Cohere. https://docs.cohere.com/reference/embed (accessed 2026-10-19)
7. Rerank API (v2) — Cohere. https://docs.cohere.com/reference/rerank (accessed 2026-10-19)
8. OpenAI compatibility — Google. https://ai.google.dev/gemini-api/docs/openai (accessed 2026-10-19)
9. Queue — fal. https://fal.ai/docs/model-endpoints/queue (accessed 2026-10-19)
10. Text to Speech — ElevenLabs. https://elevenlabs.io/docs/overview/capabilities/text-to-speech (accessed 2026-10-19)
11. Speech to Text — ElevenLabs. https://elevenlabs.io/docs/overview/capabilities/speech-to-text (accessed 2026-10-19)
12. Music — ElevenLabs. https://elevenlabs.io/docs/overview/capabilities/music (accessed 2026-10-19)
13. Moderation — OpenAI. https://developers.openai.com/api/docs/guides/moderation (accessed 2026-10-19)
14. OpenAI SDK compatibility — Anthropic. https://platform.claude.com/docs/en/api/openai-sdk (accessed 2026-10-19)
15. Background mode — OpenAI. https://developers.openai.com/api/docs/guides/background (accessed 2026-10-19)
16. Authentication — ElevenLabs. https://elevenlabs.io/docs/api-reference/authentication (accessed 2026-10-19)
17. Rate limits — OpenAI. https://developers.openai.com/api/docs/guides/rate-limits (accessed 2026-10-19)
18. Rate limits — Anthropic. https://platform.claude.com/docs/en/api/rate-limits (accessed 2026-10-19)

## Revision history

- 2026-10-19: INFRO now serves POST /v1/embeddings; the sentence that said it did not is corrected.