OpenTelemetry for LLM calls: spans, GenAI conventions, privacy

To trace LLM calls with OpenTelemetry, record one span per model request using the GenAI semantic conventions, nest it under the span of the user request or job that caused it through W3C traceparent, and keep prompt content out of the span unless a policy says otherwise. That gives you latency, token usage, errors and the model that actually answered, in the observability backend you already run, linked to the feature and the user that sent the request.

Two things changed in October 2026. Anthropic added OpenTelemetry export in beta on 14 October: its SDKs emit spans in Python, TypeScript, C#, Go, Java, PHP and Ruby, and the API itself can send spans to a destination you add in the Claude Console¹. And the GenAI conventions now live in their own repository, still marked Development²³. This guide covers the conventions, the attributes worth recording, privacy, and where SDK, provider and gateway tracing each belong.

The GenAI semantic conventions

The conventions define a shared vocabulary for model calls across vendors and libraries: client inference spans, events and metrics, embeddings, agent spans, exceptions, and vendor-specific conventions for Anthropic, OpenAI, Azure AI Inference and AWS Bedrock, plus Model Context Protocol³. Their document status is Development³, and the page on opentelemetry.io now says they "have moved to the OpenTelemetry GenAI semantic conventions repository"².

For an inference call, the span name should be {gen_ai.operation.name} {gen_ai.request.model} and the span kind CLIENT⁴. The attributes that matter most:

Two rules in that document prevent common counting mistakes: cache-read and cache-write tokens should be included in gen_ai.usage.input_tokens, and reasoning tokens in gen_ai.usage.output_tokens⁴. Do not add them again. The conventions also define metrics alongside the span, including inference duration, time to first chunk and time per output chunk, and token usage histograms⁴. Tool calls get their own operation name, execute_tool⁵.

What the conventions do not define is cost. The client inference attributes are token counts; there is no monetary attribute⁴. If you want cost on a span, compute it from tokens and your provider rates, put it under your own namespace, and label it as an estimate, because the provider's invoice is what you pay.

Because the conventions are in Development, names move. Pin the instrumentation library version, read its changelog before upgrading, and keep your dashboards on a short list of attributes you can remap in one place.

What to record on each span

Beyond the convention attributes, a useful LLM span answers four questions an on-call engineer or a finance lead will ask:

1. Who sent it? Your feature, workflow step and end-customer id as span attributes, with opaque ids rather than names or emails.
2. What did it cost to answer? Tokens by type, and a cost estimate in your own namespace.
3. How long did it take, and where? Span duration, time to first token for streams, and the retry or fallback attempts underneath.
4. What answered? The response model and the provider, so a silent model change shows up as a new value.

Attributes with unbounded values belong on spans, not on metrics: an end-customer id is fine as a span attribute and ruinous as a metric label.

Keeping prompts out of spans

Prompts and responses are the most sensitive data an AI system handles, and the conventions treat them that way: the content attributes are Opt-In, carry a warning that they are "likely to contain sensitive information including user/PII data", and capturing them "is gated by an explicit user opt-in"⁴⁵. The conventions also note that content can be larger than backend limits for telemetry envelopes or attribute values⁵.

Anthropic's export follows the same line. Each destination and each SDK client has a content mode: Metadata only is the default, with identifiers, timing, status and token counts; Full content adds prompts and responses⁶. With full content, each span holds up to 0.75 MB of content by default and Anthropic truncates beyond that, dropping the system prompt and earliest messages first⁶. Anthropic's note is the one to repeat to your security team: your observability platform stores that content "under its own access and retention settings"⁶.

A workable policy:

• Metadata only in production by default.
• Full content only for named services, with a written reason, a retention period your backend enforces, and access limited to the people who debug those services.
• Redact before export, not after: once a prompt reaches a third-party backend, deleting it there is a request, not a guarantee.
• Never put personal data in ids you export in every mode, such as a conversation id; Anthropic warns about exactly that for conversation_id⁷.

Anthropic's OpenTelemetry export

Anthropic offers two sources of spans, and recommends both together⁶.

SDK tracing runs in your application. Update the Claude SDK and register an OpenTelemetry tracer provider before the first call; the SDK then emits an anthropic.messages.create client span per request and spans for tools its tool runner executes, and propagates traceparent to the API⁷:

from anthropic import Anthropic
from opentelemetry import trace
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor
from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter

provider = TracerProvider()
provider.add_span_processor(BatchSpanProcessor(OTLPSpanExporter()))
trace.set_tracer_provider(provider)

client = Anthropic()  # content mode defaults to metadata_only
tracer = trace.get_tracer("support-app")

with tracer.start_as_current_span("draft_reply"):
    client.messages.create(
        model="claude-sonnet-5-5",
        max_tokens=1024,
        messages=[{"role": "user", "content": "Draft a reply to this ticket: ..."}],
    )

Anthropic warns that during the beta the SDK options and span schema "can change in ways that break your code, your queries, or your dashboards", and recommends pinning the SDK version⁷.

API tracing runs at Anthropic. An organization admin adds a destination in the Claude Console: an OTLP/HTTP endpoint accepting protobuf over HTTPS on port 443, with credentials in request headers⁸. The API then emits spans for each model request and each server tool use for any client, including ones you did not instrument, and nests them under your trace when the request carries a sampled traceparent⁸. Delivery is at least once and best effort, spans can be lost, and the API disables a destination after 24 hours of failed deliveries; an organization can have up to ten destinations⁸. The API emits no spans for requests to its OpenAI-compatible endpoint, and API tracing is available on the Claude API and Claude Platform on AWS⁶⁸.

Anthropic is direct about one limit: exported spans are not a complete record of usage, and for per-key usage, such as spotting a leaked key, it points to its Usage API instead⁶. Treat traces as diagnostics, not accounting.

SDK, provider or gateway tracing

Most teams need two: SDK spans for call structure inside the application, and either provider or gateway spans for coverage. Whichever you choose, count tokens from one source per request. Anthropic's own guidance is that when a request has both an SDK span and an API span, you sum the API spans, and tell them apart by span kind⁶.

For leaders, the useful question is not "do we have tracing" but "can we answer, for yesterday's slowest or most expensive AI request, which feature and which customer sent it, which model answered, and whether its prompt left our systems".

Exporting gateway spans with INFRO

INFRO is a control plane that sits between your applications and the provider accounts you already hold. Every request through it is recorded, and its OpenTelemetry export pushes a span per request to your own collector, included on every plan.

What is sent. OTLP/HTTP protobuf traces to your collector's /v1/traces endpoint, configured under Settings → Gateway → Trace export with the endpoint and authentication headers; collector credentials are encrypted separately. The service name is infro-gateway. Spans carry the request id, model, route, timing, token counts and estimated provider cost, the provider and the connection that served the request, and your attribution metadata (trace id, workflow, step, feature, end-customer id, tags). They also carry the convention attributes gen_ai.operation.name, gen_ai.request.model, gen_ai.usage.input_tokens and gen_ai.usage.output_tokens, and gen_ai.provider.name for providers the convention names.

What is not sent. Prompt and response captures are never included in spans, even when you switch on payload logging, and guardrail findings appear as counts per detector, never the matched text. Cost is INFRO's estimate of what your provider will bill you, under infro.cost, not a convention attribute and not an invoice figure.

Nesting and delivery. A W3C traceparent on the request nests INFRO's spans inside your own trace. Delivery is best effort and outside the response path: one retry on a server or network error, no retry on a 4xx, and spans dropped rather than requests blocked. Use INFRO's request history or usage exports as the record, not the trace stream. INFRO's spans cover traffic that goes through INFRO; requests made straight to a provider appear only in that provider's export.