# OpenTelemetry for LLM calls: spans, GenAI conventions, privacy

URL: https://infro.io/blog/opentelemetry-llm-tracing
Author: INFRO Editorial (https://infro.io/blog/editorial)
Publisher: INFRO
Published: 2026-10-27
Last modified: 2026-10-27
Last reviewed: 2026-10-27 (next review due 2026-11-26)
Topic: Architecture and migration
Format: Guide

> Trace LLM calls with OpenTelemetry by recording one span per model request with the GenAI semantic conventions: `gen_ai.operation.name`, `gen_ai.provider.name`, `gen_ai.request.model`, `gen_ai.response.model` and the `gen_ai.usage.*` token counts, nested under your own request span through W3C `traceparent`. The conventions are still marked Development and now live in their own repository, so pin versions. Prompt and response content (`gen_ai.input.messages`, `gen_ai.output.messages`) is opt-in in the conventions and should stay off by default. Cost is not a convention attribute; compute it from tokens and your rates. Anthropic added OpenTelemetry export in beta on 14 October 2026, from its SDKs in seven languages and from the API itself.

Key takeaways:
- Record one span per model request with the GenAI semantic conventions and nest it under your own request span through W3C `traceparent`.
- The GenAI conventions are still marked Development and have moved to their own repository; attribute names can change, so pin library versions and keep dashboards on a few stable names.
- Prompt and response content is Opt-In in the conventions and in Anthropic's export; keep spans metadata-only by default and capture content only where a policy says so.
- No convention attribute carries cost: compute it from token counts and your rates, and keep it labelled as an estimate.
- Anthropic's OpenTelemetry export (beta, 14 October 2026) emits spans from its SDKs in seven languages and from the API to a destination you add in the Claude Console.
- SDK, provider and gateway spans answer different questions; use the gateway or provider for coverage and the SDK for your own call structure, and count tokens from one source only.

To trace LLM calls with OpenTelemetry, record one span per model request using the GenAI semantic conventions, nest it under the span of the user request or job that caused it through W3C `traceparent`, and keep prompt content out of the span unless a policy says otherwise. That gives you latency, token usage, errors and the model that actually answered, in the observability backend you already run, linked to the feature and the user that sent the request.

Two things changed in October 2026. Anthropic added OpenTelemetry export in beta on 14 October: its SDKs emit spans in Python, TypeScript, C#, Go, Java, PHP and Ruby, and the API itself can send spans to a destination you add in the Claude Console[1]. And the GenAI conventions now live in their own repository, still marked Development[2][3]. This guide covers the conventions, the attributes worth recording, privacy, and where SDK, provider and gateway tracing each belong.

## The GenAI semantic conventions

The conventions define a shared vocabulary for model calls across vendors and libraries: client inference spans, events and metrics, embeddings, agent spans, exceptions, and vendor-specific conventions for Anthropic, OpenAI, Azure AI Inference and AWS Bedrock, plus Model Context Protocol[3]. Their document status is Development[3], and the page on opentelemetry.io now says they "have moved to the OpenTelemetry GenAI semantic conventions repository"[2].

For an inference call, the span name should be `{gen_ai.operation.name} {gen_ai.request.model}` and the span kind `CLIENT`[4]. The attributes that matter most:

| Attribute | Requirement | What it tells you |
| --- | --- | --- |
| `gen_ai.operation.name` | Required | The operation, such as `chat` or `generate_content` |
| `gen_ai.provider.name` | Required | The provider, such as `openai` or `gcp.vertex_ai` |
| `gen_ai.request.model` | Conditionally required | The model you asked for |
| `gen_ai.response.model` | Recommended | The model that answered; compare it with the request |
| `gen_ai.usage.input_tokens` | Recommended | Input tokens, including cached ones |
| `gen_ai.usage.output_tokens` | Recommended | Output tokens, including reasoning |
| `gen_ai.usage.cache_read.input_tokens` | Recommended when non-zero | Tokens served from the provider's cache |
| `gen_ai.usage.reasoning.output_tokens` | Recommended when non-zero | Output tokens spent on reasoning |
| `gen_ai.conversation.id` | Conditionally required | The session or thread, for grouping turns |
| `gen_ai.input.messages`, `gen_ai.output.messages`, `gen_ai.system_instructions` | Opt-In | Prompt and response content |

Table: Key attributes on a GenAI client inference span, from the conventions' client inference document, read 2026-10-27.

Two rules in that document prevent common counting mistakes: cache-read and cache-write tokens should be included in `gen_ai.usage.input_tokens`, and reasoning tokens in `gen_ai.usage.output_tokens`[4]. Do not add them again. The conventions also define metrics alongside the span, including inference duration, time to first chunk and time per output chunk, and token usage histograms[4]. Tool calls get their own operation name, `execute_tool`[5].

What the conventions do not define is cost. The client inference attributes are token counts; there is no monetary attribute[4]. If you want cost on a span, compute it from tokens and your provider rates, put it under your own namespace, and label it as an estimate, because the provider's invoice is what you pay.

Because the conventions are in Development, names move. Pin the instrumentation library version, read its changelog before upgrading, and keep your dashboards on a short list of attributes you can remap in one place.

## What to record on each span

Beyond the convention attributes, a useful LLM span answers four questions an on-call engineer or a finance lead will ask:

1. **Who sent it?** Your feature, workflow step and end-customer id as span attributes, with opaque ids rather than names or emails.
2. **What did it cost to answer?** Tokens by type, and a cost estimate in your own namespace.
3. **How long did it take, and where?** Span duration, time to first token for streams, and the retry or fallback attempts underneath.
4. **What answered?** The response model and the provider, so a silent model change shows up as a new value.

Attributes with unbounded values belong on spans, not on metrics: an end-customer id is fine as a span attribute and ruinous as a metric label.

## Keeping prompts out of spans

Prompts and responses are the most sensitive data an AI system handles, and the conventions treat them that way: the content attributes are Opt-In, carry a warning that they are "likely to contain sensitive information including user/PII data", and capturing them "is gated by an explicit user opt-in"[4][5]. The conventions also note that content can be larger than backend limits for telemetry envelopes or attribute values[5].

Anthropic's export follows the same line. Each destination and each SDK client has a content mode: **Metadata only** is the default, with identifiers, timing, status and token counts; **Full content** adds prompts and responses[6]. With full content, each span holds up to 0.75 MB of content by default and Anthropic truncates beyond that, dropping the system prompt and earliest messages first[6]. Anthropic's note is the one to repeat to your security team: your observability platform stores that content "under its own access and retention settings"[6].

A workable policy:

- Metadata only in production by default.
- Full content only for named services, with a written reason, a retention period your backend enforces, and access limited to the people who debug those services.
- Redact before export, not after: once a prompt reaches a third-party backend, deleting it there is a request, not a guarantee.
- Never put personal data in ids you export in every mode, such as a conversation id; Anthropic warns about exactly that for `conversation_id`[7].

## Anthropic's OpenTelemetry export

Anthropic offers two sources of spans, and recommends both together[6].

**SDK tracing** runs in your application. Update the Claude SDK and register an OpenTelemetry tracer provider before the first call; the SDK then emits an `anthropic.messages.create` client span per request and spans for tools its tool runner executes, and propagates `traceparent` to the API[7]:

```python
from anthropic import Anthropic
from opentelemetry import trace
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor
from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter

provider = TracerProvider()
provider.add_span_processor(BatchSpanProcessor(OTLPSpanExporter()))
trace.set_tracer_provider(provider)

client = Anthropic()  # content mode defaults to metadata_only
tracer = trace.get_tracer("support-app")

with tracer.start_as_current_span("draft_reply"):
    client.messages.create(
        model="claude-sonnet-5-5",
        max_tokens=1024,
        messages=[{"role": "user", "content": "Draft a reply to this ticket: ..."}],
    )
```

Anthropic warns that during the beta the SDK options and span schema "can change in ways that break your code, your queries, or your dashboards", and recommends pinning the SDK version[7].

**API tracing** runs at Anthropic. An organization admin adds a destination in the Claude Console: an OTLP/HTTP endpoint accepting protobuf over HTTPS on port 443, with credentials in request headers[8]. The API then emits spans for each model request and each server tool use for any client, including ones you did not instrument, and nests them under your trace when the request carries a sampled `traceparent`[8]. Delivery is at least once and best effort, spans can be lost, and the API disables a destination after 24 hours of failed deliveries; an organization can have up to ten destinations[8]. The API emits no spans for requests to its OpenAI-compatible endpoint, and API tracing is available on the Claude API and Claude Platform on AWS[6][8].

Anthropic is direct about one limit: exported spans are not a complete record of usage, and for per-key usage, such as spotting a leaked key, it points to its Usage API instead[6]. Treat traces as diagnostics, not accounting.

## SDK, provider or gateway tracing

| Where spans come from | What it sees | Where it loses |
| --- | --- | --- |
| Your SDK or instrumentation | Your code's structure: feature, tools, retries, time in your own code | Only clients you instrumented; each language and service to maintain |
| The provider's API | Every request to that provider, including server-side tools and model runs | One provider only; your backend receives a second vendor's schema |
| A gateway in front of providers | Every request through it, across providers, with routing and fallback attempts | Nothing that bypasses it; nothing inside a provider's server-side work |

Table: Three places LLM spans can come from, and what each misses.

Most teams need two: SDK spans for call structure inside the application, and either provider or gateway spans for coverage. Whichever you choose, count tokens from one source per request. Anthropic's own guidance is that when a request has both an SDK span and an API span, you sum the API spans, and tell them apart by span kind[6].

For leaders, the useful question is not "do we have tracing" but "can we answer, for yesterday's slowest or most expensive AI request, which feature and which customer sent it, which model answered, and whether its prompt left our systems".

## Exporting gateway spans with INFRO

INFRO is a control plane that sits between your applications and the provider accounts you already hold. Every request through it is recorded, and its [OpenTelemetry export](https://infro.io/docs/observability/exports) pushes a span per request to your own collector, included on every plan.

**What is sent.** OTLP/HTTP protobuf traces to your collector's `/v1/traces` endpoint, configured under **Settings → Gateway → Trace export** with the endpoint and authentication headers; collector credentials are encrypted separately. The service name is `infro-gateway`. Spans carry the request id, model, route, timing, token counts and estimated provider cost, the provider and the connection that served the request, and your attribution metadata (trace id, workflow, step, feature, end-customer id, tags). They also carry the convention attributes `gen_ai.operation.name`, `gen_ai.request.model`, `gen_ai.usage.input_tokens` and `gen_ai.usage.output_tokens`, and `gen_ai.provider.name` for providers the convention names.

**What is not sent.** Prompt and response captures are never included in spans, even when you switch on payload logging, and [guardrail](https://infro.io/docs/features/guardrails) findings appear as counts per detector, never the matched text. Cost is INFRO's estimate of what your provider will bill you, under `infro.cost`, not a convention attribute and not an invoice figure.

**Nesting and delivery.** A W3C `traceparent` on the request nests INFRO's spans inside your own trace. Delivery is best effort and outside the response path: one retry on a server or network error, no retry on a `4xx`, and spans dropped rather than requests blocked. Use INFRO's [request history](https://infro.io/docs/observability/traces) or usage exports as the record, not the trace stream. INFRO's spans cover traffic that goes through INFRO; requests made straight to a provider appear only in that provider's export.

## Questions

### Are the OpenTelemetry GenAI semantic conventions stable?

No. The conventions are marked Development, and on 27 October 2026 nearly every `gen_ai.*` attribute carried a Development badge. They have moved from the main semantic conventions repository to their own, `open-telemetry/semantic-conventions-genai`. Expect renames, pin your instrumentation versions, and build dashboards on a few attributes you can remap.

### Should LLM spans include prompts and responses?

Not by default. The conventions make `gen_ai.input.messages`, `gen_ai.output.messages` and `gen_ai.system_instructions` Opt-In and warn that they are likely to contain personal data. Anthropic's export defaults to metadata only. Capture content where a written policy allows it, with a retention period your backend enforces, and remember your observability vendor becomes a processor of that data.

### Is there a cost attribute in the GenAI conventions?

The client inference conventions define token usage attributes such as `gen_ai.usage.input_tokens`, `gen_ai.usage.output_tokens` and cache and reasoning token breakdowns, but no monetary cost attribute. Compute cost from tokens and your provider rates, and record it under your own namespace as an estimate.

### How do I turn on Anthropic's OpenTelemetry export?

For SDK spans, update the Claude SDK and register an OpenTelemetry tracer provider before the first call. For API spans, an organization admin adds a destination in the Claude Console with an OTLP/HTTP endpoint that accepts protobuf over HTTPS on port 443. The feature is in beta, and spans are delivered best effort.

### Should I trace in the SDK or at a gateway?

Both answer different questions. SDK spans show your code's structure, such as which feature made the call and what tools ran in between. Provider or gateway spans cover every request regardless of client, including services you did not instrument. Count tokens from one source only, or you will double them.

## Sources

1. Claude Platform API release notes — Anthropic. https://platform.claude.com/docs/en/release-notes/api (accessed 2026-10-27)
2. Semantic conventions for generative AI systems (moved notice) — OpenTelemetry. https://opentelemetry.io/docs/specs/semconv/gen-ai/ (accessed 2026-10-27)
3. Semantic conventions for generative AI systems — OpenTelemetry. https://github.com/open-telemetry/semantic-conventions-genai/blob/main/docs/gen-ai/README.md (accessed 2026-10-27)
4. Semantic conventions for GenAI client inference — OpenTelemetry. https://github.com/open-telemetry/semantic-conventions-genai/blob/main/docs/gen-ai/client-inference.md (accessed 2026-10-27)
5. Semantic conventions for GenAI model spans — OpenTelemetry. https://github.com/open-telemetry/semantic-conventions-genai/blob/main/docs/gen-ai/gen-ai-spans.md (accessed 2026-10-27)
6. OpenTelemetry tracing for the Claude API — Anthropic. https://platform.claude.com/docs/en/manage-claude/opentelemetry (accessed 2026-10-27)
7. Set up OpenTelemetry with the Claude SDK — Anthropic. https://platform.claude.com/docs/en/manage-claude/opentelemetry/sdk (accessed 2026-10-27)
8. Set up OpenTelemetry with the Claude API — Anthropic. https://platform.claude.com/docs/en/manage-claude/opentelemetry/api (accessed 2026-10-27)