How OpenTelemetry for LLMs brings providers into one schema

LLM observability does not break because a team lacks traces. It breaks when one dashboard calls input tokens prompt_tokens, another calls them input_tokens, a third mixes them with cached tokens, and a fourth counts a call as successful even though the user got a result only after three fallback attempts.

What you need is not a «universal» copy of every API response. You need a small internal schema in which each field answers one question, while the original provider response is stored separately. Then you can change providers, add routing, and survive SDK updates without rewriting reports on quality, latency, and spend.

A proper schema describes facts, not API names

A call record should capture what happened, not reproduce the vocabulary of a particular vendor. One provider may call tokens usage.input_tokens, another usageMetadata.promptTokenCount, and a third prompt_tokens. To an analyst, these are the same fact: how many input tokens the operation processed.

I usually divide fields into three layers.

• The shared llm.* layer stores normalized facts and remains the contract for dashboards, budgets, and alerting rules.
• The gen_ai.* layer repeats compatible OpenTelemetry attributes where the semantics match.
• The llm.raw.* layer stores references to the original request and response, their checksums, content type, and adapter version, but does not put the entire JSON into span attributes.

This separation does not duplicate data for the sake of appearance. GenAI standards in OpenTelemetry are still changing. For example, the project specifically warns existing instrumentation not to switch by default from convention versions used before v1.36.0, and introduces a stability switch for migration. If all your reporting depends on one changing attribute name, a library update becomes an analytics incident.

A minimal contract for one LLM operation might look like this:

{
  "llm.schema.version": "1.0",
  "llm.operation": "chat",
  "llm.provider.requested": "openai",
  "llm.provider.resolved": "anthropic",
  "llm.model.requested": "smart-assistant",
  "llm.model.resolved": "claude-sonnet",
  "llm.request.stream": true,
  "llm.response.finish_reason": "tool_call",
  "llm.outcome": "success",
  "llm.raw.request_ref": "obj://telemetry/req/7f2c",
  "llm.raw.response_ref": "obj://telemetry/res/7f2c",
  "llm.raw.adapter": "anthropic-messages-v3"
}

There is no unqualified model field here. That is intentional. A lone model almost always starts an argument: does it mean the model requested by the client, the model selected by the router, or the string from the final response? A month later, nobody remembers, while the cost chart looks convincing and lies.

The requested model and the actual model are not the same

llm.model.requested describes the intention of the calling code. llm.model.resolved describes what was executed. An alias, routing rule, availability check, provider rejection, or switch to another region may come between them.

Do not hide this difference even in the simplest integration. Imagine that a product team sends customer-support, the router first selects one model, encounters a capacity limit, and executes the request on another. If the trace contains only customer-support, SRE will not see the source of the latency increase. If it contains only the final physical model, the product owner will not understand which business rule caused the expense.

Also add two separate provider fields. A model and a provider are not inseparable: one model identifier may be available through several vendors, and a proxy may route the request to its own infrastructure.

A useful set looks like this:

llm.provider.requested
llm.provider.resolved
llm.model.requested
llm.model.resolved
llm.route.id
llm.route.reason
llm.fallback.count

llm.route.reason should not be free text taken from an exception. Restrict it to a dictionary: direct, cost_policy, latency_policy, capacity, provider_error, data_residency. Otherwise you will get thousands of unique values and useless aggregation.

Do not confuse fallback with retry. A fallback changes the execution location or model. A retry repeats the attempt in the same selected direction. One user call can include both, and they produce different symptoms on a chart. Retries more often point to network issues, limits, or temporary instability. A fallback shows routing-policy behavior and changes response quality, price, or processing jurisdiction.

Store tokens by purpose, not as one total

total_tokens is convenient for a short API response and almost useless for financial reporting, prompt optimization, and cache analysis. In real bills, the same amount of text can cost different amounts depending on whether it was processed as ordinary input, read from the cache, written to the cache, or generated by the model during reasoning.

For a normalized schema, use these counters:

llm.usage.input_tokens
llm.usage.output_tokens
llm.usage.reasoning_output_tokens
llm.usage.cache_read_input_tokens
llm.usage.cache_creation_input_tokens
llm.usage.total_tokens_reported
llm.usage.source

llm.usage.source should have a small set of values: provider, gateway, estimated, unavailable. This field often saves an investigation. If finance finds a discrepancy with an invoice, they can immediately see whether the number came from the API response or an approximate count from a local tokenizer.

OpenTelemetry recommends reporting billable tokens when a system provides used and billable tokens separately. That is a sensible rule for cost metrics, but it is not a reason to discard technical counters. Keep billable tokens in the cost calculation, and retain technical categories for analyzing context, caching, and generation.

Do not invent zeroes. If a provider did not return cached-token data, 0 means there definitely was no cache. That is a different fact from «the provider did not report a value». In JSON attributes, it is better to omit the field entirely. In normalized storage, use null together with llm.usage.source=unavailable or a separate indicator that the breakdown is unavailable.

Here is another trap: do not add cache_read_input_tokens on top of input_tokens until you have documented the adapter's semantics. In some APIs, the input counter already includes cache reads; in others, the categories are separate. The internal contract should say explicitly whether input_tokens is the full input or only the uncached input. I recommend defining it as the full input volume and keeping cached categories as breakdowns. Then the formula does not have to guess what was excluded from the base field.

Cost needs provenance and a calculation status

You cannot honestly derive cost from the model name at the end of the month. Pricing depends on the date, region, processing mode, token type, batch mode, and sometimes the access channel itself. If the system records only llm.cost.usd=0.004, you will not be able to explain the number to an auditor, budget owner, or engineer changing the routing.

Record more than the total:

{
  "llm.cost.currency": "USD",
  "llm.cost.amount": 0.00428,
  "llm.cost.status": "final",
  "llm.cost.method": "provider_usage",
  "llm.cost.rate_card_id": "2026-07-usage-v4",
  "llm.cost.input_amount": 0.00120,
  "llm.cost.output_amount": 0.00308
}

llm.cost.status is more useful than it may seem. I use four states: final, when the amount is based on confirmed usage and the active rate; estimated, when only part of the counters is known or the price comes from an approximate table; pending, when the stream has not finished; and unavailable, when the cost could not be calculated. Zero replaces none of these states.

Financial reports should sum only final. Operational reports can show final + estimated with an explicit label. Otherwise the team may spend a week looking for an «anomaly» that was actually an unfinished stream call.

Do not put cost in a metric label. Money has far too much cardinality. Send cost as a numeric metric or calculate it in storage from events. Keep the model, provider, operation, environment, and result status in labels. The pricing-table version is rarely suitable for the primary metric, but it works well in a trace or calculation record.

Tools need their own causal chain

A tool call is not merely a finish reason. It changes the shape of the whole operation: the model produced an intention to call a tool, the application performed an action, and then it often called the model again with the result. If you compress this into one span, you will see the total latency but not where the time went or who returned the error.

Create a parent span for the user operation and child spans for model calls and tool execution. For one tool run, these attributes are enough:

llm.tool.name
llm.tool.call_id
llm.tool.type
llm.tool.attempt
llm.tool.outcome
llm.tool.duration_ms
llm.tool.argument_size_bytes
llm.tool.result_size_bytes

You can use the tool name in the span name if your tracing backend supports it and the number of names is limited. OpenTelemetry's GenAI conventions have separately tightened the naming requirement for tool-call execution. But do not put an order ID, user ID, or file path in the span name. That is a direct route to a cardinality explosion.

Tool arguments and results are almost never suitable for attributes. They may contain addresses, document numbers, SQL, customer data, or entire pages from an internal database. Store the size, type, hash, and reference to a protected diagnostic object. If an engineer needs the specific payload, they should request it by trace ID with verified access rather than open it on a shared observability screen.

It is useful to distinguish tool_requested, tool_executed, and tool_rejected. The first means the model requested an action. The second means the application started the action. The third means a policy, schema validation, or the user stopped execution. Many teams write «tool error» for all three cases and then draw the wrong conclusion about model quality.

An attempt error does not always make the operation unsuccessful

An LLM call has at least three result levels: the transport attempt, the provider call, and the user operation. An HTTP 429 on the first attempt that succeeds after 400 milliseconds is an attempt error. It is not an error in the user's operation.

OpenTelemetry recommends leaving the span status unset when an operation completes successfully without an error, and setting Error together with error.type when the operation ends in failure. The same recommendations explicitly say not to record on the span errors that were retried or handled so that the operation completed normally.

A practical scheme looks like this:

• Each attempt's child span receives Error if the attempt ended with a network error, timeout, 429, or a response the client considers unsuccessful.
• The parent LLM-operation span receives Error only if all permitted attempts and fallbacks failed.
• An llm.retry event on the parent records the reason, attempt number, and waiting time, but does not contain the full response text.
• Take error.type from a limited dictionary: rate_limit, timeout, network, auth, invalid_request, provider_unavailable, content_policy, tool_failure.

Do not put the provider's error string in error.type. The message changes from request to request, may contain part of the prompt, and cannot be meaningfully aggregated across thousands of unique strings. Keep the text in a protected diagnostic object if needed. For an uncaught exception, OpenTelemetry defines an event named exception and recommends recording the type, message, and stack trace. Record it once where the exception actually ends the operation, not in every wrapper layer.

Mark user cancellation separately. cancelled is not the same as timeout, and content_policy is not the same as provider_unavailable. These outcomes have different owners and require different actions: the interface may address cancellation, the prompt team may investigate policy, and SRE may handle unavailability.

Store the original payload separately with a clear access policy

The phrase «we will save everything in case we need to debug» usually ends with personal data in indexed attributes, an expensive storage bill, and no safe way to give developers access to traces. Full payloads are rarely needed, but they are genuinely useful during complex incidents. Store them separately instead of pretending they do not exist.

The process is simple. Before sending the request, the adapter creates a diagnostic object, masks known PII fields, limits the size, and saves the result in protected object storage. The span receives only request_ref, SHA-256, size, sensitivity classification, and the masking result. The same happens for the response.

Example attributes:

{
  "llm.raw.request_ref": "obs://llm/2026/07/23/ab12/request",
  "llm.raw.request_sha256": "a63b...",
  "llm.raw.request_bytes": 18422,
  "llm.raw.response_ref": "obs://llm/2026/07/23/ab12/response",
  "llm.raw.redaction": "pii_masked",
  "llm.raw.retention_class": "debug_7d"
}

request_ref should not be a URL that anyone can open in a browser without an authorization check. It is an identifier that your diagnostic service reveals based on the trace ID, role, and access reason. For especially sensitive flows, do not save the payload at all. Keep the length, hash, operation type, prompt version, and token counters. A hash cannot reconstruct the text, but it can help prove that two requests were identical.

Prompt content also should not be written indiscriminately into OpenTelemetry GenAI attributes. Under the GenAI conventions, new input and output message fields are not recorded by default when content capture is disabled. That is responsible caution, not a missing feature.

One trace should answer one user question

Do not create a separate trace for every SDK call if the user performed one action. The parent trace should begin at the boundary of the request, queue task, or background process and carry context through the entire path: retrieval, route selection, LLM calls, tool calls, response validation, and result storage.

For the LLM part, I would use this hierarchy:

POST /support/reply
  llm.workflow support_reply
    llm.route select_model
    gen_ai.chat attempt=1
    llm.tool.execute search_customer
    gen_ai.chat attempt=2
    llm.output.validate

llm.workflow answers «what did the user see?». gen_ai.chat answers «what did this particular model call do?». llm.route answers «why was this execution selected?». Do not combine these entities into one enormous span, or its fields will constantly be overwritten by the latest attempt.

Metrics are built on the same schema, but they do not replace traces. Track p50, p95, and p99 latency, error share by error.type, tokens by type, cost, and fallback share in metrics. Use traces to investigate one request: what context arrived, which route branch ran, how long the tool took, and where the error occurred.

The standard gen_ai.client.token.usage metric uses token type as an attribute and recommends sending it when counters are available without an expensive approximate calculation. Do not send one histogram of «total tokens» and then try to guess the cost structure from it.

The provider adapter should be simple and testable

The adapter should not decide whether the user scenario succeeded, how long to retain the payload, or how to calculate the team's budget. Its job is narrower: receive the provider request and response, extract known fields, convert units to the contract, and report what was missing from the response.

Test the adapter with fixed examples. One should contain an ordinary response with usage. The second should be a streaming response where usage arrives only in the final chunk. The third should cover a cache hit and cache creation. The fourth should cover a tool call. The fifth should cover a 429 followed by a successful retry. The sixth should cover fallback to another model. Without these examples, teams usually test only the happy path and then spend months miscounting precisely the expensive or problematic calls.

Here is an example normalization function that can be covered by contract tests independently of the SDK:

def normalize_usage(raw: dict) -> dict:
    usage = raw.get("usage") or {}
    return {
        "llm.usage.input_tokens": usage.get("input_tokens"),
        "llm.usage.output_tokens": usage.get("output_tokens"),
        "llm.usage.reasoning_output_tokens": usage.get("reasoning_tokens"),
        "llm.usage.cache_read_input_tokens": usage.get("cache_read_tokens"),
        "llm.usage.cache_creation_input_tokens": usage.get("cache_write_tokens"),
        "llm.usage.source": "provider" if usage else "unavailable",
    }

The code deliberately does not replace missing values with zeroes or calculate total_tokens on its own. In a real adapter, add a check for the specific API's semantics: some responses report the full input, others return only parts of it, and a stream may end without final usage after a connection failure.

If your team uses AI Router as an OpenAI-compatible gateway, it is useful to record both the client's original model and the model actually selected by the route in separate attributes. This preserves the connection between application code and real execution without tying your dashboards to one provider's format.

Schema version matters more than a beautiful field set

The telemetry schema will change. New token types, server-side tools, retrieval, multimodal input, and new routing reasons will appear. That is not an argument for free-form JSON without a contract. It is an argument for versioning and disciplined migration.

Write llm.schema.version in every record. Add new fields so that existing consumers continue to work. Treat a change in the meaning of an existing field as a breaking change even if the name remains the same. If you decide that input_tokens now means «uncached tokens» instead of «all input», create a new field and a migration period. Otherwise, the historical chart will start comparing different things.

Once a week, take ten high-cost traces, ten with errors, and several with fallback. Review them manually: is it clear which model was requested, what ran, how the tokens were calculated, where the money came from, and why the call ended? If answering any of these questions requires access to the adapter source code, your schema is not ready yet.

Good LLM telemetry does not need to store every byte or predict every future API. It must avoid confusing intent with execution, unknown with zero, an attempt error with a user error, and a technical counter with a bill. That is enough to survive the next model replacement without blindly repairing every report.