Why Hidden Reasoning Tokens Break Cost Calculations?

LLM costs usually go wrong not because the price per million tokens is incorrect. They go wrong because the accounting model is wrong. A team takes prompt_tokens, adds completion_tokens, multiplies the result by the rate, and gets a neat but sometimes imaginary figure. Then reasoning, prompt caching, streaming, tool context, or routing between providers enters the picture. The old calculator keeps working even though its assumptions are no longer valid.

Hidden reasoning tokens are not a separate mystical fee. They are usually part of the model's work, billed as output but exposed differently by different APIs. The real danger is that fields with the same name can mean different things across providers, while fields with different names can describe the same category. If you do not define the semantics before building analytics, you will either double-count the cost or hide it under "other".

Usage in the response is not your financial model

The usage object describes the measurement a particular API chose to return. It does not replace your internal billing model. Start by answering three separate questions: how many tokens the model processed, how many of them are billable at each rate, and which of them consumed the request limit or throughput capacity.

These questions often produce different numbers. Context read from a cache may cost less than ordinary input while still taking up space in the context window. Internal reasoning may be part of billable output even though the user sees one short paragraph. A gateway-level response-cache hit may never reach the provider at all, in which case usage can be zero even when the HTTP response succeeds.

For that reason, do not call total_tokens "the cost in tokens". It is only a total defined by that response's schema. Sometimes it checks the arithmetic, sometimes it represents all processed content, and sometimes it does not provide enough detail to reconstruct the bill.

A minimal observability model should store at least two layers:

• the raw usage response without changes;
• a normalized record in which every number has a business meaning;
• the applied tariff table with its version and currency;
• the request ID, provider, model, and actual route.

Raw data is not needed because anyone loves logs. Two weeks later, someone will ask why spending on one scenario increased, and you will not be able to answer if you kept only a daily aggregate. The normalized record matters because a report should not need to know every quirk of every SDK.

Four buckets that must not be mixed

A practical calculation starts with four buckets: ordinary input, cache reads, cache writes, and output. Internal reasoning should not automatically become a fifth billable bucket. It is usually an analytical part of the output that is already included in it.

Let us name them:

I = ordinary input tokens
R = tokens read from the cache
W = tokens written to the cache
O = all billable output tokens
T = reasoning or thinking inside O, if the provider exposes it

Use this formula for the price:

cost = I * p_input + R * p_cache_read + W * p_cache_write + O * p_output + fixed_fees

Keep fixed_fees in the schema even if it is currently zero. Some modes and products charge for requests, images, search calls, cache storage, or separate operations rather than tokens alone. If your table cannot physically accept such a row, developers will start hiding the cost in the token price and an audit will become guesswork.

In this model, T is not added to the cost a second time. It answers a different question: what share of the output budget went to thinking, and what share went to text, a structured response, and tool calls. The check looks like this:

0 <= T <= O
visible_output_approx = O - T

The word approx is intentional. You cannot reliably recalculate visible text with a simple tokenizer on the client. The provider may summarize reasoning, hide it, add service fragments, or count multimodal parts differently. For money, O from the response or billing export is authoritative, not the number of tokens in message.content.

There is an important exception. Some APIs return only total input, which already includes cached parts, while others separate input_tokens from cache reads and cache writes. In the first case, you cannot honestly derive I, R, and W if the response provides no breakdown. Save input_total_reported, mark the detail as unavailable, and do not invent proportions.

Reasoning shows the composition of output, not a new cost

The most common mistake looks like this: an engineer sees completion_tokens: 1200 and reasoning_tokens: 900, then records 2,100 output tokens. That is double-counting if the API documentation says reasoning is already included in completion.

For OpenAI, completion_tokens_details.reasoning_tokens is a detail of the output, while completion_tokens remains the total generation counter. Anthropic's documentation for extended thinking puts it even more directly: output_tokens is the total number used for billing, while output_tokens_details.thinking_tokens shows how much of that billable output went to internal thinking. Google Gemini publishes thoughtsTokenCount in usageMetadata alongside input, cache, candidate tokens, and the total. These schemas look similar, but they do not authorize you to apply one formula without checking the documentation for the specific model and endpoint.

The rule is simple: first label every detail field as subset, additive, or unknown.

• subset is already included in the parent counter and is used for analysis;
• additive must be added to the base counter because the API excluded it from the base;
• unknown cannot be used in a money formula without checking the documentation and the bill.

This decision belongs in the adapter configuration, not in the dashboard author's head. Here is an example entry for a schema catalog:

{
  "provider": "example-provider",
  "endpoint": "responses",
  "model_pattern": "*",
  "usage_rules": {
    "output_total": "usage.completion_tokens",
    "reasoning": "usage.completion_tokens_details.reasoning_tokens",
    "reasoning_relation": "subset"
  }
}

Do not use reasoning_relation: "subset" for every integration just because one popular API works that way. The model version, native endpoint, and proxy can change the response format. The normalizer must know where the object came from rather than guessing from the field name.

Also track the gap between visible and billable output. Anthropic explicitly warns that when thinking is summarized or hidden, visible tokens will not match the original reasoning that is billed. This is normal. The error begins when a product team promises "short answers at a fixed price" while measuring only the characters the user sees.

Caching changes the price of input but does not remove the context

Prompt caching and response caching solve different problems, and their names are too similar. They must not be reduced to one "cache" column.

Prompt caching happens at the provider while the context is being processed. The model receives the same long shared prefix again, but takes part of the computation from the cache. Usage may therefore contain a cache read, a cache write, or both. A cache write means that the current request created or updated a cacheable prefix. A cache read means that the request used a previously created part. The first call in a series may cost more than ordinary input, while subsequent calls cost less. If you look only at the average price per request, you will miss the warm-up cost.

Response caching returns a ready-made result for an identical request before the model is called. OpenRouter documentation states that on a cache hit, billing token counters are reset because the call never reached the provider. This is not a prompt-cache hit and does not prove that the model processed zero tokens in the original request. It is a different class of event: a layer above the model served the response.

Store them separately:

{
  "cache": {
    "provider_prompt_read_tokens": 18400,
    "provider_prompt_write_tokens": 0,
    "gateway_response_cache": "miss"
  }
}

On the next identical request, you may see:

{
  "usage": {
    "prompt_tokens": 0,
    "completion_tokens": 0,
    "total_tokens": 0
  },
  "cache": {
    "gateway_response_cache": "hit"
  }
}

Do not compare these two records as if the first were worse than the second in terms of model quality. In the second case, the model did not work at all. For product analytics, it is a successful response. For evaluating model quality, data freshness, and provider load, it is a different kind of event.

Caching does not guarantee savings either. It requires a repeatable prefix, a suitable lifetime, and the stable part of the request to be placed correctly. Teams often put a long instruction after the user's dynamic message and then wonder why cache reads stay at zero. A long, unchanged system prompt, tool descriptions, and reference documents should come before changing data when the API caches prefixes.

The normalizer should preserve unknowns instead of replacing them with zero

When one gateway provides prompt_tokens, another provides input_tokens, and a third sends usageMetadata, it is tempting to write a dozen expressions with || 0. That is how reports are born in which missing detail looks like missing spending.

Zero and unknown have different meanings. Zero means the source confirmed that no tokens belong to the category. Unknown means the source did not provide the number, the gateway discarded it, or the adapter does not know how to read it yet. In a financial system, these are three different states: 0, null, and a parsing error.

Here is a TypeScript-like normalizer. It does not try to guess everything, but it leaves a trail that can be checked when a new response format appears.

type MaybeNumber = number | null;

type UsageLedger = {
  input_uncached: MaybeNumber;
  cache_read: MaybeNumber;
  cache_write: MaybeNumber;
  output_total: MaybeNumber;
  reasoning_output: MaybeNumber;
  reported_total: MaybeNumber;
  reasoning_relation: "subset" | "additive" | "unknown";
  source_schema: string;
};

function numberOrNull(value: unknown): MaybeNumber {
  return typeof value === "number" && Number.isFinite(value) ? value : null;
}

function openAIStyle(raw: any): UsageLedger {
  const u = raw.usage ?? {};
  return {
    input_uncached: numberOrNull(u.prompt_tokens),
    cache_read: numberOrNull(u.prompt_tokens_details?.cached_tokens),
    cache_write: numberOrNull(u.prompt_tokens_details?.cache_write_tokens),
    output_total: numberOrNull(u.completion_tokens),
    reasoning_output: numberOrNull(u.completion_tokens_details?.reasoning_tokens),
    reported_total: numberOrNull(u.total_tokens),
    reasoning_relation: "subset",
    source_schema: "openai-compatible-chat"
  };
}

This example still does not resolve the question of ordinary input. If prompt_tokens already includes cached input, calling the field input_uncached here is too confident. In a real adapter, call it input_reported_total until the documentation for the specific endpoint confirms the breakdown. Then calculate:

ordinary_input = input_reported_total - cache_read

only when three conditions are all true: cache read is actually included in the input total, both values refer to the same request, and the difference is not negative. If even one condition fails, return null, add an invalid_usage_relation flag, and preserve the original JSON.

For Anthropic, the formula is different. Its prompt-caching documentation defines total input as the sum of input_tokens, cache_creation_input_tokens, and cache_read_input_tokens. There, input_tokens refers to the part after the last cache breakpoint, not to all context sent by the user. If you add cache reads to "total input" and then subtract them again as a discount, you will distort the price once more.

Run invariant checks on every record:

if reasoning_output != null and output_total != null:
    assert reasoning_output <= output_total

if cache_read != null and cache_read < 0:
    reject record

if reported_total != null and input_total != null and output_total != null:
    compare reported_total with input_total + output_total

The last check should not automatically reject a request. Multimodality, tools, and provider-specific rules can change the equality. It is a signal that the schema has changed or that the adapter misunderstood the fields.

Context limits, generation limits, and rate limits answer different questions

Financial accounting is often mixed with API limits because both worlds talk about tokens. But a context limit answers the question, "Will the request and response fit in the model window?" A generation limit answers, "How much can the model spend on the current output?" A rate limit answers, "How quickly can requests or tokens be consumed over an interval?" Price answers, "How much does this cost?" The same token may participate in two or three of these calculations.

For a request with complex reasoning, it is useful to calculate the generation headroom like this:

requested_output_cap = max_completion_tokens
actual_output = visible_text + tool_arguments + reasoning + other_output
actual_output <= requested_output_cap

If the API documents that reasoning counts toward the output limit, a short user-facing answer will not save you from length. The model may spend nearly the entire ceiling on reasoning and fail to produce the final JSON. This is especially unpleasant in tool-use scenarios: the orchestrator receives an incomplete argument, repeats the request, and pays a second time.

Do not solve this by simply increasing max_tokens for every request. The move is popular because it quickly removes some errors. It also opens the door to long internal reasoning on tasks that need only brief fact extraction. Separate task classes: extraction, classification, text generation, document analysis, programming, and tool-based planning. For each class, set an output ceiling and an acceptable reasoning mode, then test quality on a fixed set of examples.

Do not derive rate limits from price either. A provider may include cache reads in its input-token limit, or it may count only tokens that reached inference. Anthropic's documentation separately warns that effective caching changes the appearance of input_tokens but does not remove the need to understand the full amount of context processed. If you build a queue around your own "ordinary input" counter, it may suddenly hit 429 errors with long cached conversations.

Streaming requires waiting for final usage

With streaming, you cannot calculate the actual cost from the length of received chunks. Text events show user-facing output, but they do not have to carry the final cache, reasoning, or even total usage details. OpenRouter returns usage for a stream once, in the final message before [DONE]. With Anthropic, some final numbers arrive in message_delta. Similar rules exist in other APIs, but the exact event order must be checked in their documentation.

Use a two-phase handler. During the stream, it collects content for the user and initially records request_started. After final usage arrives, it closes the financial record. If the connection ends earlier, mark the request as usage_pending or stream_interrupted rather than recording zero.

A practical sequence looks like this:

1. Create a record with request_id, the model, reasoning parameters, and the start time.
2. Save event content and metadata, but do not calculate the final price.
3. Read the final usage event and pass it through the schema adapter.
4. Calculate the price using the tariff version that applied to this model and route.
5. If final usage is missing, perform a delayed reconciliation against the server log or statistics endpoint, if available.

Do not use a local tokenizer as the final amount after a stream disconnects. It is useful for a preliminary limit estimate before sending the request, but it does not know about internal tokens, server-side system instructions, tool transformations, or caching rules. It is better for a report to say "cost pending reconciliation" than to show a precise-looking lie.

You should be able to break down one request to the tenge

Consider not a fictional bill, but a record format your calculation should understand. Suppose the adapter received this normalized usage:

{
  "model": "provider/model-x",
  "input_reported_total": 24000,
  "cache_read": 18000,
  "cache_write": 0,
  "output_total": 1400,
  "reasoning_output": 950,
  "reasoning_relation": "subset",
  "gateway_response_cache": "miss"
}

If the documentation for this schema confirms that cache reads are included in input_reported_total, ordinary input equals 6,000. Then apply separate rates rather than one input price:

ordinary_input_cost = 6000 * p_input
cache_read_cost     = 18000 * p_cache_read
cache_write_cost    = 0 * p_cache_write
output_cost         = 1400 * p_output
request_cost        = sum of the four lines

reasoning_output = 950 is not a fifth line in the formula. It creates two useful metrics:

reasoning_share = 950 / 1400
visible_output_approx = 1400 - 950

The first helps reveal which scenarios spend their output budget on thinking. The second helps investigate a complaint that a two-sentence answer costs as much as a long one. But do not use the second metric as a promise of how much text the user will see: it is approximate.

Now imagine that the next request returned the same text but with a gateway-level response-cache hit and zero usage. The price of this response may be zero if that is how the gateway rules work. But do not record reasoning_share = 0. Reasoning did not run for this request. The correct value is not_applicable, because the metric describes model work that did not happen.

Details like these are what break monthly reports. If you mix cache hits with ordinary requests, the average reasoning share will suddenly appear to improve. In reality, you simply increased the share of responses that did not call the model.

A dashboard should show causes, not one overall chart

A total-token chart is useful as a signal but useless for taking action. When it rises, an engineer should be able to see the source within a minute: the base context grew, cache reads failed, cache writes appeared, output increased, reasoning share rose, or routing changed to another model.

I would keep five breakdowns on the main dashboard: cost by model, ordinary input, cache reads and cache writes, total output, and reasoning share of output. Also show the number of requests with unknown detail separately. If it is growing, the problem is telemetry, not the model.

You also need strict alerting rules. Do not alert on every increase in total tokens. Alert on events that require action:

• cache writes suddenly increased while request volume stayed the same;
• cache reads fell after a prompt-template change;
• reasoning share increased for a specific task after a model change;
• a stream ended without final usage;
• your calculated price diverged from server statistics beyond the permitted rounding error.

Check routing separately. One OpenAI-compatible endpoint does not mean one counter or one tokenizer. AI Router preserves compatibility with familiar SDKs and routes requests to different models, so your record must retain the actual model and provider route, not only the name sent by the application.

The most useful habit here is a boring one: before changing a prompt, model, reasoning effort, or caching, run a short set of control requests and save usage alongside quality scores. Then the discussion is not reduced to "it got more expensive" or "the model thinks for too long". You will see exactly how much input stopped being cached, how much output went into internal reasoning, and where the configured limit cut off the response.

Do not try to make tokens identical across providers. Make them verifiable. Raw fields, explicit semantics, separate rates, final streaming reconciliation, and a ban on double-counting reasoning will give you a calculation that survives new models and the next "compatible" API.