Liz Zhang

LLM Engineering

Your LLM Bill Is Too High. Here’s How to Fix It (Part 4)

By Liz Zhang · · 5 min read

← Back to blog

Previously in the series

So far, the series has moved from architecture to model choice to pricing mechanics. Part 1 cut out unnecessary model calls. Part 2 pushed routine work toward the cheapest model that clears the eval. Part 3 showed how Batch, Flex, Priority, cached-input pricing, and long-context pricing can change the economics of the exact same model. Now we get to the cost driver that often hides in plain sight: reasoning tokens. This is where a response can look short in the UI and still be expensive on the bill.

Reasoning models are where budgets can quietly leak. The visible answer may look small, but the model can generate internal or separately counted reasoning tokens before, during, or between visible output. On OpenAI reasoning models, those hidden reasoning tokens occupy context-window space and are billed as output tokens. Anthropic extended thinking works similarly: summarized or omitted thinking may reduce what the user sees or waits for, but the full thinking process is still billed as output.

That means a 500-token answer may not actually be a 500-token request. It may be a 500-token answer sitting on top of thousands of hidden reasoning tokens. The fix is not to avoid reasoning models. The fix is to route reasoning only to the work where it changes task success enough to justify the cost.

The visible answer is not the full cost

The table below separates what users see from what the provider bills. The important pattern is that visible output length is a weak proxy for cost once reasoning is enabled. Small answers can be cheap, but small answers produced by high-reasoning settings can be surprisingly expensive.

Table 1 shows where reasoning tokens are most likely to become waste. Simple classification and extraction usually have constrained answer spaces, so expensive reasoning rarely changes the outcome. Complex legal, security, coding, and planning tasks are different because extra deliberation can improve correctness, but even there the budget should be capped and measured against success rate.

Figure 1 makes the hidden-cost problem visible. The user sees the light segment, but the provider may bill for the full generated output work, including hidden reasoning tokens. This is why dashboards should separate input tokens, cached input tokens, visible output tokens, and reasoning or thinking tokens instead of reporting one blended total.

Why this matters now

The current pricing makes the cost risk easy to see. If a GPT-5.4 request returns 500 visible tokens but uses 5,000 hidden reasoning tokens, the billable output is closer to 5,500 output tokens, not 500. At $15 per 1M output tokens, that example costs about $0.0825 in output charges before input cost. On GPT-5.5, the same 5,500 output tokens would cost about $0.165 in output charges because GPT-5.5 output is listed at $30 per 1M tokens. On Claude Opus 4.7, 5,500 output/thinking tokens cost about $0.1375 at $25 per MTok.

One newer detail matters for Anthropic users: Anthropic says Opus 4.7 uses a new tokenizer that may use up to 35% more tokens for the same fixed text. That does not make Opus 4.7 bad, but it means teams should measure actual usage after migration instead of assuming old token counts transfer perfectly.

When reasoning is worth it

Reasoning is valuable when the task genuinely requires multi-step judgment: hard debugging, ambiguous planning, complex agentic workflows, high-risk policy decisions, deep codebase analysis, or nuanced analysis. It is usually waste for classification, extraction, formatting, simple rewriting, schema repair, and deterministic validation. For those tasks, cheaper models, lower reasoning settings, or no extended thinking should be the default.

The practical move is to treat reasoning as a budget policy, not a default personality trait. Give routine tasks little or no reasoning. Give medium-risk tasks capped reasoning. Give high-risk tasks more reasoning only when evals show that it improves task success, and pair that with human review when the consequence of failure is serious.

Figure 2 turns reasoning into a policy ladder. The key idea is escalation. Most requests should start cheap, then move to higher reasoning only when task complexity or business risk requires it. This avoids the common anti-pattern of using the most expensive thinking mode for every request just because it feels safer.

After the policy is defined, translate it into workload defaults so engineers do not have to make a fresh judgment for every endpoint.

Table 2 turns the policy into concrete defaults. Agentic coding and tool-use workflows deserve their own category because reasoning can happen between tool calls, and tool schemas, tool results, and retries can add their own token overhead. A cost dashboard that ignores those steps will understate the real unit economics.

How to control the damage

  • Use the cheapest model and lowest reasoning setting that clears the eval.

  • Cap total generated tokens where the provider allows it. For OpenAI Responses API, max_output_tokens limits both reasoning and final output tokens.

  • For Anthropic, start extended thinking budgets at the minimum and increase only when evals show improvement. Anthropic currently documents 1,024 tokens as the minimum thinking budget, with diminishing returns possible as the budget increases.

  • Use omitted or summarized thinking for latency and UX when appropriate, but do not mistake omitted thinking for free thinking. Anthropic states that omitted thinking still bills the full thinking process.

  • Log reasoning or thinking tokens separately from visible output, then track cost per successful task instead of cost per request.

  • Route deterministic tasks away from reasoning models when a smaller model, validator, parser, or rules engine can do the job.

Rule: extended thinking should be an exception path, not the default setting for every request.

Bottom line

Reasoning tokens are a real cost driver, and short visible answers can still be expensive. Current provider docs make this clear. OpenAI explicitly documents hidden reasoning tokens as billed output tokens, and Anthropic explicitly documents full thinking tokens as billed output even when the thinking is summarized or omitted. Reasoning models are worth the money when they improve task success. If you enable reasoning everywhere, your app will quietly pay premium output-token prices for boring work.

Next in the series

Part 5 moves from token control to structural savings: prompt caching. If your agent sends the same system prompt, tools, schemas, and policies over and over, you should not be paying full input price every time. The next post shows how to structure prompts so caching actually works instead of hoping the provider magically saves you.