Liz Zhang

LLM Engineering

Your LLM Bill Is Too High. Here’s How to Fix It (Part 1)

By Liz Zhang · · 3 min read

← Back to blog

This series is for anyone building with LLMs who has already learned the uncomfortable truth: the demo is easy, but the bill is where reality shows up.

Across the next 6 posts, I’ll break down the cost levers that actually matter in production systems, from knowing when not to call a model at all, to choosing the cheapest model that passes your evals, using pricing modes like Batch and Flex, controlling reasoning tokens, designing smarter routing, measuring cost per successful task, and making prompt caching work for real agent workflows.

The goal is not to make AI slightly cheaper. The goal is to show how serious teams redesign the cost structure of LLM applications so expensive frontier models become the exception path, not the default path. If you are building agents, AI products, internal tools, or anything that touches LLM APIs at scale, this series will give you the practical playbook I wish more teams had before their first painful invoice.

The cheapest LLM call is the one you do not make.

Everyone building with LLMs eventually hits the same wall. The prototype works, usage climbs, and suddenly the API bill starts doing things nobody planned for. The problem is usually not that AI is expensive. The problem is that teams are using models for work that should never have touched a model in the first place.

Before you debate GPT versus Claude versus Gemini, ask a more basic question: Do you need an LLM at all?

Rule: use an LLM when the task requires ambiguity handling, judgment, synthesis, flexible natural-language generation, complex reasoning, or tool use. Do not use one just because the word “AI” looks good in the architecture diagram.

The no-model audit

A shocking amount of production LLM spend is expensive glue around work that deterministic code, dedicated APIs, or cheaper ML services already handle well.

Where teams waste money

The common pattern is simple. A team builds a general-purpose prompt, points every request at a strong model, and ships. It works, so nobody questions the architecture until the bill arrives. By then, the model has become the default path for classification, extraction, routing, formatting, translation, rewriting, and exception handling.

That is backwards. The model should not be the default path. The model should be the judgment path.

A better default architecture

1. Validate inputs with code. Reject malformed payloads before spending tokens.

2. Use deterministic tools first. Regex, parsers, lookup tables, and APIs are boring. That is why they are cheap and reliable.

3. Use small models for fuzzy but routine tasks. Classification, extraction, and rewriting usually do not need a frontier model.

4. Escalate only when confidence is low. Premium models should handle ambiguity, high-risk cases, and hard reasoning.

Practical checklist

  • Can the task be solved with deterministic code?

  • Can a dedicated API solve it more cheaply and consistently?

  • Can a small classifier handle the common path?

  • Are you sending repetitive context that could be cached?

  • Is the frontier model reserved for exception cases?

Bottom line

The first cost optimization step is not prompt compression. It is architectural honesty. Most requests are boring. Treat them that way, and the bill starts dropping before you even switch models.

What’s Next in This Series

In the next post, we’ll get into the mistake that quietly drives a lot of LLM spend: treating model selection like a leaderboard. Most teams ask, “What is the best model?” That is the wrong question. The better question is, “What is the cheapest model that reliably passes the eval for this exact step?”

We’ll break down how to think about model tiers, why frontier models should not be your default for every workflow, and how to build a practical routing menu for classification, extraction, summarization, rewriting, support replies, and agent tasks. Because once you stop calling a model for work that does not need one, the next cost lever is making sure every remaining task goes to the right model, not the most expensive one.