LLM Engineering
Your LLM Bill Is Too High. Here’s How to Fix It (Part 2)
The best model is the cheapest model that clears your eval for this exact task.
Previously in the series
In Part 1, we started with the most uncomfortable cost lesson in LLM engineering: many expensive calls should not exist in the first place. Before optimizing prompts or switching providers, you have to ask whether the job belongs to an LLM at all. Rules, parsers, APIs, OCR pipelines, search, and small classifiers should absorb the boring work. Now that the no-model path is on the table, the next mistake is obvious: when you do need a model, stop treating model choice like a leaderboard contest.
Teams love asking, “What is the best LLM?” That is the wrong question. Frontier models are trained to do almost everything, which means you pay for all of that capability even when you only need to extract a date, classify a support ticket, rewrite a short email, or summarize a page.
The right question is narrower and more useful: what is the cheapest model that passes your eval for this exact step?
Why leaderboards mislead production teams
Benchmarks are useful, but they are not your workload. A model that is excellent at graduate-level reasoning may be overkill for routing inbound tickets. A model that is weaker overall may be perfectly adequate for high-volume extraction. Paying for general intelligence when the step only needs a narrow behavior is how AI bills get bloated.
Rule: do not pick one model for the whole app. Pick the cheapest acceptable model for each step in the workflow.
Current model tiers worth testing
Build an eval matrix, not a vibe check
How to test properly
1. Collect real examples from production or realistic test data.
2. Define pass/fail criteria for each step.
3. Test cheap models first, then move upward only when needed.
4. Measure workflow success, not isolated model elegance.
5. Track retries and escalations, because failed cheap calls are not cheap.
Bottom line
The best LLM is not the strongest one. It is the cheapest one that reliably completes the job. Model selection is not a brand decision. It is a routing and evaluation discipline.
Next in the series
In Part 3, we go one layer deeper into pricing itself. The same model can be cheap or expensive depending on whether you call it through Standard, Batch, Flex, or Priority, and long context has a tax most teams only notice after the invoice lands. If model choice is the routing menu, pricing mode is the lane you drive it through.