Cutting LLM Token Costs in Production: A 2026 Field Guide.

The first month an LLM feature runs in production is usually when the finance conversation happens. A prototype that cost pennies suddenly costs four figures, and someone asks whether the feature is worth it. In my client work the answer is almost always yes — after applying a handful of cost levers that most teams simply have not pulled yet. There are five that matter, and they are not equal. Here they are ranked the way I apply them.

The five levers at a glance

Lever 1: prompt caching — the closest thing to free money

Every request you send re-processes your entire prompt: the system prompt, the tool definitions, the reference documents. If those are identical between requests — and in a typical SaaS feature they are — providers will cache the shared prefix. On Anthropic's API, cached reads cost about a tenth of the normal input price; a cache write costs 1.25× once, and every subsequent hit within the window is ~90% off. In practice, across features with a fat static prefix and a thin dynamic suffix, that lands at 75–90% savings on input spend.

The one rule: caching is a prefix match. A single changed byte early in the prompt invalidates everything after it. The classic self-inflicted wound is interpolating a timestamp or user name into the system prompt — put stable content first, volatile content last, and mark the boundary:

{
  "system": [
    {
      "type": "text",
      "text": "You are the support assistant for ... (4 KB of stable instructions)",
      "cache_control": { "type": "ephemeral" }
    }
  ],
  "messages": [
    { "role": "user", "content": "The per-request question goes here" }
  ]
}

Then verify: if the usage field for cache reads stays at zero across identical requests, something in your prefix is quietly changing — unsorted JSON, a request ID, a now() call.

Lever 2: model routing — stop paying flagship prices for easy work

Most products send every request to the biggest model out of launch-day caution. But workloads are mixtures: classifying a support ticket does not need the model that writes the nuanced reply. Small models are five to ten times cheaper per token than flagship models, so routing even half your traffic down a tier cuts 40–70% of the spend on that traffic.

The routing itself does not need to be clever. In client projects I usually start with static routing by task type — classification, extraction and formatting go to the small model; generation and reasoning go to the large one — plus an escalation path: if the small model's answer fails validation or confidence checks, retry on the big one. That is thirty lines of code in a Laravel service class or a Python worker, not an ML project.

Lever 3: the Batch API — a flat 50% for waiting

Every major provider offers batch processing at half price with results within hours (usually much faster). The only cost is latency, which is why the lever is underused: teams assume everything must be synchronous. It rarely is. Nightly report generation, embedding backfills, bulk classification, re-summarising old records — I have moved entire pipelines to batch endpoints and halved that line item in an afternoon. If a job already runs on a queue or a scheduler, it is a batch candidate.

Lever 4: context compaction — long conversations get quadratically expensive

In chat features and agent loops you resend the whole history on every turn, so cost per turn grows with conversation length. Past a threshold — I typically use a token count, not a message count — summarise the older turns into a compact context block and keep only recent messages verbatim. Savings of 30–60% on long sessions are normal, and quality often improves because the model stops drowning in stale context. Some providers now do this server-side; a simple summarise-and-truncate you control is fine too. The adjacent quick win: trim what enters the context in the first place — retrieved documents cut to relevant sections, tool outputs stripped of boilerplate.

Lever 5: output caps — the forgotten multiplier

Output tokens cost roughly five times input tokens, and models are chatty by default. Two cheap fixes: set max_tokens to what the feature actually needs rather than a generous default, and instruct for concision — structured output formats like JSON with a fixed schema are naturally shorter than prose. On a summarisation feature I audited, tightening the format alone cut output spend by a quarter.

The order to pull them

1. Caching first. Lowest effort, highest certainty, no product change.
2. Output caps second. An afternoon of prompt and parameter tuning.
3. Batch third — move every non-interactive job.
4. Routing fourth, once you can see per-task costs and know where the easy traffic is.
5. Compaction fifth, when long sessions show up in the bill.

Measure before optimising: tag every API call with feature and task, log the usage fields, and put them on a dashboard. Every project where the bill felt out of control, the real problem was that nobody could see which feature the tokens were going to.

Applied in that order, the combined effect on a typical SaaS LLM workload is a 60–80% reduction — which is usually the difference between an AI feature that gets cut and one that quietly becomes the product's best margin story. This is standard groundwork in my AI integration work.

Is your LLM bill growing faster than your usage justifies? I audit and fix exactly this on client systems — let's talk.