Every LLM bill is calls × tokens × price. Each lever below cuts one of those three — and cost per completed task tells you whether it worked.
Providers now discount repeated prompt prefixes and work that can wait, smaller models handle most routine steps, and most prompts carry tokens nobody needs. This guide shows how to measure first, which lever cuts which part of the bill, what each one saves according to the providers’ own documentation, and the order to apply them.
Updated · 11 sources
To reduce LLM costs, first measure cost per completed task, then cut the three factors of the bill: the price per token (prompt caching, batch APIs), the tokens per call (trimmed prompts, retrieved context that fits, shorter outputs) and the number of calls (response caching, routing routine steps to smaller models, fewer retries). Knowing how to reduce LLM costs safely means re-running an evaluation set after each change, so savings never come from worse answers.
Cost per token is the provider’s unit. Cost per completed task — an invoice processed, a ticket answered — is yours. A cheaper model that needs two retries and a longer prompt can cost more per task than the expensive one.
Team, application, feature and run on each model call — at the gateway, so nothing is missed.
Input tokens, cached input tokens, output tokens and calls per task.
One price table applied to every call, including reasoning tokens.
Retries, refusals and abandoned runs cost money and finish nothing.
Spend over tasks completed gives cost per task.
Keep an evaluation score next to the cost, so a cut that hurts answers shows up.
A support assistant answers one ticket in 3 model calls.
Each call sends 12,000 input tokens and gets 600 back.
Input: 3 × 12,000 = 36,000 tokens × $2.50/M = $0.090
Output: 3 × 600 = 1,800 tokens × $10.00/M = $0.018
Cost per ticket = $0.108
If 10,000 of each prompt is a fixed instruction block
that a provider caches at 0.1× the input price:
Cached: 3 × 10,000 × $0.25/M = $0.0075
Fresh: 3 × 2,000 × $2.50/M = $0.0150
Output (unchanged) = $0.0180
Cost per ticket = $0.0405 (−62.5%)
(ignoring the one-off cache write)Two different things share the word “cache”. Prompt caching is done by the provider and lowers the price of a repeated prefix; the model still writes a fresh answer. Response caching is done by you or your gateway and skips the call altogether.
The provider reuses its work on an identical prompt prefix — instructions, tool definitions, a long document — and bills those tokens at a discount.
A stored answer is returned for a repeated request. A semantic cache also matches prompts that are close in meaning.
Order the prompt for the cache. Caches match from the start of the prompt. Put stable instructions, tool definitions and shared reference material first, and the user’s question, timestamps and per-user details last. A date stamped at the top of the system prompt makes every call a miss.
Classifying a backlog, re-extracting fields from last year’s documents, running an evaluation set: none of it needs an answer in two seconds. The providers below sell the same tokens at half price when you submit them as a batch.
Batches have their own quota. OpenAI and Azure both keep batch tokens apart from your real-time rate limits, so moving bulk work to a batch also frees capacity for the people using your assistants.
Most agent runs mix hard steps with routine ones — classify, extract a date, format the reply. Sending every step to the largest reasoning model pays top price for work a smaller model can often do as well — which your evaluation set confirms or refutes.
Prove the switch with evaluations. Research results are averages on public benchmarks. Before moving a step to a smaller model, run your own evaluation set on both and compare the score and the cost per task side by side.
Every token sent is paid for on every call, and prompts grow by accretion over months of edits. Input and output both reward a regular audit.
Start with the levers that change nothing about your answers, then move to the ones that need an evaluation run to prove them.
On Botminds every agent reaches its model through one built-in LLM gateway. The gateway checks the allowance, applies the agent’s model choice, paces calls and prices every token — so each lever above has one place to live.
Spend by team and per agent shows where the next model choice should change.
Read the full design in the LLM gateway, and how models are set per agent in the Academy lesson LLMs and services.
The fastest ways to reduce LLM costs: order prompts so stable content comes first (prompt caching), send work that can wait through a batch API at half price, and stop stacked retries. None of these change your answers. Then trim prompts and route routine steps to smaller models, checking quality with an evaluation set.
Send fewer, re-ranked passages instead of every search hit, summarize long conversation history, include only the tool definitions a step needs, ask for structured output and set a maximum number of output tokens per step.
Prompt caching lets a provider reuse its work on an identical prompt prefix — system instructions, tool definitions or a long document — and bill those input tokens at a discount. The model still writes a fresh answer. OpenAI and Gemini apply it automatically on supported models; on Anthropic’s API the caller adds a cache_control field to the request.
Prompt caching discounts a repeated prefix at the provider; the model still answers. Semantic caching, usually in a gateway, returns an earlier stored answer when a new prompt is close enough in meaning, skipping the call — cheaper still, but a near-match can need a different answer.
OpenAI, Anthropic and Azure OpenAI each price batch work at half the synchronous rate, in exchange for results within 24 hours (a target, on Azure). Anthropic notes most batches finish within an hour and that the batch discount stacks with prompt caching.
Total model spend divided by the tasks completed — tickets answered, documents processed. It counts every call, retry and failed run, so it shows the real price of an outcome where cost per token does not.
Per token, yes; per task, not always. A smaller model that needs longer prompts, more calls or more retries — or whose errors a person must fix — can cost more per completed task. Compare both on your own evaluation set.
Botminds checks allowances before a run starts, prices every call from one table and shows cost per run, agent, project and workspace. Models are set per agent, proven by evaluations, and agents send a stable prompt cache key. See the LLM gateway.
Bring one process and its documents. We’ll show you the agents, the controls and the record, running on your data.