How to reduce your LLM API costs

· 4 min read

The fastest way to cut an OpenAI or Anthropic bill is not a cheaper model. It is to stop paying for calls that produced nothing: requests sent twice, agents repeating a step, answers cut off and asked again, and long prompts sent without caching. In most applications these are a large share of the bill, and fixing them changes nothing your users see.

This guide is a checklist, in the order that usually saves the most for the least work.

1. Find out where the money goes

Your provider invoice gives one number per model per day. It does not say which feature spent it, which customer it was for, or whether the call did anything useful. Before changing code, you need three answers:

  • Which features cost the most? A support bot and a nightly report job can use the same model and have very different bills.
  • Which calls were wasted? A call is wasted when nothing used its result: it failed and was retried, it repeated an earlier step, it was abandoned, or it was cut off and redone.
  • What is the trend? A bill that grows with usage is fine. A bill that grows faster than usage means waste is growing.

Record the model, token counts, timing and outcome of every call, with a label for the feature that made it. You do not need the prompt text for this, and keeping prompts out of your monitoring is safer. See LLM cost monitoring: what to measure.

2. Turn on prompt caching

If your requests start with the same long system prompt, tool definitions or reference document, both providers can serve that part from a cache at a fraction of the normal input price.

  • OpenAI caches automatically when the start of the prompt is identical and long enough (about 1,024 tokens on current models). Newer models also let you mark cache breakpoints and choose how long the cache lives.
  • Anthropic caches when you add cache_control, either once at the top of the request (automatic caching) or on specific content blocks. The default cache lives 5 minutes; a 1-hour option exists.

The most common reason caching does not work is a prompt that changes at the start: a timestamp, a user name or a shuffled tool list placed before the stable part. Put stable content first, changing content last. Full details: Prompt caching with OpenAI and Anthropic.

3. Stop agent loops

An agent loop is an agent calling the model again and again without making progress: the same tool call with the same arguments, the same tool result coming back, the same request resent. Every repeat is billed in full, and because agents resend their whole history each step, later repeats cost more than earlier ones.

Give every agent a step limit, stop when a tool returns the same result twice, and log the run id with every call so you can see a whole run at once. See AI agent loops: how to find and stop them.

4. Fix retries and cut-off answers

Two smaller leaks add up:

  • Retries. A request times out or hits a rate limit and your code sends it again. Most provider errors are not billed, but timeouts on long answers often are, because the model did the work. Raise timeouts for long answers and back off on rate limits instead of hammering.
  • Cut-off answers. When an answer hits max_tokens, it stops mid-sentence (finish_reason is length on OpenAI, stop_reason is max_tokens on Anthropic). If your code then asks again, you paid for the first attempt for nothing. Set the limit to fit the answers you actually need.

See Retries and cut-off answers.

5. Keep context from growing without limit

Agents and chat features usually resend the whole conversation on every call. A 20-step run whose input grows each step can cost many times more than the same work with a trimmed history. Summarise old turns, drop tool results once they have been used, and cache the stable start of the prompt.

6. Then choose models per feature

Once the waste is gone, look at model choice feature by feature. A classification step or a short extraction rarely needs the largest model. Compare cost per feature, not per model: moving one high-volume feature to a smaller model often saves more than any other change. Check the providers' current prices on their pricing pages, because they change often.

7. Use batch processing for work that can wait

Both OpenAI and Anthropic offer a batch API for requests that do not need an answer straight away, at a lower price. Nightly reports, evaluations and bulk classification are good candidates.

8. Watch the month as it happens

Set a monthly budget and look at the forecast during the month, not at the invoice after it. A loop introduced on the 3rd costs a month of spend if nobody sees it until the bill arrives.

How Zexl helps

Zexl does steps 1 to 5 for you. One import in a Python app (or a few environment variables for Claude Code) records every OpenAI and Anthropic call, without sending prompt text. Every call is sorted into useful work or one of five kinds of waste, with a dollar figure, the feature it came from, and the runs behind it. The forecast shows where the month will end.

Create a free account or read the quickstart.

See where your own AI spend goes

One import in a Python app, or five environment variables for Claude Code. Prompts never leave your servers.