LLM cost monitoring: what to measure
· 2 min read
LLM cost monitoring means knowing, at any time, what your AI features cost, why, and whether the money did useful work. Provider dashboards answer the first part. The other two need data from your own application.
What to measure
Per call:
- provider and model;
- input, output and cached tokens (as reported by the provider);
- cost, from the provider's current price for that model;
- latency;
- outcome: finish reason, status code, error class, whether a stream finished.
Labels on each call:
- feature: which part of your product made it;
- run: which unit of agent work it belonged to;
- customer: who it was for, as an id;
- environment: production, staging or development.
Over time:
- spend per feature, model and customer;
- useful spend share: the part of spend that produced something used, after retries, loops, abandoned and redone calls;
- cache-read share per feature;
- a month-end forecast against your budget.
The useful spend share is the number most dashboards miss. Total spend tells you the bill grew; the useful share tells you whether to worry.
Three ways to collect it
| Provider dashboard | Proxy or gateway | SDK in your app | |
|---|---|---|---|
| Setup | None | Change the base URL | One import |
| Spend per model and day | Yes | Yes | Yes |
| Spend per feature and customer | No | Through headers you add | Yes, from labels and code location |
| Agent runs and loops | No | Partly | Yes |
| Extra network hop | No | Yes | No |
| Prompts pass through a third party | No | Yes, unless you host it | No |
Many teams start with the provider dashboard, add an SDK or a proxy when the bill becomes a line item someone asks about, and add budgets and alerts last.
Do you need the prompts?
For cost, no. Everything above comes from metadata: token counts, model names, timings and outcomes. Even repeated requests can be found by comparing salted hashes of the text instead of the text itself: two identical prompts give the same hash, and nobody can turn the hash back into the prompt without the salt.
Keeping prompts out of your cost tool means one less copy of customer data, one less vendor to review, and no question about what happens if that tool is breached.
Check your numbers
Compare tracked spend with your provider invoice each month. If the tool tracks 95% or more of the invoice, you can trust its breakdowns. A big gap means some calls are not instrumented.
How Zexl does it
Zexl is an SDK for Python apps (OpenAI and Anthropic clients) and a receiver for Claude Code's telemetry. It records the fields above, labels calls by feature, run, customer and environment, and sorts every call into useful work or one of five kinds of waste. Prompt text never leaves your process: what is sent. Enter your invoices and Zexl shows how much of the bill it covers.
Create a free account or read the quickstart.
See where your own AI spend goes
One import in a Python app, or five environment variables for Claude Code. Prompts never leave your servers.