Concepts

Waste patterns explained

Every call gets exactly one category: useful, or one of five waste patterns. Waste is charged to the call that produced nothing, not to the call that later worked. A figure marked ≈ includes an estimate.

Retries

What it means. A call failed, timed out or was cut off, and the same request was sent again within 60 seconds.

What it costs. The failed call's cost. Providers do not bill most error responses, so the dollar figure is often small; the retry rate ("18% of calls are retries") is shown as well, because retries also cost time.

How to fix. Find why calls fail: timeouts on long answers, rate limits, or requests that will always fail. Raise the client timeout, back off on 429 responses, and stop retrying errors that cannot succeed.

Loops

What it means. Inside one run, the agent repeated itself: the same full request 4 or more times, the same tool result 4 or more times, or the same tool calls 4 or more times. The first time is useful; the repeats are waste.

What it costs. The full cost of each repeat. Measured, not estimated.

How to fix. Give the agent a step limit, and stop or change course when a tool returns the same result twice. Open a run in Run detail to see which step repeats.

Abandoned responses

What it means. A streamed answer was started and never finished, or the client gave up (timeout or cancel), and it was not retried.

What it costs. The input plus the output generated before it stopped (estimated from characters).

How to fix. Cancel requests you no longer need as early as possible, or do not start them. Check client timeouts and that your code reads streams to the end.

Cut off and redone

What it means. An answer hit the output token limit, and the same thing was asked again soon after.

What it costs. The full cost of the cut-off answer. Measured.

How to fix. Raise max_tokens for this feature, or ask for a shorter answer, so the first attempt is enough.

Prompt caching not used

What it means. The same long prompt start (system prompt and tools) was sent 20 or more times in a day, close together, with under 10% of input read from the provider's cache. Production only.

What it costs. The difference between the normal input price and the cached price on the repeated part. Estimated.

How to fix. Keep the fixed part of the prompt first and byte-for-byte the same. Anthropic needs cache_control on it. OpenAI caches prompts over 1,024 tokens automatically when the start is identical.

Growing context (not waste)

What it means. In a run of 5 or more steps, the uncached input grew at nearly every step, to at least 3 times its starting size.

What it costs. Shown separately as extra input cost. Not counted as waste, because the extra context may be needed.

How to fix. Summarise or trim the history, cache the stable part, or pass large tool results by reference.

Useful

Everything else. A call is useful unless one of the rules above applies.

Further reading