You can't optimize what you can't see. AI cost is a hard thing to see because it doesn't arrive as one big invoice; it accrues a fraction of a cent at a time, across thousands of calls, spread over dozens of agents and features. By the time it shows up on a monthly statement it's a single number with no story attached. The fix isn't discipline or guesswork. It's instrumentation. Log the spend where it happens, attribute it to what caused it, and the story writes itself.
The number on the invoice is the wrong number
When people first look at AI cost, they look at the total. The total tells you almost nothing. It can't tell you that one agent is responsible for 70 percent of it, or that a single misconfigured loop doubled your spend last Tuesday, or that you're paying full price for context you're re-sending on every call. A monthly total is a smoke alarm with no display: it tells you something is on fire, eventually, but not where.
I run a fleet of autonomous agents in production, scheduled on Quartz, backed by a Spring Boot MCP server. Some fire every ten seconds. Some fire once a day. Early on, the only cost signal I had was the provider dashboard, and it was useless for decisions. A spend number with no attribution is a number you can worry about but not act on. So the first thing I built was not an optimization. It was a ledger.
Log tokens in and out, attributed to the spender
The unit of instrumentation is the individual model call. Every call has an input token count and an output token count, and the API returns both in the response. Capture them. Then, and this is the part people skip, attach the identity of whatever spent them: the agent name, the feature, the workflow, the scheduled job. Tokens with no owner are tokens you can't manage.
Concretely, every call in my system logs a small record:
- Which agent or workflow fired. The CRM email sync agent, the planner processor, the LinkedIn inbox processor. Each has a name and that name goes in the log.
- Input tokens and output tokens, separately. They price differently and they move for different reasons. Output tokens are usually the more expensive of the two, and a prompt that rambles in its response costs more than one that answers tightly.
- Cache reads versus fresh input. More on this below; it's the single biggest lever.
- Model used. So I can see when something expensive is running work that something cheap could handle.
Once every call carries that tag, cost stops being a mystery. You can group by agent and see, in one query, that your inbox processor is quietly the most expensive thing you run because it fires constantly. That's a fact you can act on. The invoice total never told you that.
Prompt cache hit rate is the metric to watch
If you're running agents with large, stable system prompts, prompt caching is where most of your money is won or lost. A cached read costs roughly a tenth of what it costs to write that same context fresh. The difference between an agent that reuses its cache and one that doesn't is close to an order of magnitude on the input side.
The catch is the TTL. Prompt caches expire; the default window I work with is five minutes. That single fact ties your cost directly to your firing frequency. If an agent runs every three minutes, each run lands inside the previous run's cache window and reads the system prompt cheap. If it runs every seven minutes, the cache has expired by the time it fires again, and every single run pays full price to re-write the entire system prompt. Same agent, same prompt, same work, and the only thing you changed was the interval; the cost can differ by a factor of several.
So track your cache hit rate as a first-class metric, per agent. If an agent's hit rate is low, you have two levers: raise its firing frequency so runs fall inside the cache window, or restructure it so the stable, cacheable part of its prompt is genuinely stable and sits at the front where it can be cached. I tune my scheduler intervals against the cache TTL deliberately. A job that could run every six minutes sometimes runs every four instead, because the extra runs are nearly free while a slower cadence pays the full write cost every time. That's a decision you can only make if you're measuring the hit rate.
The usual cost sinks
Once you can see spend by agent, the same few culprits show up again and again.
Dragging huge irrelevant context along. The easiest way to waste money is to stuff the whole conversation, the whole file, the whole document set into every call when the model needs a fraction of it. Input tokens are cheap per unit and ruinous in bulk. If an agent is expensive and its output is small, look at what you're feeding it. Usually most of it's dead weight.
Re-sending the same system prompt uncached. This is the cache-miss problem wearing different clothes. If your framework rebuilds the prompt slightly differently each call, or your firing cadence outruns the TTL, you pay the full write cost every time for text that never changed. Stabilize it and cache it.
Running everything on the most expensive model. Opus is the right tool for planning and review and hard reasoning. It's the wrong tool for parsing a predictable email into structured fields, or for bulk code generation where a fast model does the job. Paying top-tier per-token rates for work a cheaper model handles well is pure waste, and it's invisible until you attribute cost by model. I cover the actual decision in matching the model to the task; the point here is that your cost log is what tells you which agents are on the wrong model.
Alert on anomalies, because a runaway shows up as spend first
A stuck agent, an infinite retry loop, a prompt that started pulling in ten times the context it should: these almost always announce themselves as a spend spike before they announce themselves as a broken feature. A loop that re-queries and re-queries burns tokens fast and quietly. If you're only checking cost at the end of the month, you find out after it has run up a bill for weeks.
So set alerts on the derivative, not just the total. Cost per hour for a given agent, compared to its normal baseline. A run that consumes far more tokens than that agent's typical run. A daily total that jumps outside its usual band. When one of those trips, you look, and you usually find a real bug, not just an expensive day. My agents log their per-run behavior, so an outlier is visible against the pattern rather than buried in an aggregate. The spend spike is the earliest, cheapest warning you will get that something is wrong; wire it up so it reaches you.
Cost per useful outcome is the metric that matters
Raw token count is a tempting metric because it's easy to measure, but it's the wrong one to optimize. The goal isn't to spend fewer tokens. The goal is to spend fewer tokens per useful outcome: per email correctly filed, per proposal drafted, per pull request reviewed, per lead processed. An agent that uses more tokens but resolves the task in one pass is cheaper than a frugal one that fails and has to be re-run, or worse, produces junk a human has to fix.
This reframes every optimization above. You raise the cache hit rate not to hit a number but to lower the cost of each real result. You move work to a cheaper model only where the cheaper model still produces the outcome. You trim context to what the task needs, not below it. The instrumentation gives you the numerator, tokens spent; you have to define the denominator, useful work done, for your own system. Once you divide one by the other, you can tell the difference between an agent that's expensive because it does a lot and an agent that's expensive because it's wasteful. Those look identical on the invoice. They're not the same problem, and only one of them is worth your time to fix.
Cost visibility isn't a spreadsheet you build once. It's a property you design into the system: every call tagged, every agent measured, every anomaly surfaced. Do that, and the money stops hiding a fraction of a cent at a time, and starts telling you exactly where it goes.