The Prompt Caching Trap: Why Your AI Agent Pays Full Price Every Turn

Prompt caching is the least hyped feature in AI tooling — a quiet ~90% discount on repeated context that separates cheap agents from expensive ones. Then your agent breaks it, silently, and pays full price again. This is how.

How the cache actually works

Providers cache the prefix of a request when it exactly matches a previous one. Cached tokens are served at roughly 10% of list price. The key word is exact: byte-identical, from the first token onward. Change one character anywhere in the prefix — a timestamp, a file listing, a “conversation so far” summary — and the cache misses. Everything after the change is billed at full rate.

The three silent cache-breakers

1. Session compaction. Long conversation? Many agents summarize history and continue. The summary rewrites the prefix: cache invalidated, full price to rebuild everything. It’s an “optimization” that costs money every time it fires.

2. Chatty system context. If the system prompt embeds the current time, a rotating file listing, or today’s TODOs, every request is a new prefix. Well-built agents keep volatile data out of the cached prefix and reference it via tools instead.

3. Reordering. Tools that list MCP servers, skills, or files in nondeterministic order regenerate the prefix each turn. Deterministic ordering is a cost feature.

What cache-native architecture looks like

• System prompt: compact and stable — rules live in files (AGENTS.md-style), loaded on demand.
• History: never rewritten in-band; state goes to external memory the agent references.
• File access: parse (AST) and read only what’s needed, instead of re-pasting context.
• Ordering: deterministic everywhere.

Agents built this way stretch a $20 token budget across a full working day. Agents built the other way burn it in two or three hours — on the same model, same task, same provider.

What you can do today

1. Check the cache line on your bill. Providers report cached-token counts. If it’s near zero during long sessions, you’re paying the trap tax.
2. Prefer agents that keep long sessions cache-hot — our per-tool reviews note which ones do (limits pillar).
3. BYOK compounds it. Your own key + a cache-native agent is the cheapest credible setup we measure; a subscription can’t pass cache discounts through to you as transparently.
4. Restart discipline: with cache-hostile tools, fresh short sessions can ironically cost less than one long compacting session.

The caching trap is why “same model, same price” tools produce wildly different invoices. It’s architecture all the way down.