What is prompt caching?
5 MIN READ · UPDATED 2026-08The feature quietly making AI subscriptions cheaper to run — how it works, why it matters, and what LADLE does with the savings.
Every turn in a chat sends the entire prior context back to the model — system prompt, all previous messages, any attached files. The model re-processes all of it from scratch to figure out what to say next. For long chats or heavy system prompts, this is a lot of compute paying for the same input tokens over and over.
Prompt caching addresses this specific waste. The idea: mark parts of the prompt as "cacheable." The first request writes a processed representation to a short-lived cache; subsequent requests hitting the same prefix read from the cache at a fraction of the base cost.
The mechanics (in plain English)
When the model processes your prompt, it builds an internal representation of every token — what it means, how it relates to nearby tokens, what to attend to. That representation is expensive to compute. It's also stable — if you send the same 10,000 tokens twice, the internal representation is identical both times.
Caching stores that representation between requests. When you send the same prefix again (within the cache TTL — typically 5 minutes), the model retrieves the stored representation instead of recomputing it. Only the NEW tokens in your latest turn need fresh processing.
The pricing
Anthropic prices cache writes at 1.25× base input cost (you pay a small premium the first time to seed the cache) and cache reads at 0.1× base input cost (a 90% discount on the read).
The math for a typical 5-turn chat where the system prompt (~1000 tokens) is the cacheable prefix: - Turn 1: pays the 1.25× write premium. Slight loss vs no caching. - Turns 2-5: pay only 0.1× for those 1000 tokens. Big win. - Net across 5 turns: ~40-55% savings on the system-prompt portion of input tokens.
What LADLE does with the savings
The 2026-08 margin pass enabled caching on every interactive LADLE chat. The savings land as MARGIN, not as raised user-facing budgets. That's a deliberate choice — it buffers future Anthropic price hikes, protects the meal-donation floor of $8/mo per subscription, and gives ops runway.
The next 30 days of production usage will produce real cache-hit data. We'll revisit whether to pass some of the savings through as more per-subscriber capacity (fund more turns before hitting the monthly cap) or keep them as margin.
Caveats worth knowing
- **Minimum cacheable prefix.** Claude Sonnet requires at least 1024 tokens to cache; Claude Haiku requires 4096. Short prompts silently skip caching. Not an error, just a no-op. - **Cache is content-keyed.** If your system prompt is IDENTICAL between two different users, they can share the same cache entry — real cost win at scale. - **TTL is short.** 5 minutes default. Conversations with long gaps between turns miss the cache and pay full price. A 1-hour cache write is available at 2× base but rarely worth it for chat use.
- Prompt caching stores the processed prompt prefix between turns so the model doesn't reprocess the same tokens every time
- Cache reads cost 10% of base input — a 90% discount on the cached portion
- Real savings on a 5-turn Sonnet chat: 40-55% on input tokens
- LADLE captures the savings as margin (2026-08) rather than raising per-subscriber budgets — revisit with real data
- Silent no-op below the minimum cacheable prefix size (1024 tokens Sonnet, 4096 Haiku)