Azure OpenAI is about to start charging you for the thing you did to save money.
Prompt caching — the optimization your engineering team turned on precisely because it slashed the AI input bill — is getting its own price tag. Starting this August, Azure bills for cache writes, which until now were free. The mechanism you deployed to cut costs is now a line item.
And here's the part that should actually get your attention: Azure isn't the outlier. It's the last one falling in line. OpenAI already flipped — cache writes were free on every model until the GPT-5.6 family, which now charges 1.25x the standard input rate for writes, even on automatic caching, with no opt-in. Anthropic has charged for writes since day one. AWS Bedrock has charged since GA. Google charges rent instead of a write fee, but it charges.
The free lunch is over across the board. Which means prompt caching just graduated from "engineering optimization" to "thing FinOps has to monitor, model, and defend the ROI of." Let me explain the whole notion from the top.
Every time you call an LLM, the model processes your entire prompt from scratch — the system prompt, the tool definitions, the 80-page document you attached, the full conversation history. You pay input-token rates on all of it, every single call.
But most of that content doesn't change between calls. Your system prompt is the same. The document is the same. The first 19 turns of a 20-turn conversation are the same. Reprocessing identical content on every request is pure waste — for you and for the provider.
Prompt caching fixes this. The provider stores the computed state of the repeated portion (the "prefix") the first time it sees it. On subsequent calls, instead of reprocessing those tokens at full price, it reads them back from cache at a steep discount — typically 90% off. The mechanics differ by provider, but the shape is the same everywhere: pay once to store, pay pennies to reuse.
This is genuinely one of the highest-ROI cost levers in AI workloads. Teams running agents, RAG pipelines, or long-running chat sessions routinely cut their input bill by 70–90% with it. That part hasn't changed.
What changed is that storing is no longer free.
There are now three ways providers monetize the cache, and every major provider has picked one:
The write premium (Anthropic, OpenAI, AWS Bedrock, and now Azure). Writing content into the cache costs more than a normal input token — typically 1.25x the base input rate, and up to 2x for longer-lived caches. Reads then cost roughly 0.1x. Anthropic invented this structure in August 2024. Bedrock adopted it. OpenAI held out with free automatic cache writes as a differentiator — then quietly dropped that with GPT-5.6. Azure, which inherits OpenAI's pricing with a lag, is closing the loop now.
The storage meter (Google Gemini). Google splits it differently: implicit caching is automatic and has no storage fee, but explicit caching — the kind you control — bills you per million tokens per hour the cache exists, roughly $1.00–$4.50 depending on the model. It's rent, not a write fee. The cache costs money whether or not anyone reads from it.
The hybrid (what you actually run). Most enterprises run all of the above simultaneously — Claude on Bedrock, GPT on Azure, Gemini on Vertex — which means your "caching cost" is three different billing constructs on three different invoices, none of which show up as a line called "caching mistakes."
Here's the pattern I keep seeing, and it's the same pattern as CUR 2.0, Extended Support fees, and every other quietly-introduced charge: a cost-saving mechanism becomes a cost category, and nobody is assigned to watch it.
The core issue is that caching only saves money if the cache actually gets hit. The write premium is a bet: you pay 25% extra now on the assumption you'll read that content back at 90% off, soon, before it expires. Win the bet and you save enormously. Lose it and you paid a 25% surcharge for nothing.
And losing the bet is easy. Caches expire fast — five minutes of idle time on some configurations and the whole prefix has to be re-written at the premium rate. Cache keys are brutally strict: change one character anywhere in the prefix — a timestamp in the system prompt, a reordered tool definition, a new MCP tool added mid-session — and everything downstream of that change is invalidated. Your engineers ship a harmless prompt tweak on Tuesday, and your cache hit ratio quietly collapses on Wednesday. The bill doesn't say "your prompt change broke the cache." It just says input costs went up 30%, and everyone assumes it's usage growth.
The break-even math is unforgiving at low hit ratios. On write-premium providers, a cache write that never gets read is a straight 25% tax on those tokens. Practitioners moving workloads between providers have reported breakeven hit ratios around 30% — below that, every write costs more than the reads save. On Google's storage model, an explicit cache on a large context can burn dollars per hour holding content nobody is querying.
None of this shows up in a cost dashboard that stops at "input tokens" and "output tokens." Cache writes, cache reads, cache misses, and storage-hours are separate meters, and they're the meters that determine whether your optimization is an optimization.
If your organization runs AI workloads at scale, cache hit ratio needs to move from an engineering curiosity to a monitored cost KPI — the same way commitment coverage became one for compute. Concretely, that means three things.
Instrument the split. Every major provider returns cache metrics in the API response — cache-read tokens, cache-write tokens, and what was billed at full rate. If your cost tooling collapses all of that into "input tokens," you cannot answer the only question that matters: is the caching layer net-positive this month?
Model the break-even per workload. A high-frequency agent hitting the same system prompt hundreds of times an hour is a caching slam dunk. A low-traffic internal tool with a huge cached context and four queries a day is paying a tax and calling it an optimization. These are different workloads with different answers, and the answer changes when the provider changes the multiplier — which, as of this month, they all have.
Alert on drift, not just spend. The most expensive caching failures are silent: a hit ratio that drops from 85% to 20% after a prompt change doesn't spike your bill on day one — it erodes it over weeks. A sudden surge in cache-write tokens is the signal that something upstream broke the prefix. Catching that is the difference between a one-day fix and a quarter of unexplained variance.
Every wave of cloud economics follows the same arc. A capability launches free to drive adoption. Adoption happens. The capability becomes load-bearing. Then the meter gets attached. We watched it with data transfer, with support tiers, with IPv4 addresses. Prompt caching just completed the arc faster than any of them — from launch to fully monetized across every major provider in about two years.
The lesson isn't "don't cache." Caching is still, for most real workloads, the single biggest lever on your AI input costs. The lesson is that anything with a discount and a meter attached is a FinOps surface — and as of August 21, there are no exceptions left.
The thing you did to save money now has its own ROI question. Someone in your organization should own the answer.
Original Linkedin article here.