Someone recently posted real numbers for DeepSeek V4 Flash:
- Direct API, cached: $0.0028 / 1M tokens
- Through a multi-provider router, flat: $0.09 / 1M tokens
That’s 32x. Same model, same weights. The difference is one thing almost nobody accounts for: cache locality.
Caching is most of your bill
Agentic coding workloads are front-loaded. Your system prompt, tool definitions, and repo context can run 30K–130K tokens — and they’re nearly identical on every turn. Only the tail changes. In a typical coding-agent session, 70–95% of input tokens are that same static prefix, repeated turn after turn.
Prompt caching exists for exactly this. The provider keeps your prefix “warm” and charges a fraction to re-read it instead of full input price. On a cache-heavy workload, hitting that cache is the difference between a sane bill and a brutal one.
How routing quietly breaks it
Here’s the part most people miss: a cache is per-provider. DeepSeek’s cache lives on DeepSeek’s infrastructure. Novita’s lives on Novita’s. They don’t share.
Multi-provider routers load-balance the same model across providers to optimize throughput and uptime. That’s genuinely good for availability — but it means request #2 often lands on a different provider than request #1. That provider has never seen your prefix. Cold cache. You pay full input price, every single turn. One user reported a sub-5% cache-hit rate through a rotating router versus 99.5% going direct.
You’re not being overcharged on the rate card. You’re just never hitting the cache the rate card quietly assumes you will.
The fix exists — but you have to go digging for it
On a router that rotates providers, you can pin this down. The workarounds people actually use:
- Whitelist a single provider per model (
provider.orderwithallow_fallbacks: false) - Lock provider and caching config into a preset
- Set a
session_id, or keep your first system message and first user message byte-identical so the sticky-routing hash stays stable
These work. But they’re manual, per-model, per-key, and buried in the docs. You only find them after your bill has already blown up and you go hunting for why. As one frustrated developer put it: it should be a default, so users “fall into the pit of success.” Today you fall into the pit of a 32x bill until you discover the config.
Cache locality should be the default
This is the principle Dynoyard is built on. We pin one provider per model. No rotation, no provider.order JSON to discover, no cache going cold between turns — your prefix stays warm by default. You point your coding agent at a /v1 endpoint and you actually hit the cache instead of paying cold every turn.
Same open models you’d reach for anyway — GLM-5.2, Qwen3.7, Kimi K2.7, DeepSeek V4, MiniMax M3 — behind one OpenAI-compatible endpoint, with the cache kept where it belongs. No config archaeology required.
If you’re running agents on open models and your bill looks an order of magnitude bigger than the cached rate card promised, this is almost certainly why. Check your cache-hit rate first. It’s the cheapest line item you’re probably not measuring.