Prompt Caching cuts the cost of repeated input — system prompts, tool definitions, conversation history — by up to 90%, with time-to-first-token up to 80% faster and no changes to your request code. Cache affinity now routes repeated requests to the replica already holding your context, lifting hit rates on flagship models like Kimi K3 and GLM-5.3 to 85–95%+. A new
prompt_cache_key
parameter lets you isolate cached context across teams, tenants, or end users, and a new dashboard shows your hit rate, cached vs. uncached tokens, and dollars saved, per model and over time.