AxiomLimit blog https://axiomlimit.com/blog/ Plain-English posts on KV-cache sharing, hit-rate ceilings, pooled placement and network reordering, each tied to a published result. en Thu, 01 Oct 2026 00:00:00 GMT A published cache hit rate, recomputed from the publisher's own files https://axiomlimit.com/blog/recomputing-a-published-hit-rate/ https://axiomlimit.com/blog/recomputing-a-published-hit-rate/ Thu, 01 Oct 2026 00:00:00 GMT Evidence A published prefix-cache hit rate recomputed from the publisher's own dataset matched, under controls that had to fail first and a pass mark fixed in advance. What a KV cache is, and why some AI providers share it between customers https://axiomlimit.com/blog/what-is-a-kv-cache/ https://axiomlimit.com/blog/what-is-a-kv-cache/ Wed, 30 Sep 2026 00:00:00 GMT Explainer A key-value cache saves a language model's work on a prompt, and some providers share it because many prompts start with text that others have already paid for. When shared caches leak: how one customer's prompt can show up for another https://axiomlimit.com/blog/when-shared-caches-leak/ https://axiomlimit.com/blog/when-shared-caches-leak/ Wed, 30 Sep 2026 00:00:00 GMT Security A shared cache can tell one customer whether another sent a given prompt opening, and a per-customer salt, built into vLLM, stops exact-match reuse across customers when every request carries one. No demand cache that starts empty can beat your traffic: the hit-rate ceiling on a public trace https://axiomlimit.com/blog/cache-hit-rate-ceiling/ https://axiomlimit.com/blog/cache-hit-rate-ceiling/ Wed, 30 Sep 2026 00:00:00 GMT Evidence On a test slice of our excerpt of a public trace of real requests, no demand cache that starts empty can beat a proven hit-rate ceiling, and a simulated least-recently-used cache lands far below it. Where should the KV cache live? Pooled placement across a GPU fleet https://axiomlimit.com/blog/where-should-the-kv-cache-live/ https://axiomlimit.com/blog/where-should-the-kv-cache-live/ Wed, 30 Sep 2026 00:00:00 GMT Systems On one fixed test instance, pooling cached work across a fleet reached the fewest misses the lab's model allows, and an outside solver confirmed it. Out-of-order data and the memory it costs: what a receiver must remember https://axiomlimit.com/blog/out-of-order-data-receiver-memory/ https://axiomlimit.com/blog/out-of-order-data-receiver-memory/ Wed, 30 Sep 2026 00:00:00 GMT Networking When a network spreads a transfer over many paths, data arrives out of order, and a receiver that tracks the whole window needs memory that grows with it.