Translating a KV cache between engine layouts: exact, or refused

Pairs of declared cache layouts, one square per pair, by the grade the checker gave the translation between them

Exact to the byte: 21
Exact for every decoded token: 35
Runs, but not exact: 2
Refused: no exact translation exists: 6

Declared shapes with citations, checked at every position of one small cache; no engine ran.

The result

For KV cache layouts declared from engines' own documentation, the lab's checker graded every translation between them as exact, not exact or refused, checking every position of one small cache shape; no engine was run

What it does not claim

No engine, kernel or GPU ran; the shapes are declarations with citations. A made-up engine with a declared shape is graded exact just as easily, so a grade is a statement about a declared shape, not about a vendor. The index argument covers this one cache shape. It says nothing about throughput, latency, memory or cost. The lab also records that its headline count could once be inflated by a one-character change to one engine's declaration.

The limits, in the lab’s words:

A planted 'fictional-engine-9000' control earns a full EXACT_BYTE row with 1,280 positions checked — the certificate's own known-limits say a shape matching a real engine's is 'a citation, not a measurement.'

Counts must never be converted into vendor adoption.

Excerpts, word for word from the published file, where the full text can be read.

Each serving engine stores its KV cache in its own arrangement: which axis holds the tokens, the heads, the blocks, and whether keys and values sit together. Moving a cache from one engine to another means rearranging it. The lab wrote down the arrangements that engines document, as declared shapes with citations, and for each pair asked whether a pure rearrangement carries the cache across exactly, and where no such rearrangement exists.

What it shows

In this model: each engine's cache arrangement is a declared shape, taken from that engine's documentation and cited in the certificate. A translation between two shapes is checked at every position of one small cache, the same shape for every pair.

The checker graded 21 pairs exact to the byte: translated there and back, every byte returns. It graded 35 more exact for every decoded token; after the round trip they differ only in padding that no read touches. It graded 2 pairs not exact and refused 6, out of 64 rows in all. Across the rows it checked 90880 positions.

Where no exact translation exists

The refused and inexact rows are as much the result as the exact ones. A cache of compressed latents, as in the DeepSeek design cited below, has no rearrangement into a per-head cache. A shard from a model split across two GPUs holds half the heads, so there is no way back to the whole. A layout that does not fit this cache shape cannot be built for it. Moving to an eight-bit element type runs, but changes the values, so it is graded not exact.

Why it matters, and to whom

For teams that move a KV cache between engines, between a prefill server and a decode server, or into a shared store, the question is which moves can be exact and which cannot. This table separates rearrangements that preserve every value from moves that cannot, before any engine is wired up.

Why now

Serving systems now split prefill and decode across machines and share caches through layers such as LMCache, cited below, so caches cross engine boundaries more often than before. Each crossing is a layout translation.

How it was checked

The checker builds each declared shape for one small cache, translates every pair, translates back, and compares at every position. Its certificate records the counts per grade, the positions checked for every row, the reason for each refusal, and controls that must fail, including the made-up engine in the limits.

How to reproduce

The certificate is published here; its provenance gives the lab commit and the hashes of the lab's file and the served one. The claim and limits are copied word for word. The lab's code is not public yet. Each file is listed with its hash in the site's receipt manifest.

Try it: one cache, two engines

Pick the engine a cache comes from and the engine it goes to. The grade, and the reason for it, are the ones written in the published certificate for that pair. Layouts declared from each engine’s own documentation; no engine, kernel or GPU ran.

From To

The result, in the lab’s exact words

Read it with this condition: the rows are pairs of declared shapes, each with a citation to the engine's documentation; no engine, kernel or GPU ran, and nothing shows that any engine would accept the translated bytes. "Entire index space" means every position of one small cache shape, the same for every row.

A declared-layout transformation campaign reporting 21 EXACT_BYTE, 35 EXACT_TOKEN, 2 NOT_EXACT and 6 REFUSED over 64 rows, each checked over the entire index space of its pair rather than sampled.

Quoted word for word from the lab’s published claim and limits.

Prior art

Kwon et al., "Efficient Memory Management for Large Language Model Serving with PagedAttention" (arXiv, 2023): the paged KV cache layout that vLLM introduced, one of the declared shapes
DeepSeek-AI, "DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model" (arXiv, 2024): multi-head latent attention, whose compressed cache has no rearrangement to a per-head cache
LMCache, an open-source KV cache layer: moves KV cache between engines and storage tiers, where layout translation is needed

Evidence

The measured results behind this page, read from the lab’s own files at a named version.
The lab’s current claim and limits, word for word.
Every file this site publishes.