# D1 — **the $1 probe: what the engine says its own prefix cache did**

**Certificate:** `results/d1_engine_hit_rate.json` (attempt 2) ·
`results/d1_engine_hit_rate.attempt1_no_counters.json` (attempt 1, kept as evidence) ·
sealed `results/d1_engine_hit_rate.prereg.json` · config `configs/d1_engine_hit_rate.yaml` ·
**~$1.72 total** (2 × 1,588 s on an L40S) · generator
`modal_app/serve_vllm.py::paired_sweep`

## Hypothesis

`results/sprint50_offload_oracle.json` models a GPU-only LRU hit rate of **0.038678** at the
capacity S46 ran (4,096 blocks = 65,536 tokens) on the same block stream, and S46 measured
vLLM's prefix cache raising the knee **30.5%**. A 3.9% hit rate buying a 30.5% throughput gain
implies each hit is worth eight times its share of the work. Either the LRU stand-in badly
understates vLLM's prefix-tree cache, or the 30.5% is not hit-driven. The engine reports its
own hit rate on `/metrics`; no run had ever read it. **Pre-registered branches** (in the
config): rate ≤ 0.08 → APC's gain is not explained by hits, open the mechanism question;
rate ≥ 0.12 → the model understates the engine, recompute the offload ceiling with a
prefix-tree model before Track P ships a headroom; between → n=1 does not decide.

## Attempt 1 — the instrument, not the engine

S46's config at reps=1: knees **8.2904 (APC off) → 10.516637 (APC on)**, ratio 0.788 against
S46's paired median 0.766 — consistent. And `engine_prefix_cache: null` at all sixteen load
points. Every client shard scraped the counters inside `bench/replay.py::replay`;
`bench/sharded.py::_worker` returned a dict without them and the merge never rebuilt them.
Every entrypoint uses the sharded path. The scraper "added 2026-08-28" had run, for five
days, only on a path nothing called. Recorded as `NEGATIVE_RESULTS.md` Part II·IX; fixed by
carrying `engine_counters` / `engine_timeline` across the process boundary and merging them,
and by recording the raw `prefix_cache` metric lines in the artifact so the names are never
again taken from a docstring.

## Attempt 2 — the instrument again, differently

Knees **7.06 → 9.98** (ratio 0.708; S46's median 0.766, attempt 1's 0.788 — the absolute knee
moves 15% between containers, the paired ratio less, as Sprint 2 found). Counters present
this time, and **hit rate exactly 1.0 on both arms, APC off included.** The raw metric lines
recorded by attempt 1's fix explain it: vLLM 0.28 emits `vllm:prefix_cache_hits_total` *and*
`vllm:prefix_cache_hits_created` — the Prometheus creation timestamp, 1.788×10⁹ — and a prefix
match on `vllm:prefix_cache_hits` took both; the `_created` line came last in the body, and
the max-merge across shards promoted a timestamp over the count. `bench/replay.py` now parses
exact names, keeps the last value of every `prefix_cache` line by name, and is unit-tested on
the lines that run recorded (`results/d1_engine_hit_rate.attempt2_timestamp_as_counter.json`).
The `_total` lines in those first scrapes read `queries 8.0, hits 0.0` for APC on and
`0.0 / 0.0` for APC off — so the engine does count, and the control does read zero.

## Attempt 3 — my own edit

Two-point ladder, **316 s, ~$0.17, no measurement.** Both arms failed inside every shard
worker with `ReplayResult.__init__() got an unexpected keyword argument 'engine_timeline'`.
The exact-name parser and its metric table had been inserted *inside* the `ReplayResult`
dataclass body, after `engine_counters`, which orphaned the four fields that followed
(`offered_window_s`, `engine_timeline`, `drained_cleanly`, `drain_timeout_s`) into
module-level statements. The unit test I wrote for the parser passed because it never
constructed a `ReplayResult` with those fields. A dataclass edit is a schema change;
`tests/test_sharded.py::test_replay_result_still_has_every_field` now constructs one with
all ten and refuses the module otherwise. Kept as
`results/d1_engine_hit_rate.attempt3_broken_dataclass.json`.

## Attempt 4 — the number, and what it said

**APC on: 815,968 hits / 2,111,376 queries = 0.3865.** APC off: 0 / 0 — the control reads
zero. ~$0.31. Neither pre-registered branch fired, because the number is impossible: **the
trace's theoretical maximum hit rate at any capacity is 0.1687** (cold-miss floor 0.8313,
`results/sprint50_offload_oracle.json`), and the engine reported more than twice that.

The re-derivation (`bench/run_d1_analysis.py`, $0): `bench/trace_io.block_text` chose words by
`(seed·7 + j·13) mod 33`, so the text of a block depended on `seed mod 33` only. **The real
slice's 13,712 distinct block ids became 33 distinct texts.** Chain-hashing that collapsed
stream the way the engine does and running GPU-only LRU at S46's capacity gives **0.4105**;
the engine measured 0.3865. On the real id stream the same LRU gives **0.0387** — the oracle's
number to five places.

So: the offload model was right about the trace. The bridge was wrong. S45's gate — 20,000
prefix-sharing pairs, 0 violations — tested "same ids → same text" and never "different ids
→ different text". **S46's 1.3048× knee uplift was measured on a 33-text stream and is not
evidence about Mooncake.** S50 stays unauthorised. The certificate's `branch` is
`bridge_collision`, `s46_stands_as_real_trace_evidence: false`.

**Fixed, and gated.** `block_text` now keys words on a blake2b of `(id, position)`
(13,712 ids → 13,712 texts, pinned by `tests/test_trace_io.py`); the defective mapping is kept
as `legacy_block_text` for re-derivation only; `assert_distinct_blocks_have_distinct_text` is
the converse gate, run in `modal_app/serve_vllm.py` before any request is served and in
`bench/run_sprint45_bridge.py`. `configs/s46b_apc_fixed_bridge.yaml` re-runs S46 at n=1 on the
fixed bridge with the counters read; its pre-registered ceiling on the engine's rate is 0.169.

**What D1 cost and bought.** Four attempts, ~$2.20: a merge that dropped counters, a prefix
match that took a timestamp, a dataclass I broke, and then a number that exceeded a theorem.
It bought the retraction of the program's only hardware positive and the fix that makes the
next one mean something.

## Prior art

*Searched 2026-09-02.*

- vLLM prefix-caching metrics — https://docs.vllm.ai/en/latest/design/metrics/ — the engine
  exposes `vllm:prefix_cache_queries` / `vllm:prefix_cache_hits` (V1) and a GPU hit-rate
  gauge; reading them is engineering, not a contribution.
- KV Cache in the Wild, ATC'25 — https://arxiv.org/abs/2506.02634 — measured production hit
  rates; does not reconcile hit rate against throughput uplift on a fixed trace.
- **Not found:** any published reconciliation of a modelled LRU hit rate against an engine's
  measured prefix-cache hit rate *and* its throughput effect on one trace at one capacity.
  That reconciliation is the entire content of this probe; it is a measurement, not a
  mechanism, and it decides whether Track P's headroom numbers are upper bounds in one
  direction or two.

## Does not prove

- n=1 per arm on the knee; the knee ratio is a consistency check against S46, nothing more.
- The engine counts 16-token blocks; the oracle counts 64-token trace blocks. The comparison
  holds up to that granularity and the analysis states the correction.
- One model, one GPU, one trace, one capacity.