# D3 — **is the first-request divergence a prefix-cache effect? The controlled 2×2**

**Certificate:** `results/d3_determinism_2x2.json` · config `configs/d3_determinism_2x2.yaml`
· generator `modal_app/serve_vllm.py::determinism_2x2_run` · **~$1** (24 cold boots on an L4)

## Hypothesis

`results/s53_demo.json`: of six identical temperature-0 requests to a freshly booted vLLM,
the first returned different bytes and the next five were byte-identical. The mechanism was
asserted — a cache-miss full prefill and a cache-hit partial prefill reduce in different
orders and can move an argmax — and not tested: no APC-off arm, no batch-invariant arm.

Four cells, {prefix caching on, off} × {`VLLM_BATCH_INVARIANT` 0, 1}, six cold boots each,
six one-at-a-time requests per prompt per boot, two prompts. Pre-registered readings are in
the config header: cache-state dependent and closed by the flag (KNOWN → post on #40896 and
stop); cache-state dependent and *not* closed (an open gap, first controlled study);
divergence with APC off too (S53's mechanism wrong: boot-state, not cache-state); or no
divergence anywhere under serialised requests (S53 saw batch composition, not cache state).

## Result — **prefix-cache-state dependent, and the existing flag closes it**

| cell | short prompt: boots where the first request differs and the rest agree | long prompt |
|---|---:|---:|
| APC on, invariant 0 | **6 / 6** | 0 / 6 |
| APC off, invariant 0 | 0 / 6 | 0 / 6 |
| APC on, invariant 1 | **0 / 6** | 0 / 6 |
| APC off, invariant 1 | 0 / 6 | 0 / 6 |

Across all 24 boots and 288 requests: the warm response was the same bytes on every boot
(one distinct warm completion per cell and prompt), and the divergent first response was
also the same bytes on every boot it appeared — this is deterministic cold-start divergence,
not noise. It appears only with prefix caching on, only on the short prompt (24 tokens; the
four-sentence prompt never diverged in any cell), and `VLLM_BATCH_INVARIANT=1` removes it
entirely. 895 s across four L4s, ~$0.80.

**Disposition — KNOWN.** S53's asserted mechanism (cache-hit vs cache-miss prefill reduce
differently) is confirmed by the APC-off control, and vLLM's batch-invariant mode already
closes it. The contribution is the controlled table, which vLLM issue #40896 lacks; posting it
there is an action against public content and is **owner-gated** (`plan_status` D3-post).
The prompt-length dependence is a note, not a finding: one short and one long prompt.

W1's "byte-identical outputs" clause gains its qualifier with a cause: *warm engine, or
batch-invariant mode.*

## Prior art

*Searched 2026-09-02.*

- vLLM issue #40896 — https://github.com/vllm-project/vllm/issues/40896 — (Apr 2026, open)
  identical symptom on Qwen3-8B/H100: first request after boot differs, later ones stable,
  `--no-enable-prefix-caching` removes it. No maintainer root cause.
- vLLM issue #33123 — https://github.com/vllm-project/vllm/issues/33123 — cache-miss vs
  cache-hit prefill paths diverge (ROCm); closed not-planned.
- Thinking Machines, *Defeating Nondeterminism in LLM Inference* (Sep 2025) —
  https://thinkingmachines.ai/blog/defeating-nondeterminism-in-llm-inference/ — names KV
  alignment so chunked and one-shot prefill match, and fixed split-KV reduction. The
  mechanism, by name.
- vLLM batch invariance — https://docs.vllm.ai/en/latest/features/batch_invariance/ —
  `VLLM_BATCH_INVARIANT=1` (beta); the docs do not state cache-hit/miss equivalence.
- LLM-42, arXiv 2601.17768 — https://arxiv.org/abs/2601.17768 — fast-path decode with
  verify-and-rollback under a fixed reduction schedule; owns "recompute on divergence."
- **What is left:** a controlled measurement isolating prefix caching as the cause on CUDA,
  and whether the shipped flag closes it. Nothing here is a mechanism claim. If the flag
  closes it, D3's entire output is a table posted on #40896.

## Does not prove

- One model, one GPU class, one pinned engine. Kernel- and version-specific.
- Serialised requests only; concurrent batch composition is the separate source the
  invariant flag is documented for and is not varied here.
- Six boots per cell bounds a frequent effect, not a rare one.