# Sprint 28 — **The price list already reflects the performance difference.**

**Certificate:** `results/s28_crossgpu_A10G.json` · `results/s28_crossgpu_L40S.json` · `results/sprint28_heterogeneous.json`

**Hypothesis.** Optimally assigning prefill, decode, vision encode, and cold models across
different accelerator types beats homogeneous deployment on price/performance.

**Gate.** Continue if heterogeneous assignment beats homogeneous by **>20%** on cost.

**Cost: ~$0.30** — two GPU sweeps, because the sprint's stated input did not exist.

---

## The inputs did not exist and were measured

The sprint says *"give the optimizer 2–3 hardware types with real price/performance curves."*
**No such curves existed anywhere in this project.** Sprint 3 fitted the step-time law on one
GPU and recorded it could not say whether the structure transfers.

| GPU | $/hr | decode tok/s | prefill tok/s | **decode per $/hr** | **prefill per $/hr** |
|---|---:|---:|---:|---:|---:|
| **L4** | 0.80 | 5,309 | 7,638 | **6,637** | 9,548 |
| A10G | 1.10 | 6,115 | 11,084 | 5,559 | 10,076 |
| L40S | 1.95 | 8,959 | 19,835 | 4,595 | **10,172** |

## Result: **UNDERPOWERED — NO CONCLUSION** *(revised 2026-08-27; was KILL)*

> ### ⚠️ VERDICT REVISED 2026-08-27: **UNDERPOWERED, not KILL**
>
> This block originally read **KILL**. It cannot. Both new GPU classes were swept at
> **`reps: 1`**, and **Sprint 2 measured this same rig's absolute knee as irreproducible across
> containers at CV 15.73%** — with Sprint 1 putting the *paired* knee MDE at **34% at n=3**.
>
> At n=1 no confidence interval exists at all, so **+3.59% is not distinguishable from 0% and
> not distinguishable from 20%.** A gate can only be failed by a measurement that could have
> passed it, and this one could not have. **Re-run at n≥3 per class to decide.**
>
> The *direction* is still informative and probably right — prefill $/perf really is flat at
> 9,548 / 10,076 / 10,172 across three classes, and that is arithmetic on measured curves
> rather than a hypothesis test. **What is not supported is the word KILL.**


| workload | best homogeneous | best heterogeneous | **gain** |
|---|---|---|---:|
| chat 260/128 | L4, $13 | prefill L40S + decode L4, $12 | **+3.59%** |
| summarise 4000/200 | A10G, $120 | prefill L40S + decode L4, $118 | **+2.21%** |
| generate 200/1000 | L4, $48 | prefill L40S + decode L4, $47 | **+0.75%** |

**Best gain 3.59% against a >20% gate — measured at n=1.** The point estimate is
real; the verdict it was used to support is not. See the revision note above.

## Why, and it is visible in the table

**Prefill price/performance is nearly flat across all three classes** — 9,548 / 10,076 /
10,172 tok/s per $/hr, a **6.5% spread** across GPUs whose absolute prefill throughput differs
by **2.6×**. Decode is the only phase with real dispersion (6,637 vs 4,595, a 44% spread), and
it points at a single winner.

> **Heterogeneity pays only where the price/performance ratio differs *by phase*. Here the
> vendor has already priced the SKUs so that prefill throughput per dollar is nearly constant.**

That is the finding: the arbitrage the sprint hoped for has largely been priced out. The
optimizer correctly picks prefill=L40S + decode=L4 every time — it is just worth 1–4%.

---

## What is not proven

- **Prefill throughput is derived**, as `(TTFT − one decode step)` over input length. TTFT also
  carries scheduling and HTTP overhead, which inflates the numerator and **understates**
  prefill throughput — in the direction that makes prefill-specialised hardware look *worse*
  than it is. The true spread may be larger.
- **The pricing table is `verified: false`.** These are placeholder rates inherited from the
  estate, which itself carried A100-80GB as 3.72 / 3.40 / 2.50 across five copies. **The
  ranking survives only if the rate ratios are right**, and they are unsourced.
- **The L40S never saturated** within the sweep range — knee ≥80 qps, censored, and its fitted
  `b` came out **negative**: this model never leaves the overhead-bound regime there. Its
  decode figure is a lower bound.
- One replicate per new class, against a knee Sprint 2 measured as irreproducible at CV 15.73%.
- **One model.** A larger model changes every ratio, and the L40S's advantage grows with size.
- **Vision encode and cold-model loading are absent entirely** — two of the four phases the
  sprint names.

---

## Unexpected observation

**Sprint 3's structural claim transfers to two of three GPUs, and the third says something
more interesting.**

Sprint 3 recorded *"the constants are not portable; the STRUCTURE is the claim"* with no way to
check the second half. Now:

| GPU | `a` (ms) | `b` (ms/seq) | R² | batch range reached |
|---|---:|---:|---:|---:|
| L4 | 13.33 | 0.1055 | 0.994 | 8 → 249 |
| A10G | 7.96 | 0.0700 | 0.983 | 5 → 252 |
| **L40S** | 5.24 | **−0.0245** | 0.974 | 3 → **60** |

**The structure holds where the batch range reaches the compute-bound regime.** On the L40S
this model never gets there — 80 qps produces a batch of 60, and step time is flat at ~5 ms
throughout, so the fitted slope is noise and comes out negative.

That matters more than it looks. Sprint 3 established that `b` is **the dangerous constant** —
`d ln s/d ln b = ρ/(1−ρ)`. On the L40S, `b` is **unidentifiable from this data**, so the
simulator cannot be calibrated for that GPU at all with this model. **A faster accelerator did
not make the instrument better; it made the workload too small to measure with.**