{ "artifact": "sprint44_knee_binding", "sprint": "44p", "claim": "Which constraint actually determines the knee that Sprint 2's reproducibility gate measures, and therefore what its variance is a property of.", "method": "Reads the raw sweep points already on disk. For each replicate, finds the lowest offered load ABOVE the reported knee and asks why that point was rejected: a failed SLO constraint, an in-flight growth ratio, or the client falling behind its arrival schedule. Also records what fraction of the p95 TTFT budget the knee point actually used. No new GPU time.", "sweeps": [ { "sweep": "s02_sweep_sharded", "knees": [ 32.5, 45.0, 40.0 ], "cv_pct": 15.73 }, { "sweep": "s02_sweep_tiny", "knees": [ 18.75, 16.25, 22.5 ], "cv_pct": 16.78 }, { "sweep": "s02_sweep_tiny_cpu32", "knees": [ 16.25, 22.5, 22.5 ], "cv_pct": 16.04 }, { "sweep": "s02_paired_knee", "knees": null, "cv_pct": null } ], "rows": [ { "sweep": "s02_sweep_sharded", "rep": "rep0", "knee_qps": 32.5, "ttft_p95_at_knee_ms": 108.76, "slo_ttft_p95_ms": 2000.0, "slo_budget_used_at_knee": 0.05438, "first_point_above_qps": 35.0, "bound_by": "STATIONARITY", "bound_reason": "in-flight depth grew 98.5 -> 127.6 (1.30x) across the measurement window" }, { "sweep": "s02_sweep_sharded", "rep": "rep1", "knee_qps": 45.0, "ttft_p95_at_knee_ms": 179.33, "slo_ttft_p95_ms": 2000.0, "slo_budget_used_at_knee": 0.08966, "first_point_above_qps": 50.0, "bound_by": "SLO", "bound_reason": "ttft_p95_ms" }, { "sweep": "s02_sweep_sharded", "rep": "rep2", "knee_qps": 40.0, "ttft_p95_at_knee_ms": 157.97, "slo_ttft_p95_ms": 2000.0, "slo_budget_used_at_knee": 0.07899, "first_point_above_qps": 45.0, "bound_by": "STATIONARITY", "bound_reason": "in-flight depth grew 183.0 -> 260.1 (1.42x) across the measurement window" }, { "sweep": "s02_sweep_tiny", "rep": "rep0", "knee_qps": 18.75, "ttft_p95_at_knee_ms": 70.26, "slo_ttft_p95_ms": 2000.0, "slo_budget_used_at_knee": 0.03513, "first_point_above_qps": 20.0, "bound_by": "CLIENT_LAG", "bound_reason": "the CLIENT fell behind its arrival schedule (p99 scheduling delay 12.983890000000109 ms) -- offered load was not what was configured" }, { "sweep": "s02_sweep_tiny", "rep": "rep1", "knee_qps": 16.25, "ttft_p95_at_knee_ms": 70.53, "slo_ttft_p95_ms": 2000.0, "slo_budget_used_at_knee": 0.03526, "first_point_above_qps": 17.5, "bound_by": "CLIENT_LAG", "bound_reason": "the CLIENT fell behind its arrival schedule (p99 scheduling delay 44.732849999999985 ms) -- offered load was not what was configured" }, { "sweep": "s02_sweep_tiny", "rep": "rep2", "knee_qps": 22.5, "ttft_p95_at_knee_ms": 97.3, "slo_ttft_p95_ms": 2000.0, "slo_budget_used_at_knee": 0.04865, "first_point_above_qps": 25.0, "bound_by": "CLIENT_LAG", "bound_reason": "the CLIENT fell behind its arrival schedule (p99 scheduling delay 11.20173999999999 ms) -- offered load was not what was configured" }, { "sweep": "s02_sweep_tiny_cpu32", "rep": "rep0", "knee_qps": 16.25, "ttft_p95_at_knee_ms": 70.44, "slo_ttft_p95_ms": 2000.0, "slo_budget_used_at_knee": 0.03522, "first_point_above_qps": 17.5, "bound_by": "CLIENT_LAG", "bound_reason": "the CLIENT fell behind its arrival schedule (p99 scheduling delay 18.97465999999994 ms) -- offered load was not what was configured" }, { "sweep": "s02_sweep_tiny_cpu32", "rep": "rep1", "knee_qps": 22.5, "ttft_p95_at_knee_ms": 85.3, "slo_ttft_p95_ms": 2000.0, "slo_budget_used_at_knee": 0.04265, "first_point_above_qps": 25.0, "bound_by": "CLIENT_LAG", "bound_reason": "the CLIENT fell behind its arrival schedule (p99 scheduling delay 179.31789999999944 ms) -- offered load was not what was configured" }, { "sweep": "s02_sweep_tiny_cpu32", "rep": "rep2", "knee_qps": 22.5, "ttft_p95_at_knee_ms": 89.82, "slo_ttft_p95_ms": 2000.0, "slo_budget_used_at_knee": 0.04491, "first_point_above_qps": 25.0, "bound_by": "CLIENT_LAG", "bound_reason": "the CLIENT fell behind its arrival schedule (p99 scheduling delay 108.29033999999956 ms) -- offered load was not what was configured" }, { "sweep": "s02_paired_knee", "rep": "rep3:a2", "knee_qps": 42.516875, "ttft_p95_at_knee_ms": 176.39, "slo_ttft_p95_ms": 2000.0, "slo_budget_used_at_knee": 0.0882, "first_point_above_qps": 43.641999999999996, "bound_by": "STATIONARITY", "bound_reason": "in-flight depth grew 172.8 -> 216.1 (1.25x) across the measurement window" } ], "n_replicates": 10, "bound_by_counts": { "STATIONARITY": 3, "SLO": 1, "CLIENT_LAG": 6 }, "n_slo_bound": 1, "n_stationarity_bound": 9, "n_client_lag_bound": 6, "slo_budget_used_at_knee": { "min": 0.03513, "max": 0.08966 }, "explains": "results/sprint44_variance_attribution.json -- ttft_p95 s_y falls monotonically L1 0.04191 > L2 0.03701 > L3 0.03315, so neither branch of the config's decision rule fires. A binary stationarity classifier supplies the missing amplifier.", "verdict": "THE KNEE IS NOT SLO-BOUND. In 9 of 10 replicates the first point above the knee was rejected for NON-STATIONARITY, not for violating the SLO, and at the knee itself p95 TTFT uses only 3.5-9.0% of its budget. What this program calls an SLO knee is a STATIONARITY knee. Its variance is therefore produced by a BINARY CLASSIFIER evaluated near its own boundary, not inherited from latency measurement -- which is why the L1/L2/L3 decomposition found ttft_p95 reproducible to 3.3% at the container level while the knee derived from it varies 15-20%, and why NEITHER prescribed fix (more replicates, pin engine state) addresses it. Worse: 6 of 10 replicates were bounded by the CLIENT falling behind its own arrival schedule -- the harness measuring itself, the same mechanism Sprint 2 caught manufacturing a 1.1524x phantom.", "does_not_prove": [ "It does NOT show find_knee is wrong to exclude non-stationary points. A queue growing without bound genuinely cannot sustain that load, and refusing to treat such a point as a latency measurement is correct. The defect is that the result is then reported, gated and compared as an SLO knee.", "It does not measure how much of the knee's CV the classifier contributes. That needs a sweep instrumented to record the classifier's margin at each point, which no artifact on disk carries.", "Three sweeps, all Qwen-family on one GPU class, all at slo_name=nominal. Sprint 2's E3 (SLO sensitivity) was never run and every one of the 151 rows in this repo is nominal -- so 'the SLO is never binding' is established for THIS SLO, and a tighter one might bind.", "The client-lag finding is a property of the harness configuration (shards, CPU) at the time those sweeps ran, not a permanent property of the code.", "It covers every artifact on disk carrying knee POINTS. Artifacts whose knee is recorded without its ladder cannot be classified and are silently absent rather than counted as SLO-bound." ], "cost_usd": 0.0, "generated_utc": "2026-08-28T00:47:11.930933+00:00", "git_rev": "db27da4", "git_dirty": true, "run_id": "06501a2175a54814", "host_kind": "Darwin-arm64", "provenance_schema": "gpu-cert-provenance/1" }