On a WildChat-derived stream, demand for models is concentrated
How concentrated demand for models is (Herfindahl-Hirschman index, from zero to one; one means a single model takes every request)
The WildChat-derived stream, 2 models: 0.538019
The lab’s own synthetic mix, 60 models: 0.093309
The ratio of the two indexes is 5.766. With only two models in the stream, the index cannot fall below one half, so the ratio mostly reflects how few models the stream contains. The stream is derived from public conversation text; it is not a provider’s serving mix.
On a stream derived from public WildChat conversations, demand for models was several times more concentrated than the many-model mix the lab had written for its own tests; with only two models in the stream, the index cannot fall below one half, so the ratio mostly reflects how few models the stream contains
What it does not claim
The two models in the WildChat-derived stream are two hosted chat models, not many fine-tuned variants of one base model, so this shows how demand split between the two models that service offered, not how concentrated demand is across a many-model provider. WildChat is public chat traffic; a provider's mix is a different population, and nothing here samples it. The user field is a hashed network address, which can put many people behind one value. The comparison is with a mix the lab wrote, not with any published benchmark.
The limits, in the lab’s words:
the data is not in the repo: no sha, no licence
Excerpts, word for word from the published file, where the full text can be read.
A server that hosts many models, or many variants of one model, gains from sharing work between them only when requests are spread across them. The lab had planned a test on a mix of requests it wrote itself, spread over 60 models, the most requested of them taking 24.4% of requests. Before running it, the lab measured the mix on a stream derived from WildChat, a public dataset of conversations with hosted chat models, whose records name the model each request asked for.
What it shows
In this measurement: the share of requests that went to each model, on two request mixes, and how concentrated each is.
On the WildChat-derived stream, 8,000 requests went to 2 models: 63.8% to one and 36.2% to the other. Its concentration index is 0.538019. The lab's own mix, 3,000 requests over 60 models, has an index of 0.093309. The ratio of the two is 5.766. With only two models in the stream, the index cannot fall below one half, so the ratio mostly reflects how few models the stream contains.
Entropy tells the same story: 0.944433 bits for the WildChat-derived stream, against 4.55795 bits for the lab's own mix.
Why the public Mooncake trace could not answer this
The lab looked first at the public Mooncake trace, which it uses elsewhere on this site. Its requests carry timing, lengths and block identifiers, but no model name, so they cannot be split by model even after the fact. The WildChat-derived stream can.
Why it matters, and to whom
For teams that size servers to host many models, or many fine-tuned variants of one base model, the demand mix decides how much sharing between them is worth. A synthetic mix spread over many models makes that sharing look more useful than the measured stream here would. The lab's own conclusion was to state which real demand a test is meant to represent before spending on it.
Why now
Serving systems now host many fine-tuned variants of one base model from a single server, as in the S-LoRA work cited below. How much that pays depends on how requests spread across the variants, and that is a property of the traffic, which can be measured from a request log before anything is built.
How it was checked
The lab read the model named on each request of the stream, and the model assigned to each request of its own mix, and computed each mix's shares, concentration index and entropy. The extract linked below copies every figure field of the lab's results file unchanged, with that file's hash and the lab commit it was read at. The lab's later audit located every number in its claim in the results file.
How to reproduce
The extract names the lab's results file and the commit it was read at, by hash, and the lab's entry, copied in the claim file, names the command that produced it. Anyone with access to the lab's code and the WildChat-derived stream can rerun it. The lab's code is not public yet. The figures and the claim and limits are published here, and each is listed with its hash in the site's receipt manifest.
How concentration is measured
In words: take each model's share of all requests, square it, and add the squares. The index is near zero when demand is spread thinly over many models, and it is one when a single model takes every request. It is the same index competition authorities use for market shares, as the reference cited below explains. The lab also reports the Shannon entropy of the shares, in bits, which falls as demand concentrates.
The result, in the lab’s exact words
Read it with this condition: the stream is derived from real conversation text, not recorded from a serving engine: which model was asked, from which hashed network address, and when, are real, and the cache blocks are made by hashing the text. The comparison is with the lab's own synthetic mix, not with any provider's traffic. The dataset's licence is named in the lab's results file, but the stream itself is kept outside the lab's code repository, so its bytes have no published hash.
On the WildChat-derived labelled trace the model-demand HHI is 0.538019 (entropy 0.944433 bits) against the generated multimodel family, a ratio of 5.766; the real stream is derived_from_real_text, not a real trace.
Quoted word for word from the lab’s published claim and limits.
Prior art
- Zhao et al., "WildChat", a public dataset of ChatGPT conversation logs (arXiv, 2024): the public conversation dataset the stream is derived from
- U.S. Department of Justice, Antitrust Division, "Herfindahl-Hirschman Index": the concentration index used here, defined as the sum of squared shares
- Sheng et al., "S-LoRA: Serving Thousands of Concurrent LoRA Adapters" (arXiv, 2023): serving many fine-tuned variants of one base model from one server, the case a spread-out demand mix favours
Evidence
- The measured results behind this page, read from the lab’s own files at a named version.
- The lab’s current claim and limits, word for word.
- Every file this site publishes.