The judge rewards length
Grading language models is hard, so the field found a shortcut: let a strong model do the grading. It’s fast, cheap, and correlates reasonably with human preference. It is also, in a specific and measurable way, not grading what everyone thinks it’s grading.
MT-Bench is one of the most-cited such leaderboards. The implicit claim is the natural one: a higher win rate means better answers. I took the benchmark’s own published judgments — the file of GPT-4 verdicts it ships with, nearly three thousand head-to-head comparisons — and asked a simpler question than “which model is better”: when this judge picks a winner, what is it actually responding to?
Two things the judge loves that aren’t quality
It loves the longer answer. Across comparisons where the two answers differed in length, the longer one won 68% of the time — not 52%, not 55%. That’s about eighteen standard deviations from the coin-flip you’d get if length didn’t matter; the probability by chance rounds to zero. Winners averaged ~1,080 characters to the losers’ ~950.
It loves its own family. When a GPT-family answer met a non-GPT one, the GPT side won 71.5% of the time. Narrow it to GPT-4 judging GPT-4’s own answers, and it picked itself in 82 of 94 head-to-heads — 87%. A model grading a contest that includes its own work is a conflict of interest, and here it’s firing at seventy-plus percent.
| What the judge tracks | Win rate | Significance |
|---|---|---|
| Longer answer wins | 68.0% | z=18.3, p≈0 |
| GPT-family vs non-GPT | 71.5% | z=22.8, p≈0 |
| GPT-4 judging its own answers | 87.2% | 82/94, length balanced 51.8% |
| Answer shown first wins | 52.6% | p≈1e-4; 12.9% flip on swap |
What this does and doesn’t prove
Being careful is the entire value of an audit. This does not prove that length causes the wins. There’s an honest confound: longer answers can also be better — more thorough, more complete. From this observational data alone I can’t fully separate the two. What it does prove is that the headline “quality” win-rate is entangled with a presentation variable the benchmark never controls for. It’s measuring quality and length, welded together, with no way for the reader to tell how much of the score is which.
The honest takeaway
None of this is an accusation of fraud, and none of it is even novel — the MT-Bench authors themselves flagged that LLM judges reward length. What the audit adds is the number, computed on their own file, so it can’t be waved away: if you quote a raw LLM-judge win rate, you are quoting a figure worth double-digit percentage points of length bias and a measurable dose of self-preference. The fixes are known and cheap — length-match the answers, use a disinterested judge or an ensemble, always run both orderings and report how often the verdict flips.
MT-Bench public GPT-4 pairwise judgments (
gpt-4_pair.jsonl), general templates, n=2,999.Longer wins 68.0% (z=18.3, p≈0); GPT-family 71.5% (z=22.8); GPT-4-on-GPT-4 87.2% (82/94, length balanced 51.8%);
position-1 52.6% (p=1e-4); 12.9% of paired verdicts flip on swap.
Correlation, not proven causation, for length; self-preference is length-controlled.
Check a preference rate yourself:
evalgate bias --wins 68 --n 100