The luckiest of 23 tries

You have seen this claim a hundred times without noticing it was a claim: “our model leads on [some subset].” Not the overall benchmark — a slice of it. It sounds modest, even rigorous. It is, more often than not, the least rigorous number in the whole announcement.

The benchmark here is RewardBench, which scores a model across 23 subsets. But the lesson is about any benchmark carved into many pieces — which is to say almost all of them: MMLU’s 57 subjects, a run’s dozens of checkpoints, the N ways to compute a metric.

Why the best slice is a rigged coin

The mechanism is pure arithmetic. Suppose you compare two models that are, in truth, identical — exactly equal in ability. On any single subset, one will edge out the other by chance; noise guarantees it. Now look across all 23 subsets and report the one where model A looks best. You are no longer reporting “A’s performance.” You are reporting the maximum of 23 noisy numbers — and the maximum of many noisy numbers is biased upward even when there is nothing to find.

With 23 subsets and the usual significance bar, you should expect about 0.6 “significant” wins between two identical models, purely from noise. So one marginal subset win isn’t evidence of anything. It’s the base rate.

Two real cases, opposite answers

The pair is the whole point, because they come out opposite. Per-subset two-proportion test, then the best A-direction win corrected with Šidák, 1 − (1 − p)²³:

In the first case, two essentially tied models; A’s best-subset win looks significant on its own (p = 0.009). Once you account for it being the best of 23 tries, the significance evaporates (p = 0.19, well above the bar). It’s noise. “A beats B on subset X” is not a finding.

In the second — and this is what makes the check trustworthy rather than a naysayer — a different pair, where A is actually the weaker model overall, nonetheless shows a subset difference big enough to survive the same 23-way correction (p = 0.011). That’s a real subset-level result, and the audit certifies it, even though the overall ranking runs the other way.

It’s everywhere, and the fix is one word

Once you see the shape you can’t stop seeing it. Every “best subject,” “best checkpoint,” “best metric variant” is best-of-K selection, and every one inflates the same way. The published number looks specific and humble (“just this subset”) while being, statistically, the loosest claim in the document.

The fix is a single word: correct. Adjust the reported subset win for how many subsets you could have picked from — a Bonferroni or Šidák correction, both one line of arithmetic — and claim only the slices that survive. It costs nothing and turns “we lead on X” into either a certified result or a withdrawn one.