testing
Twenty thousand tests at once
It is the turn of the 2000s, and a microarray study runs a hypothesis test for each of twenty thousand genes, asking which are differentially expressed between healthy and diseased tissue. Controlling the chance of even one false positive across twenty thousand tests, the old family-wide standard, is so conservative it would miss nearly every real gene. Instead control the false discovery rate — the expected fraction of your flagged genes that are mistakes — and defend why, at this scale, that is the honest and useful error to bound. Get it wrong and you either publish a list of hundreds of "disease genes" that is mostly noise, or tighten so hard that the handful of true signals never make the list, and the biology is lost either way.
Who this problem belongs to
The two figures whose methods fit it best, out of 46 in contention.
Efron built the exact toolkit this problem needs. Starting in the early 2000s at Stanford he developed large-scale simultaneous hypothesis testing and empirical Bayes methods explicitly for microarray data, formalizing the local false discovery rate and the two-groups model (null versus non-null gene effects) that underlies modern FDR practice. His 2004 and 2007 papers and the book Large-Scale Inference address precisely this scenario: thousands of simultaneous z-values, most null, a handful genuinely differentially expressed, and the need for an error measure that scales sensibly rather than collapsing to nothing. He also gave the empirical-Bayes justification for why Benjamini-Hochberg-style procedures behave as they do. No one on this roster mapped the actual genomics-era multiple-testing problem more directly or influentially than Efron.
Barber is a leading contemporary architect of exactly this problem's modern toolkit. With Emmanuel Candes she introduced the knockoff filter (2015), a method for controlling the false discovery rate in high-dimensional variable selection without relying on p-value calibration, directly extending the FDR framework this problem calls for into regression and genomics-scale settings. Her broader research on conformal prediction, inference after model selection, and distribution-free FDR control addresses the core tension the problem poses: how to make an honest, calibrated claim about a flagged set out of thousands of simultaneous tests. Though her career postdates the 2000s microarray moment described, her toolkit is the direct technical descendant of Benjamini-Hochberg and is what a modern statistician would actually deploy on this exact gene-expression scenario today.
In the mind map
The same ideas, as concepts rather than history — in John's ML knowledge map.
46 figures are scored on this problem. Draw it in a battle to see where you land.