regression
Regression with lies in it
It is the 1960s, and data is arriving from instruments and keypunch operators that both, occasionally, lie — a sensor glitches, a finger slips a decimal — and roughly one observation in ten is quietly corrupted, with no flag telling you which. Fit the underlying trend anyway. The classical least-squares line, which trusts every point equally, will chase the outliers and bend away from the truth: a single wild value can drag the whole fit. You need an estimator with a high breakdown point, one that resists a substantial fraction of garbage without collapsing. Get it wrong and the corruption sets the conclusion — a trend that isn't there, or a real one masked — and in an age of automatic data collection, the liars only multiply.
Who this problem belongs to
The two figures whose methods fit it best, out of 60 in contention.
Tukey is the closest thing this roster has to a founder of resistant regression. From the 1960s onward, working exactly in this era, he built exploratory methods designed to not trust every point equally: the biweight (bisquare) estimator, median-polish, trimmed and Winsorized means, and the whole vocabulary of 'resistance' and 'robustness' as engineering goals for real dirty data. His premise was precisely this problem's premise, that instruments and human operators produce occasional wild values, so a good estimator must down-weight the tails automatically. Redescending M-estimators with a high breakdown point are his native tools. He would not need to invent anything to fit a trend through ten-percent contamination; he would reach for the biweight and iterate. This is his home ground.
Bickel's research program is robust and adaptive estimation, which is this problem stated as theory. Beginning in the late 1960s and 1970s he studied one-step M-estimators, adaptive procedures that learn the error distribution, and the rigorous asymptotics of estimators built to survive heavy-tailed or contaminated noise. He understood influence functions and breakdown behavior at the mathematical level a student would need to certify that a fit resists a substantial fraction of garbage. Against ten-percent corruption of unknown location he would deploy a redescending M-estimator or a trimmed procedure and prove its efficiency and resistance simultaneously. His semiparametric efficiency work also frames how much you must give up for robustness. Few people on this list combine the practical estimator and the theory of why it holds as tightly as Bickel.
In the mind map
The same ideas, as concepts rather than history — in John's ML knowledge map.
60 figures are scored on this problem. Draw it in a battle to see where you land.