optimization
The corner that sparsity loves
It is the era when statisticians want models that are not just accurate but sparse — most coefficients exactly zero, so the fitted model names only the few variables that matter. The obstacle is that the natural penalty for counting nonzero terms is combinatorial and intractable, while the smooth penalty everyone knows how to optimize never drives anything cleanly to zero. The insight is geometric: a penalty on the sum of absolute values is convex, hence tractable, yet its diamond corners put the optimum on the axes. Solve this nonsmooth convex problem efficiently — the objective has kinks where the gradient does not exist — and exploit the sparsity it produces. Get it wrong and you face an NP-hard subset search, or a dense model hiding its signal in noise.
Who this problem belongs to
The two figures whose methods fit it best, out of 71 in contention.
Tibshirani's 1996 paper introducing the lasso is, quite literally, this problem: penalize regression coefficients by their absolute value sum rather than their squared sum, and the diamond-shaped constraint region's corners drive many coefficients exactly to zero, delivering sparse, interpretable models from a convex, tractable optimization. He named it, proved its basic properties, and spent decades extending it, group lasso, fused lasso, the adaptive lasso, elaborating exactly the geometric insight this problem describes. His score is not higher only because the exact numerical algorithms for solving the nonsmooth kink, coordinate descent, LARS, came substantially from his later collaborations with Friedman and Hastie rather than the 1996 paper alone. This is, without qualification, his own named contribution to the field.
Donoho's work on wavelets and, above all, compressed sensing with Candes and Tao, established the deep theory of when and why L1 minimization recovers sparse signals exactly or nearly exactly, providing the rigorous mathematical explanation for why this problem's geometric intuition, corners on the axes, actually works and when it fails. His 'basis pursuit' framework, developed with Chen and Saunders in the 1990s, is essentially the signal-processing twin of the lasso, solving the identical nonsmooth convex L1 problem for sparse recovery from underdetermined measurements. His score reflects direct, foundational co-ownership of the mathematical theory explaining why this sparsity-inducing geometry works, just adjacent to Tibshirani's specific regression framing. It is telling that when researchers trace the intellectual lineage of the modern technique this problem describes, David Donoho's contributions show up as load-bearing rather than incidental references.
Fought here
In the mind map
The same ideas, as concepts rather than history — in John's ML knowledge map.
71 figures are scored on this problem. Draw it in a battle to see where you land.