regression
When the predictors move together
It is 1970, and an economist's regression is behaving pathologically: two predictors that rise and fall almost in lockstep produce wildly unstable coefficients — huge, oppositely signed, flipping with a single added data point — even as the model predicts adequately. Ordinary least squares, handed near-collinear inputs, has no way to choose between them and answers with nonsense. Add a penalty on the size of the coefficients that trades a little bias for a large cut in variance, stabilizing the estimates into something interpretable. Justify the trade and how much to penalize. Get it wrong and you report a coefficient with a confident sign that would reverse on fresh data, and someone reads that unstable number as the causal pull of one variable holding the other fixed.
Who this problem belongs to
The two figures whose methods fit it best, out of 60 in contention.
Wahba's foundational work on smoothing splines and penalized regression, developed through the 1970s in exactly this period, formalizes precisely the move this problem requires: adding a penalty term that trades a controlled amount of bias for a large reduction in variance, stabilizing an otherwise ill-posed estimation problem into something usable. Her reproducing-kernel-Hilbert-space framework generalizes ridge-style regularization into a rigorous, broadly applicable mathematical theory, and her work with cross-validation for choosing the penalty's strength directly answers 'how much to penalize,' the exact question this economist's collinear regression poses. She was not the specific author of ridge regression itself, but she built the deeper mathematical apparatus that explains why penalization works and how to tune it correctly, working in the same decade as this problem's setting.
Tibshirani's lasso, introduced in 1996, is the direct intellectual descendant of exactly the ridge-regression idea this 1970 problem calls for: penalize the size of the coefficients to stabilize estimates when predictors are collinear, trading a little bias for a large cut in variance, except Tibshirani's L1 penalty additionally drives some coefficients to exactly zero, yielding sparse, interpretable models. His deep, career-long engagement with regularized regression as a solution to precisely this instability problem makes him unusually well matched to this scenario's substance. His contribution arrives a generation after 1970, refining and extending rather than originating the ridge idea itself, but the mathematical lineage runs directly and unmistakably through his work.
Fought here
In the mind map
The same ideas, as concepts rather than history — in John's ML knowledge map.
60 figures are scored on this problem. Draw it in a battle to see where you land.