regression
Why least squares, exactly?
It is the era when regression needs a foundation, not just a recipe, and the question is pointed: among all the ways to draw a line through noisy data, why the one that minimizes squared errors? Prove the theorem that answers it — that under a few assumptions about the errors, having mean zero, equal variance, and being uncorrelated, the least-squares estimator is the best linear unbiased one, with the smallest variance of any competitor in its class. Then be honest about the fine print: what happens when those assumptions fail, when errors are heteroscedastic or correlated. Get it wrong and you either treat least squares as sacred where its assumptions are violated, or abandon it where it is provably optimal — the theorem is what makes the default defensible.
Who this problem belongs to
The two figures whose methods fit it best, out of 65 in contention.
Gauss claimed to have used least squares as early as 1795 and published his justification in Theoria Motus in 1809, arguing that if errors are normally distributed the method that minimizes squared error coincides with the maximum-likelihood estimate, giving the method its first principled defense. He used it to predict the recovery of the asteroid Ceres from sparse astronomical observations, a spectacular practical vindication. The optimality argument this problem actually demands, that least squares is best linear unbiased without assuming normality at all, was sharpened by Markov decades later, but Gauss supplied both the method and its first serious justification, and the theorem bears his name for exactly that reason. No one else on this roster is closer to this problem's origin.
Markov rigorized the theorem that now carries his name jointly with Gauss's, presenting in his 1912 textbook on probability theory a careful proof that, under the assumptions of zero mean, equal variance, and uncorrelated errors, the least-squares estimator has the smallest variance among all linear unbiased estimators. His broader contribution to dependence in stochastic sequences also equips him unusually well to state precisely what happens when the uncorrelated-errors assumption fails, exactly the fine print this problem demands. He did not invent least squares or its first justification, which belongs to Gauss and Laplace, but the theorem's modern, fully rigorous statement and proof under minimal assumptions is substantially his contribution. That combination of applicable insight and honest limitation is what makes Andrei Markov a genuinely strong, if not the single strongest, carrier for this exact classical problem.
Fought here
In the mind map
The same ideas, as concepts rather than history — in John's ML knowledge map.
65 figures are scored on this problem. Draw it in a battle to see where you land.