optimization
Prove the descent
It is the era when optimization grows up into a theory with guarantees, not just recipes. For smooth convex objectives, derive the fastest possible first-order method — one using only gradients — and then prove the harder half: that no first-order method can beat it, that your rate hits an information-theoretic wall no cleverness gets past. The lower bound is what makes the result deep; without it you never know whether a faster method is waiting to be found. Get it wrong and the field either keeps hunting for speedups that provably cannot exist, or settles for a slow method believing it optimal — matching an algorithm to a matching lower bound is what turns 'this works' into 'this is the best possible,' and closes the question for good.
Who this problem belongs to
The two figures whose methods fit it best, out of 71 in contention.
Boyd is the closest thing this list has to the problem's incumbent. Convex Optimization (with Vandenberghe, 2004) trained a generation in exactly this material; his research spans first-order methods end to end — the ADMM monograph of 2011, monotone-operator syntheses, the CVX solver lineage — and he co-authored the differential-equation analysis of Nesterov's method (Su-Boyd-Candes, 2014), a central modern lens on why acceleration works at all. Deriving the optimal method is material he has taught, proved, implemented, and re-derived from multiple viewpoints; the Nemirovski-Yudin lower bound is standard equipment in his courses. The single honest caveat: the founding theorems are others' — Nesterov's 1983 method, Nemirovski-Yudin's 1983 wall — and Boyd's genius has run more to synthesis, clarity, and computational reach than to new lower bounds. On applicability, still the benchmark.
Bertsekas has spent six decades writing the actual textbooks of this material — Nonlinear Programming, Convex Optimization Theory, Convex Optimization Algorithms — with complete proofs of gradient-method rates, and his research record includes convergence analysis of gradient, proximal, and incremental methods dating to the 1970s. Deriving the accelerated method and proving its O(1/k^2) rate is squarely inside his demonstrated practice; his texts present Nesterov's method and the complexity lower bounds of Nemirovski and Yudin with full technical command, and his work with Tsitsiklis on parallel and distributed algorithms shows the same rate-proving craft under harder conditions. The one honest qualification: the information-theoretic lower bound is not his original contribution — he is the great expositor and extender of this theory rather than the author of its wall. On applicability of actual tools, nearly unmatched.
Fought here
In the mind map
The same ideas, as concepts rather than history — in John's ML knowledge map.
71 figures are scored on this problem. Draw it in a battle to see where you land.