optimization
Optimize across the datacenter
It is the era of data too large for one machine, and the optimization problem itself must be split across a cluster: each node holds a shard of the data and can solve its own piece, but the pieces are coupled through shared variables that must agree. Naively synchronizing every step drowns in communication. Decompose the global objective so each machine optimizes locally and the nodes reach consensus through a lightweight exchange of a few coordinating variables, converging to the same answer a single giant machine would find. Prove it converges and keep the communication affordable. Get it wrong and you either bottleneck on the network, or let the shards drift and converge to something optimal for no one — distributed optimization is what makes datacenter-scale learning possible.
Who this problem belongs to
The two figures whose methods fit it best, out of 70 in contention.
Boyd's 2011 monograph on the Alternating Direction Method of Multipliers, co-authored with Parikh, Chu, Peleato, and Eckstein, is precisely this problem: decompose a global convex objective into local pieces coupled through a small set of shared consensus variables, letting each machine solve its own subproblem and exchange only lightweight coordinating updates, with a rigorous proof of convergence to the global optimum. His broader convex-optimization program provided the duality theory underlying why this decomposition provably converges. His solvers and tutorials made ADMM the default tool for distributed machine learning and statistics at scale. His score is not the maximum only because the specific communication-efficiency engineering for datacenter deployment involves systems contributions from others.
Bertsekas's decades of research on parallel and distributed computation, culminating in his book with Tsitsiklis 'Parallel and Distributed Computation: Numerical Methods,' directly addresses exactly this problem: decomposing large-scale optimization across multiple processors with provable convergence despite asynchronous or limited communication. His work on distributed dynamic programming and network optimization engaged the same coupling-through-shared-variables structure this problem describes decades before datacenter-scale machine learning made it urgent. He and Tsitsiklis proved convergence for distributed gradient methods under weak communication assumptions, directly the 'prove it converges and keep communication affordable' requirement. His score reflects deep, foundational, direct authorship of this problem's core mathematics. That standing is why Dimitri Bertsekas would be treated as a serious voice in the room if a real engineering team sat down to actually build this solution, not merely to explain in the abstract why the problem is hard.
Fought here
In the mind map
The same ideas, as concepts rather than history — in John's ML knowledge map.
70 figures are scored on this problem. Draw it in a battle to see where you land.