optimization
Sketch the enormous matrix
It is the era of data too big to fit in memory, and you are handed a matrix so enormous it cannot be held all at once, let alone factored by classical methods that assume random access to every entry. You still need its dominant structure — the top singular vectors that capture most of its action. Approximate them using a few randomized passes, multiplying by random test matrices to sketch the range, with provable guarantees on the error. Randomness buys scale that exactness cannot. Get it wrong and either the computation is simply infeasible and the analysis never happens, or your sketch silently loses the structure you needed and every downstream result is corrupted — randomized methods are what make linear algebra possible when the matrix dwarfs the machine.
Who this problem belongs to
The two figures whose methods fit it best, out of 71 in contention.
This problem is essentially a description of Mahoney's own research program. From the mid-2000s onward, with Drineas and others, he built randomized numerical linear algebra into a field: CUR decompositions, leverage-score sampling, and fast approximate matrix multiplication, all with the provable additive- and relative-error guarantees the problem demands. His surveys codified exactly this recipe — multiply the enormous matrix by random test matrices, sketch its range in a few passes, then compute the dominant subspace from the small sketch. He worked the method on real out-of-core data (internet graphs, human genetics) rather than toy matrices, and later connected sketching to implicit regularization and second-order optimization in machine learning. The era gap is zero and the fit is exact; within this batch he is the specialist the problem was written for.
Tao's toolkit sits directly under the guarantees this problem requires. His work with Candes in the mid-2000s on compressed sensing showed that random measurement matrices satisfy the restricted isometry property with high probability — the same concentration-of-measure phenomenon that explains why a random test matrix captures the range of an enormous one. His deep contributions to random matrix theory, including universality results for spectra, are the analytic machinery behind error bounds for randomized sketches. Tao is a pure mathematician, not a numerical software author: he would prove that the sketch preserves the top singular subspace rather than engineer the out-of-core two-pass implementation. But randomized low-rank approximation is a theorem-driven field, and the theorems are squarely his kind — he supplies the hard part.
Fought here
In the mind map
The same ideas, as concepts rather than history — in John's ML knowledge map.
71 figures are scored on this problem. Draw it in a battle to see where you land.