It is the deep-learning era, and you must minimize a wildly non-convex loss surface over a million parameters — a landscape of valleys, ridges, and saddle points where the global optimum is unknowable and the true gradient over all data is too costly to compute each step. Second-order information (the Hessian) is unthinkable at this size. Make it converge anyway, in practice, using cheap noisy gradient estimates from small batches — trusting that the noise itself helps escape bad points. Get it wrong and training either crawls, stalls on a plateau, or diverges outright, and the model never learns — the entire modern practice of training huge networks rests on the surprising fact that stochastic first-order methods, crude as they seem, reliably find good solutions in these million-dimensional valleys.
Chose The exact network algorithm — wrong. Reduction to a known-hard problem was the one that fit.
Karp's 1972 catalogue of NP-complete problems built the map of intractability that frames this problem — global optimization of a general nonconvex function is exactly the kind of task his reductions brand hopeless — and his later work on probabilistic analysis of algorithms pioneered studying average-case behavior when worst-case theory is too pessimistic, which is philosophically the right move here. But his subject matter is combinatorial: matchings, tours, satisfiability — discrete structures with certificates. Continuous stochastic descent, minibatch estimation, and loss-surface geometry appear nowhere in his corpus, and the theory community he led long treated neural network training as beneath rigorous attack. He explains crisply why exact solution is off the table while holding no tools for the empirical workaround the problem demands.
Santerre teaches this material at Berkeley, which is exactly the problem: he can lecture fluently on why SGD escapes saddle points, sketch Robbins-Monro on a whiteboard, and assign Bottou's papers as reading — and every one of those citations is a person in this bracket who actually did the thing. Bottou spent a career making stochastic descent work; Santerre spent a career making slides about Bottou making stochastic descent work. His practical contribution to the million-parameter valley is advising companies to hire someone like Hinton's students. In a batch containing the inventor of backprop, the analyst of its convergence, and the engineer of its infrastructure, the professor's toolkit — a well-organized syllabus and genuine enthusiasm — converges to zero faster than any learning rate schedule he could name.
Battle #33 · 8/9/2026, 6:24:27 PM · this result is deterministic: the same two personas on this problem always resolve the same way.