games
The dice make it learnable
It is 1991 at IBM Research, and backgammon is about to teach the field something checkers could not: a neural network, trained purely by temporal-difference learning from self-play — no expert games, no hand-tuned features to start — is climbing toward the world's best players. Build it: let the network play itself millions of games, back the final outcome through time to shape an evaluation function, and explain the uncomfortable surprise, that the dice, which seemed to make the game harder, actually smooth the learning landscape and force robust play. The stakes are the credibility of self-play itself: if the result is a backgammon quirk, TD learning stays a footnote; if the principle travels, it becomes the engine of everything from Atari to Go.
Who this problem belongs to
The two figures whose methods fit it best, out of 59 in contention.
This problem is Sutton's algorithm in action. His 1988 paper 'Learning to Predict by the Methods of Temporal Differences' introduced TD learning, backing errors between successive predictions through time to shape a value function without waiting for a final outcome, exactly the mechanism Gerald Tesauro used to train TD-Gammon at IBM starting in 1991. Sutton's framework is precisely why self-play with no expert games and no hand-tuned features could work at all: TD learning bootstraps from its own evolving estimates, letting an evaluation function improve purely from the structure of the game. His later work with Barto systematized and popularized TD-Gammon as the canonical proof that TD methods scale to genuinely hard problems, a case study their textbook returns to repeatedly. No one else on this roster supplied more of the actual mechanism this problem describes.
Barto, with Sutton, co-developed the theoretical foundations of temporal-difference learning and reinforcement learning broadly, and their joint textbook Reinforcement Learning: An Introduction treats TD-Gammon as one of its central landmark case studies, explaining in detail why backgammon's stochasticity, which this problem calls 'the dice,' actually helps TD learning by forcing broad state-space exploration and smoothing the evaluation landscape rather than hindering it. His research program, developed alongside Sutton's through the 1980s, supplied the actor-critic architectures and convergence intuitions that made a system like Tesauro's plausible before it was built. He did not personally implement TD-Gammon, which was Tesauro's specific engineering achievement at IBM, but the learning algorithm and its theoretical justification are substantially his and Sutton's joint contribution.
In the mind map
The same ideas, as concepts rather than history — in John's ML knowledge map.
59 figures are scored on this problem. Draw it in a battle to see where you land.