games
Teach the machine checkers
It is 1952 at IBM Poughkeepsie, and the new 701 has a few thousand words of memory, executives nervous about the word "thinking," and you have promised the machine will learn. Build a program that improves at checkers by playing itself — adjusting its own evaluation function from experience, storing what it can in memory that would not hold a single modern photograph. The technical question is whether self-play plus a tuned evaluation can climb past its programmer's own skill. The stakes are larger than a board game: this is the first public test of whether machines can improve without being told how, and a failure will be cited for decades as proof that "machine learning" was always a salesman's phrase.
Who this problem belongs to
The two figures whose methods fit it best, out of 53 in contention.
This is Samuel's actual life, at the stated place and time. Joining IBM in 1949, he wrote checkers for the 701 and by the mid-1950s had it learning two ways: rote learning, caching evaluated positions so lookahead deepened effectively over time, and 'learning by generalization,' adjusting the coefficients of a weighted-feature evaluation by comparing lookahead predictions against later ones — anticipating temporal-difference learning by three decades. He invented the memory tricks (board hashing, aggressive pruning) that made any of it fit, coined the phrase 'machine learning' in his 1959 IBM Journal paper, and the program beat Samuel himself and, in a 1962 exhibition, Robert Nealey. Held a hair below perfect only because the program ultimately plateaued short of championship play — a limit he acknowledged candidly.
Temporal-difference learning (Sutton, 1988) is the formalization of exactly what Samuel improvised: adjusting an evaluation function so earlier predictions match later ones, with Sutton explicitly crediting Samuel's checkers player as the method's ancestor. TD(λ) with linear features is a few multiply-adds per move — arguably the most compute-frugal learning algorithm of consequence ever devised, and thus almost uniquely suited to a few thousand words of 1952 memory. He proved convergence for the linear on-policy case and spent a career on the credit-assignment problem this task turns on; his 'bitter lesson' essay is a meditation on precisely this design point, general learning plus search outlasting hand-coded knowledge. The only honest discounts: he arrives with 36 years of hindsight rather than 1952 invention, and Samuel's memory-packing craft tricks would still have to be re-learned on site.
Fought here
In the mind map
The same ideas, as concepts rather than history — in John's ML knowledge map.
53 figures are scored on this problem. Draw it in a battle to see where you land.