nlp
Predict the next word
It is 2019 in San Francisco, and a deliberately unfashionable bet is on the table: take the oldest objective in language modeling — predict the next token — and scale it beyond any precedent, on text scraped from the breadth of the internet, with no task-specific supervision at all. Model language this way and then characterize what capabilities emerge purely from that objective: translation, question answering, arithmetic, tasks never trained for, appearing at scales never reached before. The characterization must be empirical and honest — scaling curves, not press releases — because the field is about to reorganize itself around the answer. If emergence is real, one objective swallows a dozen subfields. If it is measurement error dressed as magic, the most expensive wrong turn in AI history follows.
Who this problem belongs to
The two figures whose methods fit it best, out of 53 in contention.
Radford was the lead author on GPT and GPT-2, the exact bet this problem describes: scale next-token prediction on broad internet text with no task-specific supervision, then characterize what capabilities emerge -- translation, question answering, arithmetic -- purely from that objective. His GPT-2 paper is explicitly built around empirical scaling curves and honest reporting of what zero-shot transfer does and does not achieve, precisely the discipline the problem demands to avoid press-release science. This is not an adjacent contribution or a later echo of the idea; it is the actual 2019 San Francisco work the problem is describing almost verbatim, down to the institution and the year, making him about as direct a match as this game format allows for any problem in the entire set.
Sutskever co-authored the GPT-2 paper and was a driving technical force behind the scaling bet this problem describes almost exactly: predicting the next token, scaled far beyond precedent, on broad internet text, then rigorously characterizing what capabilities emerge without task-specific training. His earlier seq2seq work and deep familiarity with sequence modeling made him uniquely positioned to recognize that scale itself, not architectural cleverness alone, might be the dominant lever, and his insistence on honest, careful empirical measurement over hype is exactly the discipline the problem demands. This is not an analogous contribution; it is a central, documented authorship role in the actual 2019 work this problem is describing. Few carriers in this entire roster have a more direct, documented authorship claim on the specific work this problem describes.
In the mind map
The same ideas, as concepts rather than history — in John's ML knowledge map.
53 figures are scored on this problem. Draw it in a battle to see where you land.