nlp
Attention replaces recurrence
It is 2017 at Google Brain, and machine translation's best models are recurrent networks with a structural disease: they process a sentence one token at a time, so training cannot parallelize along the sequence, and the hardware of the era — massively parallel accelerators — sits half idle. The proposal is surgical: remove recurrence entirely. Design a sequence-transduction architecture where every position attends directly to every other — attention as the only mechanism for moving information — with positional encodings to restore order, and say precisely why it scales: what the parallelism buys in wall-clock training, what the shortened gradient paths buy in learnability. The empirical bar is state-of-the-art translation at a fraction of the training cost. The historical stakes, invisible from 2017: everything after runs on this.
Who this problem belongs to
The two figures whose methods fit it best, out of 54 in contention.
Vaswani led the 2017 Google Brain team that wrote 'Attention Is All You Need,' the exact paper this problem describes: removing recurrence entirely, using self-attention so every position attends directly to every other, adding positional encodings to restore order, and demonstrating both the wall-clock training speedup from parallelism and state-of-the-art translation quality at lower training cost. This is not an analogous or adjacent contribution, it is the literal invention the problem is describing, down to the institution and the year. No other carrier in this batch can claim direct authorship of the specific architecture; this is as close to a perfect historical match as the game format allows for any problem here.
Sutskever's seq2seq architecture with Google established the encoder-decoder paradigm for neural machine translation that the transformer directly improved upon, and he understood intimately, from building and scaling those recurrent sequence models, exactly the sequential-processing bottleneck this problem's proposal is designed to eliminate. He was at Google Brain during the period this problem describes and was deeply engaged with exactly the research questions around scaling sequence models efficiently on parallel hardware that motivated the transformer's design. While the specific attention paper's primary authorship credit belongs to Vaswani's team, Sutskever's seq2seq work is the direct architectural predecessor this problem's proposal explicitly supersedes, and his research trajectory intersects this exact problem closely. Even so, the primary authorship credit for this specific paper belongs to Vaswani's team.
Fought here
In the mind map
The same ideas, as concepts rather than history — in John's ML knowledge map.
54 figures are scored on this problem. Draw it in a battle to see where you land.