information
Codes that kiss the limit
It is the mid-1990s, and Shannon's capacity theorem has sat for nearly fifty years as a promise no practical code fulfilled: reliable communication was possible up to the limit, but real codes ran well short. Then two families — one built on randomly wired parity checks decoded by passing probabilistic messages around a graph, one on interleaved convolutional codes — suddenly close nearly all of the gap. Construct codes that operate within a whisker of capacity and decode them efficiently by iterative belief propagation, and explain why this local message-passing on a sparse graph works so well. Get it wrong and deep-space links, phones, and storage all leave channel capacity unused for decades longer — instead, the theoretical limit finally becomes an engineering reality.
Who this problem belongs to
The two figures whose methods fit it best, out of 31 in contention.
Shannon's 1948 capacity theorem is the fifty-year promise this problem is explicitly about fulfilling: he proved reliable communication is possible up to a precise channel capacity limit but gave no practical construction that achieved it, a gap that stood open until LDPC and turbo codes closed it in the mid-1990s. Every concept this problem requires — channel capacity, the noisy-channel coding theorem, the very notion of 'how close to the limit' a code gets — is Shannon's own invention, and his rigorous formalization of the target is what makes the later engineering achievement precisely measurable. He did not build the specific sparse-graph, iteratively decoded constructions himself, that credit belonging to Gallager and later Berrou and Glavieux, but he defined exactly what problem they were solving.
MacKay's own 1990s research independently rediscovered and rigorously analyzed low-density parity-check codes, originally proposed by Robert Gallager in 1962 and largely forgotten until MacKay's and others' work in the mid-1990s revived them alongside the independently developed turbo codes, demonstrating they could approach Shannon's capacity limit in practice. His textbook treatment of belief propagation, sparse graphical codes, and their iterative decoding is foundational teaching material for exactly this problem's content, and his research career is centrally devoted to this exact question. Working at the heart of the mid-1990s revival this problem is set in, MacKay is not merely an expositor but a genuine historical contributor to this specific result. His research career is centrally, historically devoted to exactly the question this problem poses.
In the mind map
The same ideas, as concepts rather than history — in John's ML knowledge map.
31 figures are scored on this problem. Draw it in a battle to see where you land.