high-dim
The router's memory is smaller than its traffic
It is 2003, and a backbone router forwards millions of packets per second while its operators ask a question it cannot afford to answer exactly: which flows are the heavy hitters right now — the worm, the flash crowd, the billing anomaly? Storing per-flow counters would need gigabytes updated at line rate; the budget is kilobytes. Answer approximately with sketches: tiny randomized structures — hashed counter arrays — absorbing an adversarial stream in constant time per packet and answer frequency queries with additive error bounds and stated failure probability, mergeable across routers. Prove the space-accuracy trade-off is near-optimal. Get it wrong and network security operates blind at exactly the moments of overload when visibility matters most — or billing built on sloppy sketches overcharges by its own error term.
Who this problem belongs to
The two figures whose methods fit it best, out of 69 in contention.
Yao's foundational work on communication complexity, and his minimax principle for proving lower bounds on randomized algorithms, is the exact mathematical toolkit this problem's proof requirement demands: showing that a streaming sketch's space-accuracy trade-off is 'near-optimal' requires precisely the kind of information-theoretic lower-bound argument, how many bits must be communicated or stored to answer a query with bounded error, that his communication-complexity framework was built to supply. His techniques are the standard method by which streaming lower bounds are actually proven in the algorithms literature, establishing that no sketch can do fundamentally better than the hashed counter-array approach this problem describes. His score reflects that his own theoretical machinery is the field's primary tool for exactly this problem's optimality-proof requirement.
Shannon's information theory, particularly channel capacity and the fundamental limits on how much information a bounded-size representation can carry, is the direct theoretical ancestor of this problem's core question: a router with kilobytes of memory answering frequency queries about millions of packets per second is fundamentally an information-theoretic compression problem, how many bits are needed to represent an approximate answer with a stated failure probability. His foundational 1948 paper established the mathematical language, entropy, mutual information, capacity, in which the space-accuracy trade-off this problem demands characterizing is naturally expressed. His score reflects that his framework supplies the conceptual and mathematical bedrock for the entire streaming-sketch field this problem describes, developed decades before but directly generative of it.
Fought here
In the mind map
The same ideas, as concepts rather than history — in John's ML knowledge map.
69 figures are scored on this problem. Draw it in a battle to see where you land.