search
Plan when you cannot see
It is the era of agents acting in worlds they can only partly observe: a robot with noisy sensors, a diagnostic system that cannot see the disease, a machine acting before it knows the true state. Ordinary planning assumes you know where you are; here you don't. Plan over belief states — distributions across the states you might be in — choosing actions that gather information as well as progress, since sometimes the right move is to look before leaping. The belief space is vast and exact solutions intractable, so principled approximation is the real task. Get it wrong and the agent acts confidently on a state it only imagines, or dithers gathering information forever — planning under partial observability is where decision-making meets the fog of the real world.
Who this problem belongs to
The two figures whose methods fit it best, out of 63 in contention.
Bellman's dynamic programming and his own extensions to stochastic and partially observed control problems are the direct mathematical ancestor of planning over belief states: represent the world as a state, define a value function over that state, and solve recursively via the Bellman equation, exactly the machinery POMDP solvers use once the state is replaced by a belief distribution. His work on Markov decision processes under uncertainty anticipated the core structure this problem describes: acting to gather information as well as progress, sometimes choosing to look before leaping, is precisely an exploration term in a Bellman-style value function. He did not himself formalize the POMDP framework (developed later by Astrom, Sondik, and others building explicitly on his MDP foundations), which is the only reason his score is not higher, but the mathematical skeleton is substantially his.
Bertsekas's decades of work on dynamic programming and stochastic optimal control include direct, extensive treatment of partially observed Markov decision processes, belief-state representations, and the approximate dynamic programming techniques needed when exact POMDP solutions are intractable, precisely this problem's core demand for 'principled approximation.' His textbooks (Dynamic Programming and Optimal Control) contain some of the most rigorous published treatments of exactly this problem's mathematics: acting under partial observability, updating a belief distribution, and balancing information-gathering against goal progress. His neuro-dynamic programming work with Tsitsiklis also addresses how to approximate value functions when the belief space is too vast for exact solution. Few researchers on this roster have written more directly and rigorously about this problem's specific mathematical content.
Fought here
In the mind map
The same ideas, as concepts rather than history — in John's ML knowledge map.
63 figures are scored on this problem. Draw it in a battle to see where you land.