rl
Act on what you cannot see
It is 1995, and mobile robots in real corridors have exposed the textbook MDP's polite fiction: the robot does not know its state. Sonar stutters, walls look alike, and the true position is a hidden variable behind noisy glimpses. Formalize planning under partial observability — the agent's real state is a belief, a probability distribution over where it might be, updated after every action and observation — and face the cost: exact planning over beliefs is wildly intractable, so principled approximation is the actual deliverable. Show a robot that acts to gather information, detouring past a distinctive landmark because localization is worth the steps. Get it wrong and the robot is confidently lost — executing the optimal plan for a corridor it is not in.
Who this problem belongs to
The two figures whose methods fit it best, out of 31 in contention.
Kalman's 1960 filter is, in essence, the belief state this problem asks for: a recursive Bayesian estimate of a hidden true state given noisy observations, updated after every action and measurement, exactly the machinery a corridor robot needs to maintain a probability distribution over its position rather than pretending it knows where it is. His formalization of observability, whether a system's state can even in principle be recovered from available measurements, is the rigorous version of the robot's core dilemma, sonar stutters and walls look alike precisely because the corridor is poorly observable from certain positions. Where his classical theory falls short is the decision-making half: his filter estimates a belief but does not choose actions to gather information, that POMDP-planning layer was built by others on top of his estimation foundation.
Bertsekas's decades of work on dynamic programming and stochastic optimal control directly engage partially observable Markov decision processes, the formal object this problem needs, planning over a belief state rather than a known state, and his approximate-DP methods with Tsitsiklis address exactly why exact POMDP planning is intractable and what principled approximations look like. His neuro-dynamic programming explicitly discusses value-function approximation for large or continuous belief spaces, the actual deliverable this problem calls for once exact planning is ruled out. His research runs closer to the general control-theoretic formalism than to mobile-robotics engineering specifically, so building the sonar-based corridor robot itself sits slightly outside his own applied results, even though the underlying belief-state optimization theory is squarely his.
Fought here
In the mind map
The same ideas, as concepts rather than history — in John's ML knowledge map.
31 figures are scored on this problem. Draw it in a battle to see where you land.