perception
Where did every pixel go?
It is 1981 at Carnegie Mellon, and relating one video frame to the next is an underdetermined mess: each pixel's brightness gives one equation, but its motion has two unknowns — through a soda-straw view of a moving edge, the true direction is ambiguous. Estimate the motion field anyway: assume the flow is locally coherent, pool the constraints over a neighborhood, and solve the little least-squares system at every point — with an honest diagnostic, from the eigenvalues of that system, telling you where the image simply does not contain the answer (blank walls, straight edges) and the estimate must not be trusted. Get it wrong and everything downstream — tracking, stabilization, structure-from-motion, robot navigation — inherits confident motion vectors pointing where nothing moved.
Who this problem belongs to
The two figures whose methods fit it best, out of 57 in contention.
This problem is not an analogy for Kanade; it is his desk in 1981. With Bruce Lucas at Carnegie Mellon he formulated exactly this: treat optical flow as a local least-squares problem, assume constant flow in a small window, and solve the two-by-two system built from spatial and temporal image gradients. The Lucas-Kanade method became the standard solution to the aperture problem, and its natural companion diagnostic — checking the eigenvalues of that windowed gradient structure matrix to see whether a point has enough texture in two directions to be trusted — is the same mathematics later formalized as the Shi-Tomasi corner criterion, co-authored by Kanade's own lab. He would recognize every term in this problem statement because he wrote the paper it is describing, on the era's actual hardware, with the era's actual assumptions about local coherence.
Marr died in 1980, a year before this problem's date, but his posthumous 1982 book Vision had already circulated as drafts at MIT and shaped exactly this kind of question: motion as one of several early, local, bottom-up computations feeding the 2.5-D sketch, computed from raw image measurements before any object-level interpretation. His computational theory of vision insisted on separating the problem's mathematical formulation (what is being computed and why) from algorithm and implementation, which is precisely the discipline needed to state the aperture problem honestly — one brightness equation, two unknowns, only locally resolvable by pooling constraints. He did not derive the least-squares solution or the eigenvalue diagnostic himself, and the specific numerical machinery belongs to others, but the conceptual scaffolding for treating motion this way is substantially his.
In the mind map
The same ideas, as concepts rather than history — in John's ML knowledge map.
57 figures are scored on this problem. Draw it in a battle to see where you land.