perception
The subspaces of moving objects
It is 2004, and the factorization insight of the early nineties — that feature trajectories of a single rigid motion live in a low-dimensional subspace — collides with real video, where nothing is single: a camera pans while two cars pass and a pedestrian crosses, and the tracked points mix several rigid motions into one matrix of trajectories. Cluster the trajectories by the low-dimensional subspaces they live in, with geometry, not heuristics: the arrangement of subspaces has algebraic structure, and the segmentation should fall out of that structure with stated conditions for correctness, not a k-means initialization prayer. This is the difference between motion understanding as principled geometry and as tuned demo — and the machinery that scene understanding for robots and reconstruction pipelines will stand on.
Who this problem belongs to
The two figures whose methods fit it best, out of 54 in contention.
This problem is essentially Vidal's dissertation and early-career program described back to him. Generalized PCA (with Ma and Sastry, CVPR 2003 and after) is precisely the demanded solution: represent a union of subspaces as the zero set of homogeneous polynomials via the Veronese embedding, estimate the polynomials linearly from data, and obtain each point's subspace by differentiation — with explicit algebraic conditions on dimensions and sample positions under which the segmentation is provably correct, no initialization prayer anywhere. He carried the machinery to exactly this application: multibody motion segmentation from feature trajectories, including the multibody epipolar constraint, in the 2004-era literature this problem is set in. Every axis — multilinear structure, geometry, stated correctness — names his contribution. The only headroom withheld reflects the method's real-world noise sensitivity, which his later sparse and robust variants were needed to fix.
Kanade owns the ancestry this problem is built on. The Tomasi-Kanade factorization (1992) is 'the factorization insight of the early nineties' named in the problem statement: under affine projection, trajectories of a single rigid motion form a rank-constrained matrix whose SVD factors into shape and motion. His group then attacked the multibody extension itself — Costeira-Kanade factorization (1995-1998) introduced the shape interaction matrix to sort trajectories into independent motions from the data's algebraic structure, the first principled stab at this precise problem. Add Lucas-Kanade tracking (1981), which produces the very trajectories in question, and he supplies the pipeline end to end. What the 2004 framing demands beyond him is the full arrangement-theoretic treatment of dependent or degenerate motions with stated correctness conditions — the step from shape interaction to generalized PCA. The founder scores just below the finisher.
In the mind map
The same ideas, as concepts rather than history — in John's ML knowledge map.
54 figures are scored on this problem. Draw it in a battle to see where you land.