perception
Drive through the intersection
It is 2017, and autonomous vehicles are on public roads in Pittsburgh and Phoenix with safety drivers whose attention is the only backstop. Perceive and predict everything around the car — vehicles, cyclists, the pedestrian stepping off the curb mid-block — in rain, at night, against low sun, with a perception stack whose benchmark numbers were earned in daylight. The technical crux is the tail of the distribution: the jaywalker with a bicycle at 10 p.m. that appears in no training set, the distribution shift between the mapped test city and everywhere else. Ninety-nine percent accuracy is a fatality every few weeks at fleet scale. Within a year, a pedestrian in Tempe, Arizona will be killed by exactly this failure. Build the stack as if you know that.
Who this problem belongs to
The two figures whose methods fit it best, out of 54 in contention.
This is Urtasun's exact professional problem. Through the 2010s at Uber ATG and then Waabi she built perception and motion-forecasting stacks for real self-driving fleets in exactly this window (Pittsburgh, Phoenix), confronting distribution shift, rain, night driving, and long-tail pedestrian scenarios as the core engineering challenge rather than an afterthought. Her research on joint perception-and-prediction and on benchmarking failure in rare scenarios addresses the problem's central crux directly: the tail, not the mean, is where autonomous driving kills people. No other carrier on this list has personally shipped and iterated a production AV perception stack against these exact stakes. Even granting the era gap, Urtasun's standing on perception for self-driving; waabi is close enough to this problem's actual demands that a graduate student would expect a real, defensible showing rather than a token one.
Malik's Berkeley program on segmentation, perceptual organization, and object recognition supplied much of the vision-science foundation that AV perception stacks of the mid-2010s were built on, and his students and lab produced core detection and segmentation techniques used across the industry in this era. He did not personally build a driving stack, so credit is for foundational method rather than fleet deployment, but the technical distance from his actual published research to this problem's requirements is small. A strong, historically grounded second to Urtasun on pure perception grounds. Even granting the era gap, Malik's standing on computer vision: segmentation, perceptual organization is close enough to this problem's actual demands that a graduate student would expect a real, defensible showing rather than a token one.
Fought here
In the mind map
The same ideas, as concepts rather than history — in John's ML knowledge map.
54 figures are scored on this problem. Draw it in a battle to see where you land.