Raquel Urtasun

Life and career

The autonomous vehicle industry has produced a great deal of demonstration video and comparatively little published science. Raquel Urtasun is the researcher who has most consistently insisted on doing both — running a production self-driving program while publishing the perception and forecasting methods behind it, and building the benchmark that let everyone else measure whether they were making progress.

She trained in Europe, taking her doctorate at EPFL in Lausanne in the mid-2000s on the problem of tracking articulated human motion in video. That early work is worth knowing about because it establishes her instincts. Recovering a person's 3D pose from monocular video is badly underdetermined, and her approach was to constrain it with learned priors over the low-dimensional manifold of plausible human motions — Gaussian-process latent variable models and their dynamical extensions. The lesson, which recurs throughout her later work, is that perception problems are solved by finding the right structural prior, not by throwing raw capacity at pixels.

Postdoctoral positions at MIT and Berkeley followed, then a faculty appointment at the Toyota Technological Institute at Chicago, and then the University of Toronto, where she became one of the anchors of that city's remarkable concentration of machine learning talent and a co-founder of the Vector Institute. During her Chicago years she co-created, with Andreas Geiger and Philip Lenz, the KITTI benchmark suite — and it is difficult to overstate what that did to the field. Before KITTI, autonomous driving perception research was evaluated on whatever data each group had. After it, there was a common set of calibrated, synchronized stereo, LiDAR, and GPS recordings from a real instrumented car with real annotations, and a public leaderboard for stereo, optical flow, odometry, tracking, and 3D detection. It became the standard against which a decade of work was measured.

In 2017 she took on an unusual dual role, becoming Chief Scientist and head of research for Uber's Advanced Technologies Group while remaining at Toronto, building a large industrial research operation in Canada. When Uber divested its self-driving effort, Urtasun founded Waabi in 2021, with a thesis distinct from most of the industry: rather than accumulating fleet miles, build an extremely high-fidelity closed-loop simulator and train and validate the driver primarily inside it, targeting long-haul trucking first. The argument is fundamentally statistical — the rare events that determine safety occur too infrequently in real-world driving to be validated by mileage alone, so the only tractable path is to generate them.

Key contributions

**KITTI and the discipline of benchmarking.** More than a dataset, KITTI was an argument about evaluation. Its accompanying paper asked bluntly whether the computer vision methods of the day were ready for autonomous driving, and the answer, demonstrated by a sharp drop in performance relative to existing academic benchmarks, was no. The suite's design — multiple sensor modalities, calibrated and time-synchronized, with tasks spanning low-level correspondence to high-level detection — established the template that later driving datasets followed.

**Deep structured models.** A recurring theme in Urtasun's Toronto-era work is the combination of deep networks with structured output prediction: rather than treating each pixel or each object independently, encode the dependencies between outputs in a graphical model whose potentials are produced by a network, and train the whole thing end to end. This appears in her efficient stereo matching work, where a network computes the matching cost and structure enforces smoothness and consistency, achieving accuracy at speeds compatible with a moving vehicle — the real-time constraint being non-negotiable in her setting.

**Joint detection, tracking, and forecasting.** Perhaps her most influential architectural argument is that the conventional autonomy pipeline — detect, then track, then forecast, then plan, each module trained separately — is wrong. Errors compound across module boundaries, latency accumulates, and each stage optimizes a proxy for what actually matters. Her group's work demonstrated single networks that consume a sequence of LiDAR sweeps and produce detections, tracks, and multi-second motion forecasts in one forward pass, sharing computation and trained against a unified objective. Later work extended this to explicitly model actor *intent* and to fold the motion planner into the same learned system, producing plans that are interpretable in the sense that intermediate representations remain inspectable.

**LiDAR representations for real-time 3D detection.** Detecting objects in a point cloud fast enough for a vehicle at highway speed is an architectural problem. Her group's work on bird's-eye-view representations — projecting the point cloud into a top-down grid where objects do not change scale with distance and convolution is naturally applicable — produced detectors running at real-time rates, and related work on sparse computation exploited the fact that most of a LiDAR scene is empty to skip computation where nothing exists.

**Simulation as the core method.** The Waabi thesis rests on generating sensor data faithful enough to train and validate on. Her group's work on LiDAR simulation reconstructs real scenes from collected data and re-renders them with modified geometry and new actors, so that a rare scenario — a pedestrian emerging from between parked cars, an unusual merge — can be instantiated in thousands of variations. The accompanying traffic simulation work learns realistic multi-agent behavior from data rather than scripting it, since a simulator populated with rule-following vehicles trains a driver for a world that does not exist.

In battle

Urtasun has one of the narrowest cards in the game: 100 problems, mean 22.4, median 14, and 66 problems at or below 20. But her peak is 96, and her top category, `perception`, spans nineteen problems at an average of 40.9. She is a scalpel for one class of problem.

**P074, "Drive through the intersection"** (96) is her exact professional problem, and the game says so. Building a perception and forecasting stack for a real fleet, confronting distribution shift, weather, night driving, and the long tail of pedestrian behavior as the central engineering challenge rather than an afterthought, is what she has done for a decade at industrial scale. The problem's crux — that the tail rather than the mean is where autonomous driving kills people — is the precise argument behind her simulation-first strategy. No other figure on the roster has personally shipped and iterated a production AV perception stack.

Her supporting wins are all vision. **P071, "Segment anything"** (84) rewards deep segmentation expertise, though the specific foundation-model result belongs to others. **P234, "Where did every pixel go?"** (80) is optical flow, a KITTI benchmark task she helped define. **P235, "Rebuild the city from vacation photos"** (80) is large-scale structure from motion, adjacent to the mapping and localization work any AV program requires. **P070, "Find every face"** (62) and **P025, "Sixty thousand digits"** (63) are generic vision competence rather than authorship. **P253, "The model is five percent of the system"** (68) is a genuinely earned score — few people on this roster have more direct experience of the fact that the trained model is a small component surrounded by data infrastructure, monitoring, and validation.

Her losses are among the most complete on the roster, and they cluster in classical statistics with total consistency. **P103, "Count the fish you cannot see"** (4) is capture-recapture estimation; **P133, "Why tall fathers have shorter sons"** (4) is regression to the mean; **P118, "The factor you can't keep changing"** (4) is split-plot experimental design; **P144, "The spam in the inbox"** (4) is naive Bayes; **P145, "Just look at the neighbors"** (4) is nearest neighbors. Her `regression` (11.3), `small-sample` (9.4), and `experimental-design` (11.0) averages are all in single or low double digits. This is the profile of a researcher whose entire career assumes abundant data and heavy compute, meeting problems designed for the opposite regime.

**P173, "Can these codeword lengths even exist?"** (4) — the Kraft inequality and the entropy bound on prefix-free codes — is the cleanest mismatch on her card. Her `information` and `computability` averages, both 6.3, mark territory she has simply never entered, and the same holds for `games` at 8.0.

The strategic read is unambiguous and should shape how a student plays her: Urtasun is close to unbeatable on driving, LiDAR, real-time perception, and the systems reality of deploying vision at scale, and she contributes nearly nothing anywhere else. She is a high-variance pick — save her for the problem she was built for.