Josh Tenenbaum

Life and career

A three-year-old shown a single picture of a giraffe will recognize giraffes for the rest of her life — from the side, from behind, cartoon giraffes, a giraffe made of Lego. No machine learning system of the 2000s could do this, and a great many of them still cannot. Josh Tenenbaum has built a career on treating that gap not as an engineering embarrassment but as the central scientific question about intelligence: how does the mind get so much from so little?

Tenenbaum studied physics as an undergraduate at Yale before moving to MIT's Department of Brain and Cognitive Sciences for his doctorate, completed at the end of the 1990s. The training combination matters. He came to cognitive science with a physicist's appetite for formal models and a willingness to write down equations for things psychologists had described qualitatively. His dissertation developed a Bayesian account of concept learning that remains the foundation of everything he has done since.

He spent a few years on the psychology faculty at Stanford before returning to MIT, where he has been ever since as Professor of Computational Cognitive Science. His group sits at a genuine crossroads — brain and cognitive sciences, the AI lab, and the interdisciplinary center for brains, minds, and machines — and its output has consistently been legible to three audiences that rarely read each other: cognitive psychologists, statisticians, and AI researchers. He was named a MacArthur Fellow in 2019.

What is striking about Tenenbaum's position in the field is that he spent the deep learning decade politely dissenting. As the community's attention consolidated around scaling gradient-trained networks on very large datasets, his lab kept insisting that human learning is characterized by structured priors, compositional representations, and causal models — and that a system which needs a million examples has not solved the problem the child solved with one. He made that case most directly in a widely discussed position paper with Brenden Lake, Tomer Ullman, and Sam Gershman on building machines that learn and think like people. His students and collaborators — including a long line who have gone on to lead their own labs — have carried probabilistic programming and structured cognitive modeling into the mainstream of AI, and the recent resurgence of interest in reasoning, world models, and sample efficiency has made his position look less like dissent and more like foresight.

Key contributions

**Isomap and the geometry of perceptual data.** Before he was known for Bayesian cognition, Tenenbaum co-authored one of the founding papers of nonlinear dimensionality reduction. Isomap, developed with Vin de Silva and John Langford, addresses a specific failure of classical PCA and multidimensional scaling: when data lies on a curved manifold, Euclidean distance in the ambient space is the wrong metric. Two points on opposite faces of a rolled-up sheet are close in $\mathbb{R}^3$ and far apart along the sheet. Isomap builds a neighborhood graph, computes shortest-path distances through that graph as an estimate of geodesic distance on the manifold, and then applies classical MDS to the geodesic distance matrix. Run on the canonical Swiss roll, it recovers the flat two-dimensional parameterization that PCA cannot see. Published alongside locally linear embedding in the same issue of *Science* in 2000, it launched the manifold learning field.

**The size principle and Bayesian generalization.** Tenenbaum's doctoral work asked why humans generalize confidently from a handful of positive examples and, crucially, how the *pattern* of examples controls the breadth of generalization. His answer, developed further with Tom Griffiths, is a Bayesian model in which hypotheses are candidate concepts and the likelihood of observing $n$ examples under a hypothesis covering a region of size $|h|$ scales as $1/|h|^n$. Tight clusters of examples therefore rapidly overwhelm broad hypotheses — three examples that happen to all be Dalmatians support "Dalmatian" far more than "dog," while three examples spanning different breeds support "dog." This gives a quantitative account of graded generalization gradients that matches human judgments, and it explains one-shot learning without any appeal to special mechanisms.

**Hierarchical Bayes and learning to learn.** With Charles Kemp and others, Tenenbaum showed how abstract knowledge can itself be learned from data via hierarchical priors — the "blessing of abstraction," where higher-level structure is often identifiable from *less* data than the lower-level details it constrains. Related work on the discovery of structural form asks the machine to infer not just parameters but the *kind* of structure organizing a domain — tree, ring, grid, hierarchy — recovering the taxonomic tree for animals and the periodic ring for colors from raw similarity data.

**Probabilistic program induction.** The mature version of the program treats concepts not as regions of feature space but as *programs* — generative procedures with parts, subparts, and compositional structure. The best-known result, with Brenden Lake and Ruslan Salakhutdinov, learns handwritten characters from single examples by inferring the stroke-level program that generated them, then demonstrates human-level performance on one-shot classification *and* on generating new examples that human judges cannot distinguish from human-drawn ones. This required probabilistic programming languages as infrastructure, and Tenenbaum's group was central to building them.

**Intuitive physics and theory of mind.** A parallel thread models human commonsense reasoning as approximate simulation. Given a tower of blocks, people predict whether it will fall by running a noisy mental physics engine — a model that predicts human judgments, including their systematic errors, better than heuristic accounts. The social analogue is inverse planning: inferring an agent's goals by running a planner backwards, asking what desires would make the observed trajectory rational.

In battle

Tenenbaum's card is bimodal in an unusually clean way, and reading the profile carefully will save a player from misplaying him. Across 100 problems he averages 28.5 with a median of 25 — modest — yet his `high-dim` category averages 91.0, the single highest category figure he posts.

That number comes from Isomap. **P284, "Unroll the swiss roll"** (97) is his paper, literally: geodesic distance estimated through a neighborhood graph, followed by classical scaling, applied to the exact synthetic manifold the problem names. **P278, "All your neighbors are equidistant"** (85) is the curse of dimensionality — the observation that in high dimensions the ratio of nearest to farthest neighbor distance approaches one — and it is the motivating pathology that manifold methods exist to escape. Together these two problems are his most reliable points, and they are worth remembering because they come from a part of his career that his public reputation has largely eclipsed.

His other summit is cognitive. **P062, "Concepts from three examples"** (97) is set in his laboratory. The size principle, the hierarchical priors that learn to learn, and the Bayesian program induction account of one-shot character learning are all his; the problem's demand for structured priors, a likelihood that makes three examples sufficient, and validation against human judgments is a description of his research program rather than an application of it. The only headroom above him is that the program remains unfinished — inference is expensive and its scope is genuinely contested.

Below the peaks, his strength shades into solid-but-not-decisive. **P056, "The doctor's update"** (70) and **P057, "The web of symptoms"** (68) reward general Bayesian fluency and graphical-model literacy without being his inventions. **P229, "The room with no reward"** (65) — exploration without external reward — connects to his work on curiosity and intrinsically motivated learning. **P259, "The imitation game, scored"** (62) plays to his long-standing interest in what would actually constitute human-like intelligence. **P025, "Sixty thousand digits"** (57) is MNIST, where he is credible but standing in a crowded field.

The losses are all of one type: engineering, discrete search, and classical statistics with no cognitive content. **P199, "Prune the game tree, provably"** (8) is alpha-beta and **P200, "Beat the world champion"** (8) is competitive game AI — his `games` average of 20.4 across sixteen problems is one of his weakest bands, which surprises players who expect a cognitive scientist to be strong on how people play games. He is not; that is a different literature. **P165, "The optimal codebook"** (8) is quantization and rate-distortion, and his `information` average of 10.5 confirms he carries nothing there. **P136, "Counting accidents"** (8) is Poisson regression, and his `regression` average of 9.0 is his floor. **P082, "Ship it to a hundred contributors"** (8) is software engineering practice, which is simply not what he does. **P289, "The variable you removed is still there"** (8) is algorithmic fairness auditing — his work concerns how minds build causal models, not how deployed scoring systems produce disparate impact, and the two only look related from a distance.

The rule of thumb: play Tenenbaum when the problem involves few examples, structured or compositional concepts, human-like generalization, or the geometry of high-dimensional perceptual data. Bench him for anything about scale, throughput, adversaries, or estimating a rate from counts.