David Blei

Life and career

David Blei's career is a case study in what happens when one graduate-school paper turns out to name a whole research area. He studied computer science as an undergraduate at Brown and went to Berkeley for his doctorate, where he worked with Michael I. Jordan during the period when Jordan's group was systematically rebuilding machine learning on graphical-model foundations. Berkeley in the late 1990s and early 2000s was the place where variational methods — imported from statistical physics, adapted to inference in Bayesian networks — were being turned from a curiosity into a general-purpose tool, and Blei arrived exactly in time to apply them to something people cared about.

The something was text. In 2003, with Andrew Ng and Jordan, he published latent Dirichlet allocation. The paper did not invent the idea that documents mix themes; it gave that idea a generative probabilistic model with a proper prior, a tractable approximate-inference algorithm, and a form clean enough to extend. Within a few years topic modeling was a standard method not only in machine learning but in digital humanities, political science, and biology, and LDA became one of the most cited machine-learning papers of its decade.

After a postdoc at Carnegie Mellon with John Lafferty, Blei joined the faculty at Princeton, where he spent roughly eight years and trained a remarkable group of students, and then moved to Columbia in 2014 with a joint appointment in statistics and computer science. He was named a MacArthur Fellow in 2010. At Columbia he sits inside the university's data-science institute and works alongside Andrew Gelman's group — a productive adjacency, since Blei's approximate-inference machinery is exactly what applied Bayesian modeling at scale had been missing, and the automatic differentiation variational inference his group developed with Alp Kucukelbir was folded into Stan.

The through-line of the work is a particular research posture, which he has articulated repeatedly: probabilistic modeling as a loop. You encode your assumptions about hidden structure in a generative model, you compute the posterior over that hidden structure, you check the fit against the data, and you revise. Blei's contribution has been mostly to the second step — making posterior computation cheap enough that the loop can actually turn on real corpora — but the framing matters, because it explains why his group's papers so often come in pairs: a new model, and the inference algorithm that makes it usable.

Key contributions

**Latent Dirichlet allocation.** The generative story: for each document, draw a topic proportion vector from a Dirichlet prior over the simplex; for each word in the document, draw a topic from those proportions and then a word from that topic's distribution over the vocabulary. Topics are shared across the corpus; proportions are per-document. The model is a hierarchical mixed-membership model, and that is its whole advantage over the clustering methods it displaced — a document is not assigned to *one* cluster, it is a mixture, which is both more realistic and what makes the fitted topics interpretable to a domain expert. Compared to its predecessor, probabilistic latent semantic indexing, LDA's Dirichlet prior gives a proper generative model of unseen documents and a principled way to smooth, rather than a per-document parameter that grows with the corpus.

**Making approximate Bayesian inference practical.** The posterior in LDA is intractable, and most of Blei's subsequent career is about that intractability in general. Mean-field variational inference posits a factorized family over the latent variables and maximizes the evidence lower bound (ELBO), turning integration into optimization. Three successive generalizations matter. *Stochastic variational inference*, with Matt Hoffman, Chong Wang and John Paisley, applied stochastic optimization with natural gradients to the ELBO so that fitting requires only subsampled minibatches — the step that took topic models from thousands of documents to millions. *Black-box variational inference*, with Rajesh Ranganath and Sean Gerrish, used Monte Carlo estimates of the ELBO gradient with variance reduction so that the practitioner no longer had to derive model-specific update equations — you write down the model, the algorithm handles it. *Automatic differentiation variational inference* completed the automation inside a probabilistic programming system. His review article with Kucukelbir and Jon McAuliffe, written explicitly for statisticians, is the standard entry point and is worth reading for its honest treatment of what variational methods get wrong: the mean-field posterior systematically understates variance and misses dependence.

**The model zoo.** Correlated topic models replace the Dirichlet with a logistic-normal so that topics can co-occur in correlated ways. Dynamic topic models let topic-word distributions drift over time, which is how you track the vocabulary of a scientific field across a century of journals. Supervised LDA attaches a response variable so topics are learned to be predictive. Relational topic models handle documents linked in a network. The nested Chinese restaurant process, drawing on Bayesian nonparametrics, learns a *hierarchy* of topics with the depth and branching determined by the data rather than fixed in advance.

**Causal inference with latent confounders.** More recently, with Yixin Wang, Blei proposed the deconfounder: in settings with many simultaneous causes, fit a latent-variable model of the assignment mechanism and use the inferred latent variable as a substitute for unobserved confounding. The proposal has been genuinely contested in the causal-inference literature — the identification conditions are strong and were debated vigorously in print — but it is a serious attempt to bring latent-variable modeling to bear on confounding, and it is the reason causality shows up as a strength on his sheet.

In battle

Blei is the sharpest specialist among the modern statisticians on this roster, and his numbers say so bluntly: mean 26.6, median 20, fifty-one problems at or below 20, and only three above 80. Almost his entire value is concentrated in a handful of matchups where he is not applying a method but *is* the method.

"The topics in the archive" (98) is LDA restated — a historian with a corpus, wanting interpretable latent themes with honest uncertainty rather than hard cluster assignments. There is no one on the roster who can contest it. "The posterior at web scale" (97) is stochastic variational inference; again, his own paper. "The web of symptoms" (84) rewards the graphical-model and structured-latent-variable side of his training, and with the deconfounder work behind it his causality average of 79.0 is the highest category score on his sheet — an unusual profile for someone the field files under "topic models." "Fill in the hidden variables" (70) is the general latent-variable/EM problem, where he is strong but not the owner, since that lineage belongs to Dempster–Laird–Rubin. Below that the numbers fall off fast: "Who will know whom next year?" (58) is link prediction, where relational topic models are relevant but not decisive; "Sample from the impossible posterior" (56) is telling, because it is *MCMC's* problem, and Blei's whole methodological bet was on the variational alternative — he loses the sampling problems to the samplers. "p = 20,000, n = 200" (52) is high-dimensional inference, adjacent to his work but owned by the sparsity and multiple-testing people.

The floor is a clean lesson in what a probabilistic-modeling toolkit cannot do. Everything discrete and combinatorial bottoms out: "A good-enough tour of a thousand cities" (5), "The variables that must be whole" (5), "Prune the game tree, provably" (8). Variational inference is continuous optimization of an evidence bound; nothing in it touches approximation algorithms, branch-and-bound, or adversarial search. Logic and computability are equally foreign — "The truth it cannot prove" (8), the Gödel problem, is as far from his work as anything on the board. "Just look at the neighbors" (8) is instructive in a different way: nearest-neighbor methods are simple, nonparametric, and model-free, and Blei's entire contribution is the opposite bet — write down the generative structure first. And "The coefficient that flips sign" (7) is classical regression diagnostics, a tradition he never worked in.

Even his NLP average is modest at 26.5 across thirteen problems, which surprises people. The reason is era: the NLP problems in this game are mostly parsing, translation, embeddings, and attention, and topic models are none of those. Contextual representations ate the ground that bag-of-words topic models used to hold. Play Blei when the problem says *find the interpretable hidden structure in a large collection of counts, and tell me how sure you are*. Play anyone else otherwise.