Bin Yu

Life and career

Ask Bin Yu what makes a data-analytic conclusion trustworthy and you will not get an answer about p-values. You will get an answer about *perturbation*: change the data slightly, change a defensible modeling choice, change the random seed — and if the conclusion moves, it was never a conclusion. This idea, which she has pursued with unusual persistence for two decades, has quietly become one of the organizing principles of how careful people now do data science.

Yu grew up and did her undergraduate work in mathematics in China, at Peking University, before coming to the University of California, Berkeley for a doctorate in statistics. Berkeley in that era was the home of Le Cam's asymptotic decision theory and of a strong information-theoretic tradition, and both are visible in her early work: sharp minimax analysis, entropy and coding arguments, and a taste for the question of what is *achievable* rather than merely what is convenient.

After early faculty appointments elsewhere, she returned to Berkeley, where she has spent the bulk of her career and holds a Chancellor's Professorship spanning Statistics, Electrical Engineering and Computer Sciences, and the campus's computational biology program. She has chaired the statistics department and served as president of the Institute of Mathematical Statistics, and she has been elected to the National Academy of Sciences and the American Academy of Arts and Sciences. But the honors undersell the more distinctive thing about her position, which is that she is one of the very few senior statisticians who is simultaneously fully credible inside a machine learning department and inside a wet-lab collaboration.

That last part is not decorative. A large share of her research is done inside genuine scientific partnerships — with neuroscientists modeling how the visual cortex responds to natural movies, with geneticists hunting for interaction effects in developmental data, with groups working on remote sensing and climate. These collaborations are where her methodological convictions came from. If you have watched a genomics pipeline produce a different gene list because someone changed a normalization step, you stop treating stability as a nice-to-have.

Her book *Veridical Data Science*, written with Rebecca Barter, is the mature statement of the resulting position: an entire curriculum organized not around a catalogue of methods but around the question of when a data-driven claim deserves to be believed. The word *veridical* is deliberate — it means truthful, corresponding to reality — and choosing it over "reproducible" or "robust" was itself an argument.

Key contributions

**Model selection consistency of the lasso, and the irrepresentable condition.** With Peng Zhao, Yu answered a question that the lasso's popularity had outrun: under what conditions does ℓ₁-penalized regression recover the *correct set* of nonzero variables, not merely a good predictor? The answer is a condition on the design matrix — the **irrepresentable condition** — which, roughly, requires that the irrelevant predictors not be too well explained by the relevant ones. Formally, the regression of each noise variable on the signal variables must have coefficient vector with ℓ₁ norm bounded below one. The condition is almost necessary as well as sufficient, and it has a bracing consequence: with correlated designs, the lasso can be a fine predictor and a *systematically wrong* variable selector, no matter how much data you collect. Anyone who has been told that the lasso "does variable selection" should read this paper and adjust.

**Minimax lower bounds, made usable.** Her expository and technical work on the trio of Le Cam's method, Assouad's lemma, and Fano's inequality took three lower-bound techniques that were scattered across the literature and presented them as a coherent toolkit with a shared logic — reduce estimation to testing, then bound the testing problem information-theoretically. It is the standard reference for a generation of students who needed to prove that no estimator can do better than theirs.

**MDL and information-theoretic model selection.** With Andrew Barron and Jorma Rissanen, Yu developed the statistical theory of the minimum description length principle: model selection as the search for the shortest total encoding of model-plus-data, with precise connections between codelength, redundancy, and statistical risk. This is the cleanest bridge between coding theory and model choice, and it explains why complexity penalties of a particular form keep reappearing.

**Boosting with early stopping.** With Tong Zhang, she analyzed boosting as a regularization path rather than as an optimizer, showing that stopping early is not a practical hack but a principled form of regularization with provable consistency — the iteration count plays the role of an inverse penalty. This reframing is now the standard way to think about implicit regularization by optimization.

**Stability as a statistical principle.** Yu's stability program argues that a statistical procedure's output should be reproducible under appropriate perturbations of the data and of the analysis, and that quantifying this should be a routine part of reporting. Concretely, this produced tools — estimation stability with cross-validation (ESCV), for instance, which selects the lasso penalty by how stable the estimate is rather than by predictive error alone, and often lands on sparser, more reliable models than CV does.

**The PCS framework and veridical data science.** The mature form is **predictability, computability, stability**. *Predictability*: a model's claims must survive checking against held-out reality. *Computability*: the analysis must be actually executable and reproducible at the relevant scale, and computational choices are modeling choices. *Stability*: results must be robust to reasonable perturbations of the data, the model, and the whole analytic pipeline — including the human judgment calls that never make it into the methods section. PCS comes with a documentation discipline: enumerate the defensible alternatives at each step, run them, and report what survives. It is not a metaphor; her group has applied it to genomics and neuroscience pipelines and used it to kill findings that had looked solid.

**Interpretable machine learning and iterative random forests.** Related work laid out a taxonomy for what "interpretability" actually means (model-based versus post-hoc, and for whom), and developed iterative random forests, which use repeated reweighted forest fits to find stable high-order *interaction* signatures — motivated by developmental genetics, where the biology genuinely is combinatorial and a main-effects model has nothing to say.

In battle

Yu's profile is that of a strong, broad statistician with one towering peak: mean 41.0, median 40, five problems above 80, and a strength table topped by `high-dim` at 74.5.

**P097 — Does your pipeline reproduce?** at 99 is the highest score in her set and among the highest anywhere in the game, and the reason is simple: the problem asks for the formalization of stability as a criterion for trustworthy inference, stress-testing an analytic pipeline and reporting what survives. That is the PCS framework, written by the person who wrote it, in the decade she wrote it, at the university that was living the replication crisis at the time. There is no era gap and no interpretive stretch.

Her other dominant problems are the applied face of the same conviction. **P153 — Trained here, deployed there** (88) is distribution shift, which in her vocabulary is just stability across the largest perturbation of all. **P282 — The eigenvalues are lying** (82) and **P283 — Five hundred stocks, one year of days** (78) are high-dimensional covariance estimation, where sample eigenvalues are systematically biased and the honest answer requires knowing what the theory permits. **P022 — Predict, or explain?** (80) is a question she has argued in print for years. **P253 — The model is five percent of the system** (80) is the deployment-pipeline view that gives her the surprisingly high `systems` score of 49.0 — a rare thing for a statistician, and a genuine reflection of the computability leg of PCS. **P018 — p = 20,000, n = 200** (78) is the lasso-consistency work directly, and **P291 — The census under a privacy budget** (78) is a stability-and-tradeoff problem about a real institutional decision.

Her category profile rewards broad play within statistics: `small-sample` 45.1, `regression` 41.4, `causality` 37.2, `testing` 36.5, `experimental-design` 33.2 — nothing catastrophic across the statistical core. `fairness` at 58.2 is notably high, which follows from her interpretability work and her habit of asking who a model's claims are actually being made about.

The losses are outside statistics and are decisive. `computability` at 8.0 and `games` at 11.5 are her floor. She scores 5 on **P200 — Beat the world champion** — Deep Blue's search hardware and hand-tuned evaluation function have no contact with statistical methodology — 8 on **P163 — The fastest-growing function** (computability), 8 on **P157 — The equation with no algorithm** (Hilbert's tenth problem territory), 8 on **P190 — Sequence the robot's actions** (planning), 10 on **P127 — Weight of evidence at Bletchley** (a cryptanalysis problem where the statistical content is real but the setting and the sequential-decision machinery are Good and Turing's, not hers), and 15 on **P216 — Calculus for a jagged path** (stochastic calculus).

The tactical note: Yu is unusually safe as a broad pick within statistics and unbeatable on reproducibility and pipeline-integrity problems. Where she will disappoint you is anything algorithmic, symbolic, or game-theoretic — and, more subtly, anything whose difficulty is a *single* clever construction rather than the discipline of checking whether a result holds up.