classification
Trained here, deployed there
It is the era when models ship from the lab to places the lab never saw, and a diagnostic classifier trained on one hospital's scanners quietly loses accuracy at the next hospital down the highway — different machines, different patient mix, the same disease. The training and deployment distributions do not match, and the model has no idea. Detect that the input distribution has shifted, quantify how much the labels can still be trusted, and correct for it where the shift is in the features and not the rule. State which shifts are recoverable and which are not. Get it wrong and a system validated to clinical standards degrades silently, its published accuracy a number only ever true in one building.
Who this problem belongs to
The two figures whose methods fit it best, out of 71 in contention.
This problem is Pearl's own research agenda by name: his theory of transportability, developed through the 2000s and 2010s with collaborators like Elias Bareinboim, formally addresses exactly this question, when can results learned in one environment, one hospital's scanners and patient mix, be transferred validly to another, and what causal structure must be shared for the transfer to be licensed. His do-calculus provides the formal machinery to state precisely which shifts are recoverable through reweighting or adjustment and which are not, this problem's exact final demand. His decades-long critique of purely associational machine learning as blind to exactly this kind of silent distributional failure anticipated this problem's stakes years in advance. Almost no one in this roster has a more direct, named claim on this exact problem.
Scholkopf's causal machine learning research program, developed substantially through the 2010s, is centrally organized around exactly this problem: distinguishing causal mechanisms that remain invariant across environments from spurious correlations that break under distribution shift, with explicit application to the kind of covariate shift between hospitals or scanners this problem describes. His work on causal transfer learning and invariant prediction directly addresses detecting and correcting for shifted input distributions while preserving valid inference. He did not develop Pearl's original do-calculus formalism, keeping him just below Pearl, but his applied, machine-learning-facing research program is arguably the most direct technical toolkit for actually building the kind of shift-robust system this problem demands. Few in this roster have a more direct, applied claim on this exact concern.
In the mind map
The same ideas, as concepts rather than history — in John's ML knowledge map.
71 figures are scored on this problem. Draw it in a battle to see where you land.