AI History Battle

experimental-design

Roll it out in waves

It is the present, and a health system wants to introduce a new safety checklist across forty hospitals — but it cannot switch them all at once, and it will not deny the checklist to anyone permanently. Design a stepped-wedge trial: every hospital starts in the control condition and crosses over to the intervention at a randomly assigned time, so the rollout itself becomes the experiment. The design must separate the checklist's effect from secular trends — outcomes drifting over the study period for reasons of their own — which are entangled with time exactly as the treatment is. Get it wrong and a general improvement in care over the year gets miscredited to the checklist, or a real gain is masked by the drift.

stepped-wedgetime-confoundingcausal-infer

Who this problem belongs to

The two figures whose methods fit it best, out of 34 in contention.

b. 1943 · stat-learning
92

Rubin's potential-outcomes framework (1974, building on Neyman) is built precisely for the question this problem poses: what would each hospital's outcome have been under control versus intervention at each point in time, and how do you separate that causal quantity from a secular trend that would have happened anyway. His and his students' extensions into time-varying treatments and staggered adoption designs directly anticipate the stepped-wedge structure, where every unit eventually receives treatment but at a randomly assigned time. His insistence that the assignment mechanism -- the random crossover order -- licenses the causal comparisons a stepped-wedge analysis can validly make is exactly the discipline the problem demands. The score falls short of perfect only because the stepped-wedge design's terminology and dedicated mixed-model analysis were formalized by later health-services statisticians extending his framework.

b. 1965 · stat-learning
88

Stepped-wedge trials are, in modern practice, analyzed almost universally with the hierarchical mixed-effects models Gelman championed and popularized through Stan and Bayesian Data Analysis: hospitals as random-effect clusters, a fixed time effect capturing the secular trend, and a treatment indicator that switches on when each hospital crosses over, letting the model cleanly separate 'improvement over time' from 'improvement because of the checklist.' His multilevel-modeling philosophy is the field's standard toolkit for exactly this structure, and his applied Bayesian workflow is built for the kind of messy, real-world health-system data this problem describes. He sits just below Rubin because the specific stepped-wedge terminology and its earliest formal treatments emerged from biostatisticians (Hussey and Hughes, 2007) working somewhat independently of, though compatible with, Gelman's broader hierarchical-modeling program.

In the mind map

The same ideas, as concepts rather than history — in John's ML knowledge map.

A/B Testing

34 figures are scored on this problem. Draw it in a battle to see where you land.