experimental-design
Just-in-time on the phone
It is the smartphone decade, and a health app can nudge a user to walk, to breathe, to take a pill — but a nudge at the wrong moment is noise the user learns to ignore. You must design micro-randomized interventions: thousands of tiny experiments per person, each testing whether a prompt delivered in this context, right now, actually changes behavior. The effect of any single nudge is minuscule and drowned in the noise of a life; only the aggregated design recovers it. Learn per-context, per-individual effects at mobile scale. Get it wrong and the app either spams people into deleting it or nudges so timidly it changes nothing — and the chance to move real health outcomes at population scale slips away.
Who this problem belongs to
The two figures whose methods fit it best, out of 37 in contention.
This problem is essentially a description of Murphy's own research program. In the 2010s she formalized the micro-randomized trial (MRT): each user is randomized to prompt-or-no-prompt at hundreds or thousands of decision points, and weighted estimators recover time-varying, context-specific proximal effects from exactly the noise this problem describes. Her HeartSteps activity-coaching studies deployed this design on real phones, and her earlier work on dynamic treatment regimes supplies the sequential-decision scaffolding for adapting per individual. She also confronted the practical failure modes named here — habituation, burden, users disabling notifications — building availability and burden constraints directly into the design. No one else in this batch has both the causal estimand and the deployed mobile infrastructure. The only gap is engineering scale, which she solved by collaboration rather than alone.
Sutton's toolkit — temporal-difference learning from the 1980s, and the contextual-bandit and online-RL machinery his field matured through the 2000s — is the natural engine for choosing when to nudge: each decision point is a state, each prompt an action, each behavior change a delayed, noisy reward. Learning per-context policies incrementally from streams of weak signal is precisely what TD methods were built for, and modern mobile-health systems (including Murphy's later work) literally embed bandit algorithms of this lineage. His bitter-lesson stance also fits mobile scale: lean on data and computation, not hand-crafted nudge rules. What he lacks is the statistician's side of the problem — randomization-based causal inference, inference after adaptive sampling, and trial ethics — so his agent might optimize engagement while leaving the scientific question of whether nudges work unanswered.
In the mind map
The same ideas, as concepts rather than history — in John's ML knowledge map.
37 figures are scored on this problem. Draw it in a battle to see where you land.