rl
Ride the winds, stay on station
It is 2020, and stratospheric balloons are delivering internet to remote regions — if they can hold position over their service area with the only control anyone has at sixty thousand feet: rising or sinking into wind layers blowing different directions, on a power budget metered by sunlight. The winds are known only through forecasts that go stale, so the controller flies through uncertainty: choose altitude to stay on station now without spending the battery or drifting somewhere the wind field offers no way home. Train the controller in simulation on historical wind data, then certify the transfer to real aircraft in real jet streams. Get it wrong and the fleet scatters and the connectivity blinks out; this became RL's first controller flying autonomously, for months, in the real sky.
Who this problem belongs to
The two figures whose methods fit it best, out of 31 in contention.
Abbeel's entire research program is built around exactly the pipeline this problem describes: train a controller in simulation, using historical or modeled dynamics, then certify the transfer to real, physically consequential hardware where failure has real cost, sim-to-real transfer for robotic manipulation and locomotion is his signature contribution to deep reinforcement learning. His Berkeley lab's work on robustifying policies against the gap between simulated and real dynamics, domain randomization and careful validation before deployment, is the direct methodological ancestor of certifying a balloon controller trained on historical wind data before it flies in real jet streams. Where the fit is not perfect is the specific domain: his best-known results are terrestrial robots rather than stratospheric aircraft, but the sim-to-real discipline itself transfers almost completely.
Silver's deep reinforcement learning career, pairing learned value functions with neural-network function approximation and validating extensively offline before real deployment, matches this problem's structure closely: a controller trained in simulation on historical wind data, then trusted to fly autonomously for months once certified. His instinct for combining planning and learning under uncertainty, visible across his work on games and control, is philosophically the same move needed to choose altitude under stale forecasts without a live model of the true wind field. Where the connection weakens is domain specificity: his signature results are in board games and simulators with cleaner, fully specified dynamics than a real stratospheric jet stream, so the actual balloon-control engineering runs through a different, less famous corner of the RL community.
In the mind map
The same ideas, as concepts rather than history — in John's ML knowledge map.
31 figures are scored on this problem. Draw it in a battle to see where you land.