rl
How much stock to hold
It is 1951 at RAND, and the quartermasters' question turns out to be mathematics: each period a demand arrives at random, holding stock costs money, running out costs more, and every reorder pays a fixed charge regardless of size. Optimize not one order but the standing rule — and discover the structure that makes the answer humane: under broad conditions the optimal policy is two numbers, reorder when stock falls below one threshold, fill up to the other. Prove it from the dynamic-programming recursion, characterize when it fails, and hand the clerk a policy card instead of a supercomputer. Get it wrong and every warehouse either drowns in capital tied up as inventory or bleeds customers through stockouts — multiplied across an economy.
Who this problem belongs to
The two figures whose methods fit it best, out of 38 in contention.
Bellman's dynamic programming, formalized at RAND in the early 1950s, is precisely the machinery this problem requires: express the inventory problem as a recursion over periods, cost-to-go functions, and demand uncertainty, and solve it backward from the recursion itself. The celebrated (s,S) threshold-policy result, reorder when stock falls below one number, fill up to another, is a direct and famous product of exactly this kind of dynamic-programming analysis of periodic-review inventory with fixed ordering costs, developed by Bellman-era operations researchers working in his framework at RAND in the 1950s. His broader insistence that hard sequential problems decompose into simple recursive subproblems is the conceptual key that turns an intractable-looking inventory policy into two numbers on a card. This is close to the exact problem his own methods were built to solve.
Bertsekas's dynamic programming and optimal control texts contain the canonical rigorous treatment of exactly this problem: periodic-review inventory control with fixed ordering costs, stochastic demand, and holding and shortage penalties, proving under precisely stated convexity conditions on the cost structure that the optimal policy reduces to a simple threshold, an (s,S) policy characterized by K-convexity of the cost-to-go function. His textbooks are where generations of operations researchers and control theorists actually learned to derive and prove this exact structural result from the dynamic-programming recursion. His broader career synthesizing Bellman's foundational ideas into rigorous, general convergence and structural theorems makes him arguably the sharpest possible modern authority on proving this problem's threshold-policy structure from first principles.
In the mind map
The same ideas, as concepts rather than history — in John's ML knowledge map.
38 figures are scored on this problem. Draw it in a battle to see where you land.