AI History Battle

experimental-design

Where to place the measurements

It is around 1960, and the theory of experiments is turning on itself to ask: given a model you intend to fit and a fixed budget of observations, at which values of the inputs should you actually measure to learn the parameters most precisely? Left to intuition, an experimenter clusters points where it feels natural; the optimal-design theory says the right placement can shrink the variance of every estimate for free. Choose the design points that minimize the uncertainty of the fitted model, and justify the criterion you optimize. Get it wrong and you spend the same budget for a fuzzier answer — when observations are the currency, placing them by feel rather than by design silently wastes a fraction of every experiment.

optimal designminimize variancedesign-before-data

Who this problem belongs to

The two figures whose methods fit it best, out of 41 in contention.

1920–2023 · early-stat
95

Rao is the closest living match to this problem's exact question. His 1945 paper 'Information and the Accuracy Attainable in the Estimation of Statistical Parameters' gave the Cramer-Rao bound and, with it, the information-geometric view that turns 'where to place measurements' into a precise optimization: choose design points that maximize the Fisher information matrix in whatever sense the criterion demands. His Rao-Blackwell theorem (1945-47) sharpens exactly which statistics are worth collecting at all. Working at the Indian Statistical Institute under Mahalanobis he also handled combinatorial experimental layouts and orthogonal arrays for real agricultural and anthropometric surveys, so the 1960 framing sits squarely inside his active decades. The deduction is small: pure optimal-design theory (D-optimality, equivalence theorems) was formalized by Kiefer and Wolfowitz just as Rao worked adjacent territory, not the center of it.

1894–1981 · early-stat
90

Neyman's 1934 paper 'On the Two Different Aspects of the Representative Method' is the direct ancestor of this problem: given a fixed total sample size, how should it be allocated across strata to minimize the variance of the resulting estimate? His answer, Neyman allocation, is optimal design in miniature -- placing observational effort where the payoff in precision is greatest rather than where intuition points. He built this into a full theory of survey sampling and confidence intervals (1937), all hand-computable and criterion-driven exactly as the problem demands, and by 1960 he had spent three decades at Berkeley extending this program. He loses a little to Rao and Fisher because his allocation problem is about sample sizes across strata rather than continuous design points over an input space -- adjacent, not identical.

In the mind map

The same ideas, as concepts rather than history — in John's ML knowledge map.

A/B Testing

41 figures are scored on this problem. Draw it in a battle to see where you land.