Herbert Simon

Life and career

Herbert Alexander Simon holds a distinction no one else does: a Turing Award and a Nobel Prize in Economics, for work that he insisted was the same work. That insistence is the key to him. Simon spent sixty years arguing one thesis in a dozen different departments — that human beings are information-processing systems with severely limited computational resources, and that any theory of decision, in economics or psychology or organizational management or artificial intelligence, has to start there.

He was born in Milwaukee in 1916. His father was a German Jewish immigrant, an electrical engineer who later became a patent attorney; an uncle's books on economics reached him as a teenager and convinced him that the social sciences could be as rigorous as the physical ones, a conviction he never abandoned. He went to the University of Chicago in 1933, took his bachelor's degree in 1936, and did his doctoral work in political science, finishing in 1943.

The dissertation was about how administrators in municipal organizations actually make decisions — specifically, he had studied recreation-department budgeting in Milwaukee and been struck that nobody was doing anything resembling the marginal-utility calculation the textbooks described. It became *Administrative Behavior* (1947), and it contains the seed of everything: the argument that organizational decision-making cannot be understood as optimization because the decision-makers do not have, and could not process, the information optimization would require. The Nobel committee cited this book, thirty-one years later.

He taught at Illinois Institute of Technology through the 1940s, and in 1949 moved to Carnegie Institute of Technology in Pittsburgh — later Carnegie Mellon — to help build its new Graduate School of Industrial Administration. He stayed for the rest of his life, more than fifty years, and was central to establishing Carnegie Mellon as a computer science and cognitive science power.

The decisive encounter was with Allen Newell, whom he met around 1952 through RAND. Simon, Newell, and the programmer J. C. Shaw set out to build a machine that did not calculate but *reasoned*. In late 1955 and early 1956 they produced the Logic Theorist, which proved theorems in propositional logic by heuristic search through a space of possible derivations — and proved thirty-eight of the first fifty-two theorems in chapter two of *Principia Mathematica*, including one with a proof shorter and more elegant than Whitehead and Russell's. Simon submitted a paper on that proof to the *Journal of Symbolic Logic* with the machine listed as a co-author; it was rejected. He is reported to have told his class that winter that they had invented a thinking machine, and the claim, extravagant as it sounded, was closer to true than the reception suggested. The Logic Theorist was presented at the 1956 Dartmouth workshop, and was arguably the only running AI program there.

To build it they had to invent the infrastructure: IPL, the Information Processing Language, introduced list processing, dynamically allocated linked structures, recursion, and symbol manipulation as a programming paradigm — directly influencing McCarthy's LISP. The General Problem Solver followed in 1957, generalizing the approach with means–ends analysis.

Simon and Newell also made predictions in the late 1950s about how quickly machines would beat human chess champions and prove significant new mathematical theorems, which proved wildly optimistic and were held against them for decades. Simon's response was characteristically unrepentant about the direction if not the timing.

His later career was a sustained empirical psychology programme with Newell, Chase, and others: verbal protocol analysis (having subjects think aloud while solving problems, then modelling the transcript as a production system), the EPAM model of memory, and the famous chess expertise studies showing that masters reconstruct real board positions far better than novices but do no better on random arrangements — evidence that expertise is a large learned vocabulary of perceptual *chunks* rather than superior general processing. He estimated a chess master's vocabulary at roughly fifty thousand chunks, and the estimate has held up reasonably well.

He received the Turing Award with Newell in 1975, the Nobel Memorial Prize in Economic Sciences in 1978, the National Medal of Science in 1986, and the American Psychological Association's lifetime contribution award in 1993 — an unmatched spread. He taught undergraduates until near the end and died in 2001.

Key contributions

**Bounded rationality and satisficing.** The classical model of choice assumes an agent with a complete preference ordering, full knowledge of alternatives and consequences, and unlimited computational capacity. Simon's objection is not that people are irrational but that this model is computationally absurd: the alternatives are not given, the consequences are uncertain, and evaluating them all is intractable. Real agents therefore *satisfice* — they set an aspiration level, search until they find an alternative meeting it, take it, and adjust the aspiration based on how easily it was met. Simon's scissors metaphor is the compact statement: behavior is shaped jointly by the structure of the task environment and the computational limits of the actor, and you cannot explain the cut by examining one blade.

For a machine learning audience the relevance is direct. Bounded rationality is the philosophical charter for anytime algorithms, early stopping, metareasoning (deciding how much to compute before deciding), approximate inference, and the entire literature on computational rationality. Russell and Wefald's bounded-optimality framework and the modern "rational metareasoning" line are explicit formalizations of Simon's position.

**Heuristic search as the mechanism of intelligence.** Simon and Newell's technical contribution was to identify *search through a problem space* as the general form of problem solving, and *heuristics* as the means of making it tractable. The Logic Theorist and GPS operate on a state space with operators, a goal test, and evaluation heuristics that prune. GPS's means–ends analysis computes a difference between the current and goal states, selects an operator known to reduce that difference, and recurses on subgoals when the operator's preconditions are unmet — which is the ancestor of STRIPS planning and of goal-regression planners generally.

**The physical symbol system hypothesis.** In their 1976 Turing Award lecture, Newell and Simon stated the claim explicitly: a physical symbol system has the necessary and sufficient means for general intelligent action. It is empirical, falsifiable in principle, and it defined the symbolic AI research programme — and, by giving it a sharp statement, gave the connectionist counter-programme something specific to reject.

**The Sciences of the Artificial (1969).** Simon's argument that design — of organizations, of programs, of artifacts — is a science in its own right, with its own methods. The book contains the parable of the ant on the beach, whose complicated path is mostly a reflection of the terrain rather than of the ant, offered as a hypothesis about human cognition. It also develops *near-decomposability*: complex systems that survive tend to be hierarchies of nearly independent subsystems, because such systems can be assembled incrementally. The parable of the watchmakers Hora and Tempus makes the point about modularity that software engineering rediscovered decades later.

**Preferential attachment.** Simon's 1955 paper "On a Class of Skew Distribution Functions" derives power-law distributions from a rich-get-richer stochastic process — the Yule–Simon distribution. This is the same mechanism rediscovered as preferential attachment in network science in 1999, and Simon had it forty-four years earlier.

**Causal ordering.** In the early 1950s Simon worked on the problem of what causal claims a system of simultaneous equations licenses, developing an account of causal ordering from the structure of the equation system — early and genuine work on the identification problem that connects to modern structural causal models.

In battle

Simon carries 100 problems at a mean of 36.9 with seven dominant cells and twenty-nine at 20 or below. He is a broad thinker whose scoring is narrower than his reputation, and the reason is instructive: much of his influence is conceptual framing, and the matrix rewards specific machinery.

**Search** is his category at 67.1 across sixteen problems — his strongest showing and the largest coherent block. **Games** follows at 45.2 across sixteen, then **networks** (42.0), **fairness** (42.5), **optimization** (40.0), and **rl** (39.8).

His peak is **P196 — Best answer before the buzzer** at 96 — anytime computation, which is bounded rationality stated as an algorithmic requirement. The matrix is precise about the credit: he supplied the concept; the interruptible-search machinery came slightly after him, which is why it is 96 rather than 100. **P195 — Let the machine find the proof** (94) is the Logic Theorist, literally. **P190 — Sequence the robot's actions** (88) is planning by means–ends analysis, GPS's direct descendant. **P188 — Color the map, meet the constraints** (83) is heuristic constraint search. **P194 — Prune the adversary's replies** (78) is alpha-beta — his and Newell's chess work engaged the tree-size problem directly, though the algorithm is not his. **P259 — The imitation game, scored** (85) rewards his lifelong position on what machine intelligence would consist of. **P203 — The tournament of strategies** (84) is the Axelrod iterated-prisoner's-dilemma setting, where bounded-rationality agents with simple strategies is exactly the frame. **P049 — Schedule the moonshot** (85) draws on his organizational and hierarchical-decomposition work.

The losses are uniformly quantitative. **Regression** at 11.0, **experimental-design** at 16.0, **high-dim** at 18.0, **information** at 19.0, **small-sample** at 19.5. His floor cells are **P216 — Calculus for a jagged path** (5, Itô calculus), **P145 — Just look at the neighbors** (6, k-nearest neighbors), **P212 — Sample from the impossible posterior** (8, MCMC), **P169 — The scratch on the disc** (8), **P136 — Counting accidents** (8, Poisson regression), and **P217 — How high must the dike be?** (9, extreme value theory). Simon's method is symbolic and discrete throughout; he modelled cognition with production rules and problem spaces, not with distributions, and there is no continuous probability anywhere in his toolkit.

Two mid-range numbers deserve comment because they are lower than a reader of his biography would guess. **Classification** at 21.5 across seventeen problems is low for the co-founder of AI — but Simon's AI was symbolic reasoning, and statistical pattern recognition was, for most of his career, the rival programme he argued against. And **causality** at 28.8 across eighteen problems is modest despite his genuine 1950s work on causal ordering, because the game's causality problems are built on potential outcomes and graphical identification, machinery that arrived through Rubin and Pearl rather than through him.

Play Simon anywhere the crux is *searching a space under a resource constraint*: planning, theorem proving, game trees, constraint satisfaction, anytime decisions, and the design of agents that must act before they have finished thinking. Anywhere the crux is a number estimated from data, he is close to the floor.