Thomas Bayes

Life and career

Almost everything we know about Thomas Bayes comes from the margins of other people's records. He was born around 1701 into a prosperous English Nonconformist family — his father Joshua was one of the first Presbyterian ministers ordained in England after the Act of Toleration — and because Dissenters were barred from Oxford and Cambridge, Bayes went north to the University of Edinburgh, where he studied logic and theology. He was ordained, served as Presbyterian minister at Tunbridge Wells in Kent, and lived the quiet, comfortable life of a well-off provincial clergyman with a mathematical hobby. He never held an academic post. He published, under his own name, only two works in his lifetime, and neither was the one he is remembered for: a theological tract on divine benevolence, and an anonymous 1736 defense of Newton's calculus against Bishop Berkeley's famous attack on the "ghosts of departed quantities." That second pamphlet was good enough that it is generally credited with his election to the Royal Society in 1742 — a striking honor for a man with no publication record to speak of.

The essay that made him immortal he did not publish at all. Bayes died in 1761, and his mathematical papers passed to his friend Richard Price, a Welsh minister and moral philosopher who was himself a serious figure in probability and actuarial science. Price found among the papers an unpublished manuscript, recognized its importance, added an introduction and worked examples of his own, and read it to the Royal Society in 1763. It appeared in the *Philosophical Transactions* as "An Essay towards solving a Problem in the Doctrine of Chances." How much of the final published argument is Bayes and how much is Price remains a live scholarly question; Price's contribution was substantial enough that some historians argue for shared credit.

The essay is short, difficult, and framed around a physical thought experiment rather than an abstract axiom. Bayes asks you to imagine a ball rolled onto a level square table, coming to rest at some unknown position; then further balls are rolled, and you are told only whether each lands to the left or right of the first. What can you say about where the first ball is? This is a strange way to introduce the most consequential formula in modern statistics, but it is doing something precise: it constructs a physical situation in which the unknown quantity genuinely has a prior distribution, so that reasoning backward from data to cause is unambiguously legitimate. Bayes was worried about exactly the objection that would dog his followers for two centuries — where does the prior come from? — and he chose a setup where the answer was not in doubt.

He was buried in Bunhill Fields, the Nonconformist cemetery in London. There is no verified portrait of him; the image reproduced everywhere is of uncertain provenance, which is a fitting piece of irony for a man whose name is attached to reasoning under uncertainty.

Key contributions

The theorem itself is a one-line consequence of the definition of conditional probability: P(H | D) ∝ P(D | H) P(H). Stated that way it is trivial, and it was not what Bayes was after. His actual contribution was the *inversion* — the recognition that the machinery of forward probability, which computes the chance of data given a known mechanism, can be run backward to compute the credibility of a mechanism given observed data. Before Bayes, probability was overwhelmingly a forward discipline: given a fair die, what is the chance of three sixes? Bayes posed what was then called the problem of inverse probability, and what a modern reader recognizes as the whole of statistical inference.

For a graduate audience the substantive content is best located in three places. First, the essay effectively works with what we would now call a beta-binomial model: a uniform prior over an unknown success probability, binomial data, and a posterior that Bayes computes as a ratio of areas under a curve — an integration problem he had to attack by hand, and which occupies most of the paper's technical difficulty. That the hard part of the 1763 essay is *computing the normalizing constant* is a joke that has aged extremely well; it is still the hard part.

Second, Bayes made the prior an explicit object of the calculation rather than an embarrassment to be hidden. His billiard-table construction is a defense of the uniform prior via a physical randomization mechanism, and the scholium in which he argues for extending that assumption to cases of pure ignorance is the origin of two centuries of argument about objective priors, invariance, and what "knowing nothing" could possibly mean formally.

Third, and most important for anyone doing applied work, Bayes gave the correct rule for accumulating evidence. Each new observation multiplies the current belief by a likelihood ratio. Weak evidence compounds; independent observations multiply; the posterior from today's data is the prior for tomorrow's. This sequential, multiplicative structure is what makes Bayesian inference an *engine* rather than a formula. Every naive Bayes classifier, every Kalman filter update, every particle filter resampling step, every posterior-sampling scheme in a modern probabilistic programming language is running Bayes's combination rule at scale on hardware he could not have imagined.

What he did not have is worth stating plainly. He had no decision theory, no notion of a likelihood function as an object in its own right (that is Fisher), no asymptotics, no hierarchical models, and no computational method beyond hand integration. Laplace, working independently a decade later, is the one who turned inverse probability into a working research program and applied it to real astronomical and demographic data. Bayes supplied the seed.

In battle

Bayes is a specialist of unusual sharpness. Across a hundred problems he averages 32.6 with a median of 24 — a low baseline — but his distribution is violently bimodal: eight problems where he scores 80 or above, and thirty-eight where he scores 20 or below. He is not a generalist who is decent everywhere. He is a scalpel.

Where he wins, he wins on home turf: problems whose actual historical solution *is* his theorem. His single best showing is "Who wrote the disputed papers?" at 99 — the Federalist Papers authorship study, which combined per-word likelihood ratios under competing authorship hypotheses with a prior and updated across many weak signals into a single odds statement. That is not an analogy to Bayes's rule; it is his rule, executed literally. The same structure explains "The spam in the inbox" (96) and "The inbox under siege" (98), both of which are naive Bayes classification with a different noun attached, and it explains his top category score of 98.5 in NLP. "The doctor's update" (96) is the base-rate/diagnostic-test problem that is the canonical classroom illustration of the theorem. "The one-in-a-million event" (92) and "Are boys more likely than girls?" (92) reach further — those are really Laplace's rule of succession and Laplace's sex-ratio analysis — but the inferential skeleton is Bayes's, and he scores just below Laplace on both. "Concepts from three examples" (81) is Bayesian concept learning: extreme small-sample generalization where a prior is the only thing standing between you and nonsense. His 55.1 average in small-sample problems, across fifteen of them, is his most reliable broad strength, and it is exactly the regime where a prior earns its keep.

Where he loses, he loses completely, and the pattern is instructive. Anything requiring a machine, an algorithm, or an engineered artifact bottoms out: "The language for the job" (3) — programming-language design in 1959 — is his floor case, and the game's own explanation says so explicitly, calling it an honest scoring of total mismatch. "The robot in the warehouse" (8), "Learning on the molecule graph" (8), "Cut the image, weight the graph" (8), and "Cut through the interior" (6) are graph learning, spectral methods, and interior-point optimization — continuous, high-dimensional, algorithmic machinery invented one to two centuries after his death. "The dice make it learnable" (8) is randomized learning theory: the failure here is not era alone but framework, since PAC-style worst-case guarantees are deliberately prior-free and answer the exact question Bayes's approach declines to ask.

The broader shape: his category profile collapses to single digits in optimization (7.0), systems (9.0), search (10.0), and games (11.0), and stays weak in networks (16.0) and perception (16.0). Play Bayes when the problem is *belief under uncertainty with structure you can encode in advance* — diagnosis, authorship, filtering, rare events, tiny samples. Do not play him when the problem is combinatorial, geometric, adversarial, or a matter of engineering something that has to run. He gives you a principled way to weigh evidence and no way at all to compute at scale, which is a fair summary of both his strengths and the two hundred years of work that followed him.