nlp
Find the needle documents
It is 1972 at the Cambridge computer laboratory, and the documentation explosion is a policy crisis: scientific literature doubling every decade, and researchers repeating experiments because finding the prior paper costs more than redoing the work. Retrieve the twenty relevant documents from a million, ranked, when the query is three ambiguous words typed by someone who does not know the vocabulary of the answer. The crux is weighting: a term's usefulness is not its frequency but its rarity across the collection — a statistical insight about specificity that must be formalized, justified, and shown to beat exact matching on real test collections. The scheme you choose becomes the invisible arbiter of what knowledge is findable, and decades later its descendants will rank the entire web.
Who this problem belongs to
The two figures whose methods fit it best, out of 47 in contention.
This is Sparck Jones's own 1972 paper, 'A Statistical Interpretation of Term Specificity and Its Application in Retrieval,' written at the Cambridge Computer Laboratory and formalizing inverse document frequency: a term's retrieval usefulness scales inversely with the number of documents it appears in, giving a rigorous, testable answer to exactly the weighting problem this brief poses. She justified it against real test collections (Cranfield-style evaluations), meeting the problem's demand to 'beat exact matching on real test collections,' not just propose an idea. Her scheme, combined with term frequency, becomes TF-IDF, the ancestor of essentially all ranked web search. No score above hers is defensible; this problem is a description of her actual paper.
Manning's career is substantially the modern synthesis and extension of Sparck Jones's own project: his statistical NLP research and his co-authored Introduction to Information Retrieval textbook treat TF-IDF, vector-space ranking, and evaluation against test collections as core material, and his own research (probabilistic parsing, word embeddings) extends the statistical tradition she founded into representation learning. He can execute and teach the exact 1972 solution fluently and connect it to everything downstream. The deduction is originality and chronology: his career begins over a decade after this problem's 1972 setting, and he inherits rather than invents the term-weighting insight this brief specifically asks to be discovered.
In the mind map
The same ideas, as concepts rather than history — in John's ML knowledge map.
47 figures are scored on this problem. Draw it in a battle to see where you land.