fairness
The model memorized the secrets
It is 2020, and a red team has coaxed a deployed language model into completing a prompt with a real person's name, address, and phone number — verbatim text from its training crawl, regurgitated on request. The model is not a database, its makers insist; the extraction says otherwise. Quantify memorization rigorously: which training sequences are extractable and with what probability, how memorization scales with model size, duplication, and sequence rarity, and how a membership-inference adversary can test whether a specific person's data was trained on. Then evaluate the defenses — deduplication, differentially private training — and their real costs. Get it wrong and every model trained on scraped data is a slow leak of the personal information of people who never consented, queryable by anyone with an API key.
Who this problem belongs to
The two figures whose methods fit it best, out of 40 in contention.
Song's research program on adversarial machine learning is the direct methodological source of this exact scenario: her group's work on membership inference and training-data extraction attacks against language models formalizes precisely how a red team coaxes a deployed model into regurgitating verbatim training text, and quantifies extraction probability as a function of model scale, duplication, and sequence rarity. She would treat 'the model is not a database' as a claim to be tested empirically rather than trusted, running systematic extraction and membership-inference sweeps before evaluating deduplication and differentially private training as defenses with honestly measured cost. No other carrier on this list built the specific adversarial-evaluation methodology this problem requires from the ground up.
Gebru co-authored 'On the Dangers of Stochastic Parrots' (2021, with Bender, McMillan-Major, and Mitchell), the paper that most directly forced the field to confront exactly this failure: large language models trained on scraped web text memorize and can regurgitate personal information nobody consented to include, with real people's data queryable by anyone holding an API key. Her broader push for documentation and accountability in how training corpora are assembled speaks directly to the extraction problem's root cause, an undocumented, unaudited crawl. She did not build the specific extraction or membership-inference attack methodology herself, so the purely technical measurement of memorization is not her direct contribution, keeping her just below the researchers who built that machinery.
In the mind map
The same ideas, as concepts rather than history — in John's ML knowledge map.
40 figures are scored on this problem. Draw it in a battle to see where you land.