LECTURES A GRATIS GLOBAL SERVICE
⌕ SEARCH GRATIS GLOBAL ↗
LECTURES
Why Language Models Hallucinate
SOURCE: YOUTUBE · NO TRACKING UNTIL YOU PRESS PLAY · TROUBLE PLAYING? WATCH AT THE SOURCE ↗

Why Language Models Hallucinate

70 MIN · EN · STATUS: [ STREAMING ]
RATE THIS
IAS

Adam Kalai, a researcher at OpenAI, argues that language model hallucinations are not mysterious architectural failures but predictable results of how models are trained and evaluated. He presents hallucination as a classification problem: when a pretrained model cannot reliably tell a false statement from a true one, it tends to produce the false answer rather than admit uncertainty. He then traces the problem to benchmark design, showing that most evaluation metrics reward confident guessing over calibrated abstention, which pushes models away from saying I don't know. The talk closes with a proposed fix, revising benchmarks so they score abstention favorably rather than penalizing it. The work is joint with Santosh Vempala of Georgia Tech and Ofir Nachum and Edwin Zhang of OpenAI. Delivered as part of the Institute for Advanced Study's Computer Science and Discrete Mathematics seminar series, the lecture runs seventy minutes and includes technical detail suited to a research audience.

At a glance

Lecture facts

Runtime compared with the other 734 Mathematics lectures
Runtime1 h 10 m
Compared with MathematicsLonger than 51%
Source channelInstitute for Advanced Study (YouTube)