LECTURES A GRATIS GLOBAL SERVICE
⌕ SEARCH GRATIS GLOBAL ↗
LECTURES
Intuitions on Language Models / Shaping the Future of AI from the History of Transformers
SOURCE: YOUTUBE · NO TRACKING UNTIL YOU PRESS PLAY · TROUBLE PLAYING? WATCH AT THE SOURCE ↗

Intuitions on Language Models / Shaping the Future of AI from the History of Transformers

77 MIN · EN · STATUS: [ STREAMING ]
RATE THIS
STANFORD

Two guest talks from Stanford's CS25 seminar on transformers and large language models, delivered by OpenAI researchers Jason Wei and Hyung Won Chung. Wei argues that next word prediction can be understood as massive multi task learning, and uses that framing to reconcile scaling laws with the sudden appearance of individual emergent capabilities, drawing on close manual inspection of training data. Chung takes a historical approach, tracing early Transformer architecture choices back to the driving force behind them all, the exponentially falling cost of compute, and showing how design decisions that once mattered became unnecessary as scaling continued. He uses this history to sketch where the field is likely heading next. Both talks include slides and run as part of the same 77 minute session, aimed at an audience already familiar with transformer basics but looking for a deeper conceptual frame.

At a glance

Lecture facts

Runtime compared with the other 734 Mathematics lectures
Runtime1 h 17 m
Compared with MathematicsLonger than 69%
Source channelStanford Online (YouTube)