
Shaping the Future of AI from the History of Transformer
Hyung Won Chung, a research scientist at OpenAI who previously worked on Flan-T5 and Flan-PaLM at Google Brain, argues that keeping up with individual AI advances matters less than understanding the force driving them all: exponentially cheaper compute. Speaking in Stanford's CS25 Transformers United series, he walks through the early history of Transformer architectures, encoder-decoder, encoder-only, and decoder-only designs, explaining what motivated each one and why each became less necessary as compute scaled up. He spends real time on why decoder-only models dropped the encoder-decoder's extra structure, framing the choice as a consequence of scaling rather than architectural cleverness. The talk uses this historical thread to build a way of projecting where the field is heading next, rather than simply cataloging what has already happened. Slides accompany the recorded session.