
Aligning Open Language Models
Nathan Lambert, research scientist at the Allen Institute for AI, traces how open language models went from raw pretrained systems to chat-ready assistants after ChatGPT's release. Delivered as part of Stanford's CS25: Transformers United seminar series, the talk walks through the major techniques and datasets that defined this shift, including Alpaca's instruction tuning, QLoRA's efficient fine-tuning, and alignment methods DPO and PPO. Lambert, who previously built RLHF research efforts at HuggingFace, situates each method in the timeline of open-source releases and explains the tradeoffs between them. He closes with his own predictions for where alignment techniques for open models are heading next. The talk assumes familiarity with transformer architectures and is aimed at viewers already working with language models, offering a working researcher's map of a fast-moving subfield rather than an introductory overview.