
Retrieval Augmented Language Models
Douwe Kiela of Contextual AI gives this Stanford CS25 Transformers United guest lecture on retrieval augmented language models, the technique of pairing a language model with an external document retriever so outputs can be grounded in up-to-date or specialized text rather than relying solely on parameters learned during training. He opens by cataloging the shortcomings of standalone large language models, including hallucination, stale knowledge, and poor attribution, then surveys the recent research landscape of retrieval augmented approaches, covering how retrievers and generators are trained together or separately and how retrieved passages get incorporated into generation. The talk closes with open questions the field has not resolved, such as how to scale retrieval corpora, evaluate faithfulness, and handle multi-hop reasoning. Delivered as part of Stanford's CS25 seminar series on transformer architectures, the session runs just over an hour and assumes familiarity with standard transformer and language model concepts.