
Lecture 13: Representation Learning Theory
Jeremy Bernstein lectures on the theory behind representation learning in MIT's 6.7960 Deep Learning course, Fall 2024. The lecture examines architectural inductive biases, the assumptions built into a network's structure that shape what it can learn efficiently, and works through the connection between wide neural networks and Gaussian processes. Bernstein develops the mathematical framework linking these two areas, showing how infinite-width limits of neural networks converge to Gaussian process behavior and what that reveals about generalization. The talk builds on earlier sessions in the course and assumes familiarity with basic deep learning concepts and probability. Runtime is 75 minutes, delivered as a standard chalkboard and slides lecture with MIT OpenCourseWare production values.