
Generalization Theory
MIT's 6.7960 Deep Learning, taught by Phillip Isola, devotes this lecture to why overparameterized neural networks generalize at all despite classical statistical learning theory predicting they should overfit. Isola walks through VC dimension and explains why it fails to account for the behavior of modern networks with far more parameters than training examples. He covers the double descent phenomenon, where test error falls, rises, and then falls again as model capacity grows past the interpolation threshold, and connects this to empirical observations across model sizes. The lecture closes on inductive bias, the built-in assumptions that steer a network toward some solutions over others even when many fit the training data equally well. Delivered as a chalkboard and slides lecture at 81 minutes, it assumes familiarity with basic statistical learning concepts and is aimed at students already working through the course's earlier material on optimization and architectures.