
Approximation Theory
Jeremy Bernstein teaches the third lecture of MIT's 6.7960 Deep Learning course, asking how well a deep neural network can approximate a given function. He works through universal approximation theorems, which establish that neural networks can in principle represent a wide class of functions, then moves to Barron's theorem, which quantifies how approximation error scales with network width for certain function classes. The lecture also takes up the question of depth, examining whether making a network deeper provably improves its expressive power rather than just its width. The style is chalkboard-and-slides mathematics, building definitions and theorem statements step by step rather than running code, aimed at students who already have the calculus and linear algebra background the course assumes. Eighty three minutes, part of a full OCW playlist covering the semester.