
Metrized Deep Learning
Jeremy Bernstein presents his research on metrized deep learning as part of MIT's 6.7960 Deep Learning course, Fall 2024. The lecture works through modular neural network design, treating a network as a composition of modules that each carry their own notion of size, and shows how this metric structure informs optimization. Bernstein connects the framework to scaling behavior, explaining why certain architectural choices allow training to remain stable as models grow larger. He develops duality principles that link the geometry of weight space to the geometry of gradients, and uses these ideas to motivate optimizer design choices that depart from standard gradient descent. Running 68 minutes, the talk assumes familiarity with neural network basics and moves quickly into original research territory, aimed at students and researchers who want the mathematical reasoning behind modern large scale training methods rather than an introductory survey of deep learning.