
Transfer Learning: Models
MIT's 6.7960 Deep Learning course, taught by Sara Beery, devotes this lecture to transfer learning, the practice of adapting a model trained on one task or dataset to work on another. Beery walks through fine-tuning, where a pretrained network's weights are further adjusted on new data, and linear probing, which freezes the backbone and trains only a lightweight output layer on top of its learned representations. She covers knowledge distillation, the technique of training a smaller student model to reproduce a larger teacher model's behavior, and closes with foundation models, large networks pretrained on broad data that serve as a starting point for many downstream applications. The lecture runs 86 minutes and builds on the course's earlier material on representation learning, treating transfer learning as the practical link between research-scale pretraining and deployable models with limited data or compute.