
Architectures: Memory
MIT's 6.7960 Deep Learning course continues with a lecture on architectures built to handle memory and sequence modeling. The instructor covers recurrent neural networks and their limitations, then moves to Long Short-Term Memory networks and related gated architectures, explaining how these models retain information across time steps and process sequential data. The lecture works through the mechanics of how these networks pass hidden states forward, why plain RNNs struggle with long sequences, and how gating mechanisms address that problem. Part of the MIT OpenCourseWare Fall 2024 offering, this is lecture ten in the series, aimed at students who already have grounding in neural network basics and are moving into sequence-specific architectures ahead of attention and transformer models.