
Architectures: Transformers
Phillip Isola teaches lecture 8 of MIT's 6.7960 Deep Learning, laying out the transformer architecture through three components: tokens, attention, and positional codes. He works through how attention lets a model weigh relationships between tokens regardless of their distance in a sequence, and why positional encoding is needed to recover the order information that attention discards. The lecture situates transformers within the broader landscape of neural network architectures, comparing them to multilayer perceptrons, graph neural networks, and convolutional networks as different instances of shared underlying principles like weight sharing and locality versus global mixing. Isola uses the blackboard and slides to build the math step by step rather than treating the transformer as a black box. The lecture assumes familiarity with earlier course material on MLPs and CNNs and is aimed at students building toward implementing modern deep learning architectures.