LECTURES A GRATIS GLOBAL SERVICE
⌕ SEARCH GRATIS GLOBAL ↗
LECTURES
Architectures: Transformers
SOURCE: YOUTUBE · NO TRACKING UNTIL YOU PRESS PLAY · TROUBLE PLAYING? WATCH AT THE SOURCE ↗

Architectures: Transformers

75 MIN · EN · STATUS: [ STREAMING ]
RATE THIS
MIT · Deep Learning · LECTURE 8

Phillip Isola teaches lecture 8 of MIT's 6.7960 Deep Learning, laying out the transformer architecture through three components: tokens, attention, and positional codes. He works through how attention lets a model weigh relationships between tokens regardless of their distance in a sequence, and why positional encoding is needed to recover the order information that attention discards. The lecture situates transformers within the broader landscape of neural network architectures, comparing them to multilayer perceptrons, graph neural networks, and convolutional networks as different instances of shared underlying principles like weight sharing and locality versus global mixing. Isola uses the blackboard and slides to build the math step by step rather than treating the transformer as a black box. The lecture assumes familiarity with earlier course material on MLPs and CNNs and is aimed at students building toward implementing modern deep learning architectures.

More from this course

12 LECTURES
Introduction to Deep Learning

Introduction to Deep Learning

MIT · 61 MIN
How to Train a Neural Net

How to Train a Neural Net

MIT · 80 MIN
Approximation Theory

Approximation Theory

MIT · 83 MIN
Architectures: Grids

Architectures: Grids

MIT · 84 MIN
Architectures: Graphs

Architectures: Graphs

MIT · 81 MIN
Generalization Theory

Generalization Theory

MIT · 81 MIN
Scaling Rules for Optimization

Scaling Rules for Optimization

MIT · 81 MIN
Hacker's Guide to Deep Learning

Hacker's Guide to Deep Learning

MIT · 76 MIN
Architectures: Memory

Architectures: Memory

MIT · 73 MIN
Lec 11: Representation Learning: Reconstruction-Based

Lec 11: Representation Learning: Reconstruction-Based

MIT · 81 MIN
Representation Learning: Similarity-Based

Representation Learning: Similarity-Based

MIT · 76 MIN
Lecture 13: Representation Learning Theory

Lecture 13: Representation Learning Theory

MIT · 75 MIN