LECTURES A GRATIS GLOBAL SERVICE
⌕ SEARCH GRATIS GLOBAL ↗
LECTURES
Transformers that Transform Well Enough to Support Near-Shallow Architectures
SOURCE: YOUTUBE · NO TRACKING UNTIL YOU PRESS PLAY · TROUBLE PLAYING? WATCH AT THE SOURCE ↗

Transformers that Transform Well Enough to Support Near-Shallow Architectures

80 MIN · EN · STATUS: [ STREAMING ]
RATE THIS
STANFORD

Jake Ryland Williams, Associate Professor of Information Science at Drexel University, presents this Stanford CS25 guest lecture on efficient methods for training language models. He introduces what he calls precision LMs, or PLMs, built on non-random parameter initializations for self-attention architectures that cut the cost of training both large and small models. Williams explains how untrained PLMs can be localized directly onto microprocessors to act as hardware controllers for small electronics, including training in air-gapped environments and scaling progressively bigger models on ordinary CPUs. He walks through a fully built control system and its user interface, including experiments on the Le Potato single-board computer, where the system learned to interpret spoken commands for a microphone and light switch after just 20 minutes of untrained interaction with a lay user. The talk is aimed at listeners with some background in transformer architectures and machine learning training methods.

At a glance

Lecture facts

Runtime compared with the other 387 Computer Science lectures
Runtime1 h 20 m
Compared with Computer ScienceLonger than 75%
Source channelStanford Online (YouTube)