LECTURES A GRATIS GLOBAL SERVICE
⌕ SEARCH GRATIS GLOBAL ↗
LECTURES
Lecture 16: Data Compression and Shannon's Noiseless Coding Theorem
SOURCE: YOUTUBE · NO TRACKING UNTIL YOU PRESS PLAY · TROUBLE PLAYING? WATCH AT THE SOURCE ↗

Lecture 16: Data Compression and Shannon's Noiseless Coding Theorem

69 MIN · EN · STATUS: [ STREAMING ]
RATE THIS
MIT · Principles of Discrete Applied Mathematics · LECTURE 16

MIT professor Ankur Moitra teaches this lecture from 18.200, Principles of Discrete Applied Mathematics, covering the mathematics of data compression. He opens with the history of the problem, then defines a first-order information source and what it means to compress one. From there he builds up the definition of entropy as a measure of information content, and uses it to state and prove Shannon's noiseless coding theorem, which establishes the optimal compression ratio achievable for a first-order source. The sixty-nine minute session is blackboard-style, with Moitra working through definitions and the proof step by step rather than relying on slides. It assumes prior exposure to probability and discrete math from earlier in the course, and serves as the rigorous foundation for why compression algorithms cannot beat the entropy bound.

At a glance

Lecture facts

Runtime compared with the other 148 Computer Science lectures
Runtime1 h 9 m
Compared with Computer ScienceShorter than 58%
This series

Principles of Discrete Applied Mathematics

Every lecture in order, sized by its length.

  • Earlier lectures
  • This lecture
  • Still to come
Lecture 15 of 1917 h 19 m before this · 23 h 23 m in total

More from this course

12 LECTURES
Pigeonhole Principle

Pigeonhole Principle

MIT · 74 MIN
Lecture 2: Independence and Conditioning

Lecture 2: Independence and Conditioning

MIT · 71 MIN
Lecture 3: Inclusion-Exclusion

Lecture 3: Inclusion-Exclusion

MIT · 79 MIN
Lecture 4: Counting

Lecture 4: Counting

MIT · 78 MIN
More Counting and Generating Functions

More Counting and Generating Functions

MIT · 72 MIN
More on Generating Functions

More on Generating Functions

MIT · 80 MIN
Generating Functions for Catalan Numbers

Generating Functions for Catalan Numbers

MIT · 72 MIN
Tail Bounds

Tail Bounds

MIT · 81 MIN
Lecture 9: Chernoff Bounds

Lecture 9: Chernoff Bounds

MIT · 55 MIN
Basic Group Theory

Basic Group Theory

MIT · 75 MIN
Introduction to Linear Programming

Introduction to Linear Programming

MIT · 74 MIN
Duality in Linear Programming

Duality in Linear Programming

MIT · 76 MIN