
Transmitting Information Reliably over a Noisy Channel & Shannon's Noisy Coding Theorem
Ankur Moitra, teaching MIT's 18.200 Principles of Discrete Applied Mathematics, works through how information can be sent reliably even when a channel introduces errors. He starts with repetition codes, the simplest scheme imaginable, and shows concretely why they waste bandwidth without buying much reliability. The lecture then turns to Shannon's noisy coding theorem, which guarantees that far more efficient codes exist, and Moitra proves the theorem's simplest case for the binary symmetric channel, where each transmitted bit flips independently with some fixed probability. The proof leans on probabilistic arguments about random codes and typical error patterns rather than explicit constructions. This is lecture eighteen in the course sequence, assuming familiarity with earlier probability and combinatorics material, and it gives a rigorous entry point into information theory for students who want the mathematics behind data transmission rather than just the engineering intuition.