Information Theory
MIT's graduate course 6.441 introduces the quantitative theory of information and its role in reliable, efficient communication systems. Topics include the mathematical definition and properties of information, the source coding theorem, lossless data compression and optimal lossless coding, noisy communication channels, the channel coding theorem, and the source channel separation theorem. Later sessions cover multiple access channels, broadcast channels, Gaussian noise, and time-varying channels. Materials come from MIT OpenCourseWare and typically include lecture notes, problem sets, and readings drawn from Shannon's foundational work and standard information theory texts. The course is aimed at students with a solid background in probability and signals, and builds toward the mathematical tools engineers use to design compression schemes and communication systems that approach theoretical performance limits.