EE 376A Handout #2Information Theory Tuesday, March 30, 2011T. Cover

Information TheoryCourse Information

Web Page:

http://www.stanford.edu/class/ee376b/

Instructor:

Prof. Tom CoverPackard 254723-4505cover@stanford.eduOffice hours: Wednesday 2-3 pm. Generally available mid-afternoons.

Administrative Associate:

Denise MurphyPackard 267723-4731denise@ee.stanford.edu

TA:

Gowtham Kumar, Packard 251, gowthamr@stanford.edu

Textbook:

Elements of Information Theory, Second Edition, T. Cover and J. Thomas, Wiley, 2006.Several texts can serve as auxiliary or reference texts:

• “Information Theory and Reliable Communications,” by Gallager, New York, Wiley1968.

• “Key Papers in Development of Information Theory,” edited by Slepian, New York,IEEE Press, 1974.

• “Information Theory,” by Csiszar and Korner, New York, Academic Press, 1981.

• “Claude Shannon, Collected Papers,” edited by N.J.A. Sloane and Aaron D. Wyner,New York: IEEE Press, 1993.

PrerequisitesEE376a

Course Description

Maximum entropy and Burg’s theorem. Gaussian Channel Capacity. Rate Distortion theoryand Kolmogorov Complexity. Information theory and statistics. Stein’s Lemma. AEP.Network Information theory. Slepian-Wolf Theorem. Optimal Investment and informationtheory. Universal Portfolios and data compression.