Syllabus
Document Actions
Prerequisite
Preferably, ECE 6660, or instructors sufferance Co-requisite: ECE 5660.
Textbook
- Elements of Information Theory, by Thomas Cover and Joy Thomas, Wiley, 1994
- Information Theory, Inference, and Learning Algorithms, David J.C. Mackay.
Available at http://www.inference.phy.cam.ac.uk/mackay/itila/
Portions of the book Mathematical Methods and Algorithms for Signal Processing will also be used.
Also, some portions of Communications Systems Engineering by Proakis and Salehi will be helpful.
Supplementary reading
- Silicon Dreams, Robert W. Lucky, St. Martins Press, 1989.
Homework
- There will be assignments approximately weekly. It is your responsibility to make sure that you do it and understand the concepts.
Programming
A great deal of learning can be accomplished by programming algorithms and testing them for yourself. To reinforce this type of learning, two or three programming assignments will be assigned. There is also a term project which will also probably involve some programming. The programs can be written in any language on any machine you choose.
Course Summary
Information theory explores the fundamental limits of the representation and transmission of information. We will focus on the definition and implications of (information) entropy, the source coding theorem, and the channel coding theorem. These concepts provide a vital background for researchers in the areas of data compression, signal processing, controls, and pattern recognition.
This class is highly mathematical. The direct applications are, in a sense, only recently emerging, despite the nearly 50 year history. A firm determination and a fair degree of mathematical maturity will be required by students hoping to do well in the class. After all, how many engineering classes have you had which have the word "theory" in the title.
