Channel Capacity

Document Actions

Definitions :: Symmetric Channels :: Closer Look :: Typical Sequences :: Theorem

The channel coding theorem and its proof

We are now ready for the big theorem of the semester.

$\begin{theorem} (The channel coding theorem) All rates below capacity $C$\ are ... ...\ codes with $\lambda^{(n)} \rightarrow 0$\ must have $R \leq C$. \end{theorem}$

Note that by the statement of the theorem, we only state that "there exists'' a sequence of codes. The proof of the theorem is not constructive, and hence does not tell us how to find the codes.

Also note that the theorem is asymptotic. To obtain an arbitrarily small probability of error, the block length may have to be infinitely long.

Being the big proof of the semester, it will take some time. As we grind through, don't forget to admire the genius of the guy who was able to conceive of such an intellectual tour de force. Aren't you glad this wasn't your homework assignment to prove this!

Whew!

The proof used a random-coding argument to make the math tractable. In practice, random codes are never used. Good, useful, codes are a matter of considerable interest and practicality.

Copyright 2008, by the Contributing Authors. Cite/attribute Resource. admin. (2006, May 17). Channel Capacity. Retrieved March 20, 2010, from Free Online Course Materials — USU OpenCourseWare Web site: http://ocw.usu.edu/Electrical_and_Computer_Engineering/Information_Theory/lecture8_5.htm. This work is licensed under a Creative Commons License.

Course Contents Information Theory Home About the Professor Syllabus Schedule Homework Assignments Programming Assignments Download This Course	Personal tools Log in You are here: Home → Electrical and Computer Engineering → Information Theory → Channel Capacity
	Info Channel Capacity Document Actions Definitions :: Symmetric Channels :: Closer Look :: Typical Sequences :: Theorem The channel coding theorem and its proof We are now ready for the big theorem of the semester. $\begin{theorem} (The channel coding theorem) All rates below capacity $C$\ are ... ...\ codes with $\lambda^{(n)} \rightarrow 0$\ must have $R \leq C$. \end{theorem}$ Note that by the statement of the theorem, we only state that "there exists'' a sequence of codes. The proof of the theorem is not constructive, and hence does not tell us how to find the codes. Also note that the theorem is asymptotic. To obtain an arbitrarily small probability of error, the block length may have to be infinitely long. Being the big proof of the semester, it will take some time. As we grind through, don't forget to admire the genius of the guy who was able to conceive of such an intellectual tour de force. Aren't you glad this wasn't your homework assignment to prove this! Whew! The proof used a random-coding argument to make the math tractable. In practice, random codes are never used. Good, useful, codes are a matter of considerable interest and practicality. Copyright 2008, by the Contributing Authors. Cite/attribute Resource. admin. (2006, May 17). Channel Capacity. Retrieved March 20, 2010, from Free Online Course Materials — USU OpenCourseWare Web site: http://ocw.usu.edu/Electrical_and_Computer_Engineering/Information_Theory/lecture8_5.htm. This work is licensed under a Creative Commons License.	Reuse Course Download this course

Sections

Personal tools

Channel Capacity

Document Actions

The channel coding theorem and its proof