More On Channel Capacity

Document Actions

The Converse To The Coding Theorem

We will begin with a special case: a zero-error channel, and show that this requires that the rate be less than the capacity. A zero error channel is one in which the sequence Yⁿ determines the input symbol W without any error, so H(W|Yⁿ)=0. We can obtain a bound by assuming that W is uniformly distributed over the 2^nR input symbols, so H(W) = nR. Now

$\begin{displaymath}\begin{aligned} nR &= H(W) = H(W\vert Y^n) + I(W;Y^n) \\ &= ... ...\text{definition of information channel capacity} \end{aligned}\end{displaymath}$

from which we conclude that for a zero-error channel, $R \leq C$ .

Recall that Fano's inequality related the probability of error to the entropy. Using the notation of the current context, it can be written as

$\begin{displaymath} H(X^n|Y^n) \leq 1 + P_e^{(n)} nR \end{displaymath}$

We also observe (and could prove) that

$\begin{displaymath} I(X^n;Y^n) \leq nC \end{displaymath}$

(Using the channel n times, the capacity per transmission is not increased.)

We now have the tools necessary to prove the converse to the coding theorem: any sequence of (2^nR,n) codes with $\lambda^{(n)} \rightarrow 0$ must have $R \leq C$ ..
$\begin{proof} Observe that $\lambda^{(n)} \rightarrow 0$\ implies $P_e^{(n)} \... ...en for $n$\ sufficiently large, $P_e^{(n)}$\ is bounded away from 0. \end{proof}$

Hamming codes?

Feedback channels: the same capacity!

Copyright 2008, Todd Moon. Cite/attribute Resource. admin. (2006, May 15). More On Channel Capacity. Retrieved November 23, 2009, from Free Online Course Materials — USU OpenCourseWare Web site: http://ocw.usu.edu/Electrical_and_Computer_Engineering/Information_Theory/lecture9_1.htm. This work is licensed under a Creative Commons License.

Course Contents Information Theory Home About the Professor Syllabus Schedule Homework Assignments Programming Assignments Download This Course	Personal tools Log in You are here: Home → Electrical and Computer Engineering → Information Theory → More On Channel Capacity
	Info More On Channel Capacity Document Actions Converse :: Source/Channel The Converse To The Coding Theorem We will begin with a special case: a zero-error channel, and show that this requires that the rate be less than the capacity. A zero error channel is one in which the sequence Yⁿ determines the input symbol W without any error, so H(W\|Yⁿ)=0. We can obtain a bound by assuming that W is uniformly distributed over the 2^nR input symbols, so H(W) = nR. Now $\begin{displaymath}\begin{aligned} nR &= H(W) = H(W\vert Y^n) + I(W;Y^n) \\ &= ... ...\text{definition of information channel capacity} \end{aligned}\end{displaymath}$ from which we conclude that for a zero-error channel, $R \leq C$ . Recall that Fano's inequality related the probability of error to the entropy. Using the notation of the current context, it can be written as $\begin{displaymath} H(X^n\|Y^n) \leq 1 + P_e^{(n)} nR \end{displaymath}$ We also observe (and could prove) that $\begin{displaymath} I(X^n;Y^n) \leq nC \end{displaymath}$ (Using the channel n times, the capacity per transmission is not increased.) We now have the tools necessary to prove the converse to the coding theorem: any sequence of (2^nR,n) codes with $\lambda^{(n)} \rightarrow 0$ must have $R \leq C$ .. $\begin{proof} Observe that $\lambda^{(n)} \rightarrow 0$\ implies $P_e^{(n)} \... ...en for $n$\ sufficiently large, $P_e^{(n)}$\ is bounded away from 0. \end{proof}$ Hamming codes? Feedback channels: the same capacity! Copyright 2008, Todd Moon. Cite/attribute Resource. admin. (2006, May 15). More On Channel Capacity. Retrieved November 23, 2009, from Free Online Course Materials — USU OpenCourseWare Web site: http://ocw.usu.edu/Electrical_and_Computer_Engineering/Information_Theory/lecture9_1.htm. This work is licensed under a Creative Commons License.	Reuse Course Download this course

Sections

Personal tools

More On Channel Capacity

Document Actions

The Converse To The Coding Theorem