More On Channel Capacity

Document Actions

The Converse To The Coding Theorem

We will begin with a special case: a zero-error channel, and show that this requires that the rate be less than the capacity. A zero error channel is one in which the sequence Y ⁿ determines the input symbol W without any error, so H ( W |Y ⁿ )=0. We can obtain a bound by assuming that W is uniformly distributed over the 2 ^nR input symbols, so H ( W ) = nR . Now

$\begin{displaymath}\begin{aligned} nR &= H(W) = H(W\vert Y^n) + I(W;Y^n) \\ &= ... ...\text{definition of information channel capacity} \end{aligned}\end{displaymath}$

from which we conclude that for a zero-error channel, $R \leq C$ .

Recall that Fano's inequality related the probability of error to the entropy. Using the notation of the current context, it can be written as

$\begin{displaymath} H(X^n|Y^n) \leq 1 + P_e^{(n)} nR \end{displaymath}$

We also observe (and could prove) that

$\begin{displaymath} I(X^n;Y^n) \leq nC \end{displaymath}$

(Using the channel n times, the capacity per transmission is not increased.)

We now have the tools necessary to prove the converse to the coding theorem: any sequence of (2 ^nR , n ) codes with $\lambda^{(n)} \rightarrow 0$ must have $R \leq C$ ..
$\begin{proof} Observe that $\lambda^{(n)} \rightarrow 0$\ implies $P_e^{(n)} \... ...en for $n$\ sufficiently large, $P_e^{(n)}$\ is bounded away from 0. \end{proof}$

Hamming codes?

Feedback channels: the same capacity!

Copyright 2008, Todd Moon. Cite/attribute Resource . admin. (2006, May 15). More On Channel Capacity. Retrieved January 07, 2011, from Free Online Course Materials — USU OpenCourseWare Web site: http://ocw.usu.edu/Electrical_and_Computer_Engineering/Information_Theory/lecture9_1.htm. This work is licensed under a Creative Commons License

Sections

Personal tools

More On Channel Capacity

Document Actions

The Converse To The Coding Theorem

Course Information Theory Home About the Professor Syllabus Schedule Homework Assignments Programming Assignments	Personal tools You are here: Home → Electrical and Computer Engineering → Information Theory → More On Channel Capacity
	Info More On Channel Capacity Document Actions Converse :: Source/Channel The Converse To The Coding Theorem We will begin with a special case: a zero-error channel, and show that this requires that the rate be less than the capacity. A zero error channel is one in which the sequence Y ⁿ determines the input symbol W without any error, so H ( W \|Y ⁿ )=0. We can obtain a bound by assuming that W is uniformly distributed over the 2 ^nR input symbols, so H ( W ) = nR . Now $\begin{displaymath}\begin{aligned} nR &= H(W) = H(W\vert Y^n) + I(W;Y^n) \\ &= ... ...\text{definition of information channel capacity} \end{aligned}\end{displaymath}$ from which we conclude that for a zero-error channel, $R \leq C$ . Recall that Fano's inequality related the probability of error to the entropy. Using the notation of the current context, it can be written as $\begin{displaymath} H(X^n\|Y^n) \leq 1 + P_e^{(n)} nR \end{displaymath}$ We also observe (and could prove) that $\begin{displaymath} I(X^n;Y^n) \leq nC \end{displaymath}$ (Using the channel n times, the capacity per transmission is not increased.) We now have the tools necessary to prove the converse to the coding theorem: any sequence of (2 ^nR , n ) codes with $\lambda^{(n)} \rightarrow 0$ must have $R \leq C$ .. $\begin{proof} Observe that $\lambda^{(n)} \rightarrow 0$\ implies $P_e^{(n)} \... ...en for $n$\ sufficiently large, $P_e^{(n)}$\ is bounded away from 0. \end{proof}$ Hamming codes? Feedback channels: the same capacity! Copyright 2008, Todd Moon. Cite/attribute Resource . admin. (2006, May 15). More On Channel Capacity. Retrieved January 07, 2011, from Free Online Course Materials — USU OpenCourseWare Web site: http://ocw.usu.edu/Electrical_and_Computer_Engineering/Information_Theory/lecture9_1.htm. This work is licensed under a Creative Commons License