Channel Capacity

Definitions :: Symmetric Channels :: Closer Look :: Typical Sequences :: Theorem

Channel Capacity Definition and Examples

We are now ready to talk about the fundamental concept of the capacity of a channel. This is a measure of how much information per channel usage we can get through a channel.

$\begin{definition} The {\bf information channel capacity} is defined as the max... ... maximum is taken over all possible input distributions $p(x)$. \end{definition}$

$\begin{example} Consider the noiseless channel, with error-free transmission. Fo... ...no equivocation: $C = 1$\ bit, which occurs when $p(x) = (1/2,1/2)$\end{example}$

$\begin{example} Noisy channel with non-overlapping outputs. $C=1$\ bit, when $p(x) = (1/2,1/2)$. \end{example}$

$\begin{example} Noisy typewriter, with crossover of 1/2 and 26 input symbols. O... ...vert X)] = \max H(Y) - 1 = \log 26 - 1 = \log 13. \end{displaymath}\end{example}$

$\begin{example} The BSC, with crossover probability $p$. \begin{displaymath} \b... ...l information left over for every bit of information received. \par\end{example}$

$\begin{example} Binary erasure channel: Probability $1-\alpha$\ of correct, $\a... ...\end{displaymath}This is maximized when $\pi=.5$, and $C=1-\alpha.$\end{example}$

The channel capacity has the following properties:

$C \geq 0$ . (why?)
$C \leq \log \vert\Xc\vert$ . (why?)
$C \leq \log\vert\Yc\vert$ .
I ( X ; Y ) is continuous function of p ( x ).
I ( X ; Y ) is a concave function of p ( x ). (It has a maximum)

Citation: admin. (2006, May 15). Channel Capacity. Retrieved January 07, 2011, from Free Online Course Materials — USU OpenCourseWare Web site: http://ocw.usu.edu/Electrical_and_Computer_Engineering/Information_Theory/lecture8_1.htm.