The Gaussian Channel
We now come to the first time in the book where the information is
actually carried by a
, instead of a random
variable. We will consider transmission over a band-limited channel
(such as a phone channel). A key result is the sampling theorem:
This is the classical Nyquist sampling theorem. However, Shannon's name is also attached to it, since he provided a proof and used it. A representation of the function f ( t ) is
From this theorem, we conclude (the dimensionality theorem) that a bandlimited function has only 2 W degrees of freedom per second.
For a signal which has "most'' of the energy in bandwidth W and "most'' of the energy in a time T , then there are about 2 WT degrees of freedom, and the time- and band-limited function can be represented using 2 WT orthogonal basis functions, known as the prolate spheroidal functions. We can view band- and time-limited functions as vectors in a 2 TW dimensional vector space.
Assume that the noise power-spectral density of the channel is
/2. Then the noise power is
. Over the
time interval of
seconds, the energy per sample (per channel use)
Use this information in the capacity:
There are 2 W samples each second (channel uses), so the capacity is
This is the famous and key result of information theory.
we have to do a little calculus to find that
This is interesting: even with infinite bandwidth, the capacity is not infinite, but grows linearly with the power.
We cannot do better than capacity!