Personal tools
  •  
You are here: Home Electrical and Computer Engineering Information Theory The Gaussian Channel

The Gaussian Channel

Document Actions
  • Content View
  • Bookmarks
  • CourseFeed

Definitions   ::   Band-limited   ::   Kuhn-Tucker   ::   Parallel   ::   Colored Noise

Channels with colored Gaussian noise

We will extend the results of the previous section now to channels with non-white Gaussian noise. Let K z be the covariance of the noise K x the covariance of the input, with the input constrained by

 

 \begin{displaymath}
\frac{1}{n} \sum_i EX_i^2 \leq P
\end{displaymath}

 

which is the same as

 

 \begin{displaymath}
\frac{1}{n} \trace(K_X) \leq P.
\end{displaymath}

 

We can write

 

 \begin{displaymath}
I(X_1,\ldots,X_n; Y_1, \ldots ,Y_n) = h(Y_1,\ldots, Y_n) -
h(Z_1,\ldots,Z_n)
\end{displaymath}

 

where

 

 \begin{displaymath}
h(Y_1,\ldots, Y_n)  \leq \frac{1}{2} \log((2\pi e)^n|K_x + K_z|)
\end{displaymath}

 

Now how do we choose K x to maximize K x + K z , subject to the power constraint? Let

 

 \begin{displaymath}
K_z = Q\Lambda Q^T
\end{displaymath}

 

then

 

 \begin{displaymath}
\begin{aligned}
|K_x + K_z| &= |K_x + Q \Lambda Q^T| = |Q| |Q^T K_x Q + \Lambda|
|Q^T| \\
&= |Q^T K_x Q + \Lambda| = |A+\lambda|
\end{aligned}
\end{displaymath}

 

where A = Q T K x Q . Observe that

 

 \begin{displaymath}
\trace(A) = \trace(Q^T K_x Q) = \trace(Q^T Q K_x) = \trace(K_x)
\end{displaymath}

 

So we want to maximize $\vert A+\Lambda\vert$ subject to $\trace(A) \leq nP$ . The key is to use an inequality, in this case Hadamard's inequality. Hadamard's inequality follows directly from the "conditioning reduces entropy'' theorem:

 

 \begin{displaymath}
h(X_1,\ldots,X_n) \leq \sum h(X_i).
\end{displaymath}

 

Let $\Xbf \sim \Nc(0,K)$ . Then

 

 \begin{displaymath}
h(\Xbf) = \frac{1}{2}\log (2\pi e)^n |K|
\end{displaymath}

 

and

 

 \begin{displaymath}
h(X_i) = \frac{1}{2} \log (2\pi e)K_{ii}
\end{displaymath}

 

Substituting in and simplifying gives

 

 \begin{displaymath}
|K| \leq \prod_i K_{ii}
\end{displaymath}

 

with equality iff K is diagonal.

Getting back to our problem,

 

 \begin{displaymath}
|A+\Lambda| \leq \prod_i(A_{ii} + \Lambda_{ii})
\end{displaymath}

 

with equality iff A is diagonal. We have

 

 \begin{displaymath}
\frac{1}{n} \sum_i A_{ii} \leq P
\end{displaymath}

 

(the power constraint), and $A_{ii} \geq 0$ . As before, we take

 

 \begin{displaymath}
A_{ii} = (\nu - \lambda_i)^+
\end{displaymath}

 

where $\nu$ is chosen so that

 

 \begin{displaymath}
\sum A_{ii} = nP.
\end{displaymath}

 

Now we want to generalize to a continuous time system. For a channel with AWGN and covariance matrix K Z ( n ) , the covariance is Toeplitz. If the channel noise process is stationary, then the covariance matrix is Toeplitz, and the eigenvalues of the covariance matrix tend to a limit as $n \rightarrow \infty$ . The density of the eigenvalues on the real line tends to the power spectrum of the stochastic process. That is, if K ij = K i - j are the autocorrelation values and the power spectrum is

 

 \begin{displaymath}
S(\omega) = \Fc[r_k]
\end{displaymath}

 

then

 

 \begin{displaymath}
\lim_{M \rightarrow \infty} \frac{ \lambda_1 + \lambda_2 + \cdots +
\lambda_M}{M} = \frac{1}{2\pi} \int_{-\pi}^\pi S(\omega)d\omega.
\end{displaymath}

 

In this case, the water filling translates to water filling in the spectral domain. The capacity of the channel with noise spectrum N ( f ) can be shown to be

 

 \begin{displaymath}
C = \int \frac{1}{2} \log(1+ \frac{(\nu- N(f))^+}{N(f)}) df
\end{displaymath}

 

where $\nu$ is chosen so that

 

 \begin{displaymath}
\int (\nu - N(f))^+ df = P
\end{displaymath}

 

Copyright 2008, by the Contributing Authors. Cite/attribute Resource . admin. (2006, May 17). The Gaussian Channel. Retrieved January 07, 2011, from Free Online Course Materials — USU OpenCourseWare Web site: http://ocw.usu.edu/Electrical_and_Computer_Engineering/Information_Theory/lecture11_4.htm. This work is licensed under a Creative Commons License Creative Commons License