The Gaussian Channel

Document Actions

Definitions :: Band-limited :: Kuhn-Tucker :: Parallel :: Colored Noise

Channels with colored Gaussian noise

We will extend the results of the previous section now to channels with non-white Gaussian noise. Let K_z be the covariance of the noise K_x the covariance of the input, with the input constrained by

$\begin{displaymath} \frac{1}{n} \sum_i EX_i^2 \leq P \end{displaymath}$

which is the same as

$\begin{displaymath} \frac{1}{n} \trace(K_X) \leq P. \end{displaymath}$

We can write

$\begin{displaymath} I(X_1,\ldots,X_n; Y_1, \ldots ,Y_n) = h(Y_1,\ldots, Y_n) - h(Z_1,\ldots,Z_n) \end{displaymath}$

where

$\begin{displaymath} h(Y_1,\ldots, Y_n) \leq \frac{1}{2} \log((2\pi e)^n|K_x + K_z|) \end{displaymath}$

Now how do we choose K_x to maximize K_x + K_z, subject to the power constraint? Let

$\begin{displaymath} K_z = Q\Lambda Q^T \end{displaymath}$

then

$\begin{displaymath} \begin{aligned} |K_x + K_z| &= |K_x + Q \Lambda Q^T| = |Q| |Q^T K_x Q + \Lambda| |Q^T| \\ &= |Q^T K_x Q + \Lambda| = |A+\lambda| \end{aligned} \end{displaymath}$

where A = Q^T K_x Q. Observe that

$\begin{displaymath} \trace(A) = \trace(Q^T K_x Q) = \trace(Q^T Q K_x) = \trace(K_x) \end{displaymath}$

So we want to maximize $\vert A+\Lambda\vert$ subject to $\trace(A) \leq nP$ . The key is to use an inequality, in this case Hadamard's inequality. Hadamard's inequality follows directly from the "conditioning reduces entropy'' theorem:

$\begin{displaymath} h(X_1,\ldots,X_n) \leq \sum h(X_i). \end{displaymath}$

Let $\Xbf \sim \Nc(0,K)$ . Then

$\begin{displaymath} h(\Xbf) = \frac{1}{2}\log (2\pi e)^n |K| \end{displaymath}$

and

$\begin{displaymath} h(X_i) = \frac{1}{2} \log (2\pi e)K_{ii} \end{displaymath}$

Substituting in and simplifying gives

$\begin{displaymath} |K| \leq \prod_i K_{ii} \end{displaymath}$

with equality iff K is diagonal.

Getting back to our problem,

$\begin{displaymath} |A+\Lambda| \leq \prod_i(A_{ii} + \Lambda_{ii}) \end{displaymath}$

with equality iff A is diagonal. We have

$\begin{displaymath} \frac{1}{n} \sum_i A_{ii} \leq P \end{displaymath}$

(the power constraint), and $A_{ii} \geq 0$ . As before, we take

$\begin{displaymath} A_{ii} = (\nu - \lambda_i)^+ \end{displaymath}$

where $\nu$ is chosen so that

$\begin{displaymath} \sum A_{ii} = nP. \end{displaymath}$

Now we want to generalize to a continuous time system. For a channel with AWGN and covariance matrix K_Z⁽ⁿ⁾, the covariance is Toeplitz. If the channel noise process is stationary, then the covariance matrix is Toeplitz, and the eigenvalues of the covariance matrix tend to a limit as $n \rightarrow \infty$ . The density of the eigenvalues on the real line tends to the power spectrum of the stochastic process. That is, if K_ij = K_i-j are the autocorrelation values and the power spectrum is

$\begin{displaymath} S(\omega) = \Fc[r_k] \end{displaymath}$

then

$\begin{displaymath} \lim_{M \rightarrow \infty} \frac{ \lambda_1 + \lambda_2 + \cdots + \lambda_M}{M} = \frac{1}{2\pi} \int_{-\pi}^\pi S(\omega)d\omega. \end{displaymath}$

In this case, the water filling translates to water filling in the spectral domain. The capacity of the channel with noise spectrum N(f) can be shown to be

$\begin{displaymath} C = \int \frac{1}{2} \log(1+ \frac{(\nu- N(f))^+}{N(f)}) df \end{displaymath}$

where $\nu$ is chosen so that

$\begin{displaymath} \int (\nu - N(f))^+ df = P \end{displaymath}$

Copyright 2008, by the Contributing Authors. Cite/attribute Resource. admin. (2006, May 17). The Gaussian Channel. Retrieved November 24, 2009, from Free Online Course Materials — USU OpenCourseWare Web site: http://ocw.usu.edu/Electrical_and_Computer_Engineering/Information_Theory/lecture11_4.htm. This work is licensed under a Creative Commons License.

Course Contents Information Theory Home About the Professor Syllabus Schedule Homework Assignments Programming Assignments Download This Course	Personal tools Log in You are here: Home → Electrical and Computer Engineering → Information Theory → The Gaussian Channel
	Info The Gaussian Channel Document Actions Definitions :: Band-limited :: Kuhn-Tucker :: Parallel :: Colored Noise Channels with colored Gaussian noise We will extend the results of the previous section now to channels with non-white Gaussian noise. Let K_z be the covariance of the noise K_x the covariance of the input, with the input constrained by $\begin{displaymath} \frac{1}{n} \sum_i EX_i^2 \leq P \end{displaymath}$ which is the same as $\begin{displaymath} \frac{1}{n} \trace(K_X) \leq P. \end{displaymath}$ We can write $\begin{displaymath} I(X_1,\ldots,X_n; Y_1, \ldots ,Y_n) = h(Y_1,\ldots, Y_n) - h(Z_1,\ldots,Z_n) \end{displaymath}$ where $\begin{displaymath} h(Y_1,\ldots, Y_n) \leq \frac{1}{2} \log((2\pi e)^n\|K_x + K_z\|) \end{displaymath}$ Now how do we choose K_x to maximize K_x + K_z, subject to the power constraint? Let $\begin{displaymath} K_z = Q\Lambda Q^T \end{displaymath}$ then $\begin{displaymath} \begin{aligned} \|K_x + K_z\| &= \|K_x + Q \Lambda Q^T\| = \|Q\| \|Q^T K_x Q + \Lambda\| \|Q^T\| \\ &= \|Q^T K_x Q + \Lambda\| = \|A+\lambda\| \end{aligned} \end{displaymath}$ where A = Q^T K_x Q. Observe that $\begin{displaymath} \trace(A) = \trace(Q^T K_x Q) = \trace(Q^T Q K_x) = \trace(K_x) \end{displaymath}$ So we want to maximize $\vert A+\Lambda\vert$ subject to $\trace(A) \leq nP$ . The key is to use an inequality, in this case Hadamard's inequality. Hadamard's inequality follows directly from the "conditioning reduces entropy'' theorem: $\begin{displaymath} h(X_1,\ldots,X_n) \leq \sum h(X_i). \end{displaymath}$ Let $\Xbf \sim \Nc(0,K)$ . Then $\begin{displaymath} h(\Xbf) = \frac{1}{2}\log (2\pi e)^n \|K\| \end{displaymath}$ and $\begin{displaymath} h(X_i) = \frac{1}{2} \log (2\pi e)K_{ii} \end{displaymath}$ Substituting in and simplifying gives $\begin{displaymath} \|K\| \leq \prod_i K_{ii} \end{displaymath}$ with equality iff K is diagonal. Getting back to our problem, $\begin{displaymath} \|A+\Lambda\| \leq \prod_i(A_{ii} + \Lambda_{ii}) \end{displaymath}$ with equality iff A is diagonal. We have $\begin{displaymath} \frac{1}{n} \sum_i A_{ii} \leq P \end{displaymath}$ (the power constraint), and $A_{ii} \geq 0$ . As before, we take $\begin{displaymath} A_{ii} = (\nu - \lambda_i)^+ \end{displaymath}$ where $\nu$ is chosen so that $\begin{displaymath} \sum A_{ii} = nP. \end{displaymath}$ Now we want to generalize to a continuous time system. For a channel with AWGN and covariance matrix K_Z⁽ⁿ⁾, the covariance is Toeplitz. If the channel noise process is stationary, then the covariance matrix is Toeplitz, and the eigenvalues of the covariance matrix tend to a limit as $n \rightarrow \infty$ . The density of the eigenvalues on the real line tends to the power spectrum of the stochastic process. That is, if K_ij = K_i-j are the autocorrelation values and the power spectrum is $\begin{displaymath} S(\omega) = \Fc[r_k] \end{displaymath}$ then $\begin{displaymath} \lim_{M \rightarrow \infty} \frac{ \lambda_1 + \lambda_2 + \cdots + \lambda_M}{M} = \frac{1}{2\pi} \int_{-\pi}^\pi S(\omega)d\omega. \end{displaymath}$ In this case, the water filling translates to water filling in the spectral domain. The capacity of the channel with noise spectrum N(f) can be shown to be $\begin{displaymath} C = \int \frac{1}{2} \log(1+ \frac{(\nu- N(f))^+}{N(f)}) df \end{displaymath}$ where $\nu$ is chosen so that $\begin{displaymath} \int (\nu - N(f))^+ df = P \end{displaymath}$ Copyright 2008, by the Contributing Authors. Cite/attribute Resource. admin. (2006, May 17). The Gaussian Channel. Retrieved November 24, 2009, from Free Online Course Materials — USU OpenCourseWare Web site: http://ocw.usu.edu/Electrical_and_Computer_Engineering/Information_Theory/lecture11_4.htm. This work is licensed under a Creative Commons License.	Reuse Course Download this course

Sections

Personal tools

The Gaussian Channel

Document Actions

Channels with colored Gaussian noise