Personal tools
  •  
You are here: Home Electrical and Computer Engineering Information Theory Application of Information Theory to Blind Source Separation

Application of Information Theory to Blind Source Separation

Document Actions
  • Content View
  • Bookmarks
  • CourseFeed

Introduction   ::   BSS   ::   Mackay's Approach   ::   Natural Gradient   ::   p(u)

So what about p ( u )

We have observed that if $p(u) = \vert\partiald{g}{u}\vert$ , then the last terms of the density go away. We also commented above that the given form works only for super-Gaussian sources. So what do we do in the case where this is not a good assumption. One approach is to deal with two densities. On the one hand, we take

 

\begin{displaymath}p(u) = \frac{1}{2}(N(\mu,\sigma^2) + N(-\mu,\sigma^2))
\end{displaymath}

 

giving (where $a = \mu/\sigma^2)$

 

\begin{displaymath}\phi(u) = \frac{u}{\sigma^2} - a\left(\frac{\exp(au) - \exp(-...
...\right) = \mu/\sigma^2 - \mu/\sigma^2 \tanh(\mu
u/\sigma^2).
\end{displaymath}

 

This p is subGaussian. When $\mu=1$ and $\sigma^2$ = 1 we obtain

 

\begin{displaymath}\phi(u) = - \tanh(u),
\end{displaymath}

 

and the learning rule (employing natural gradient) is

 

\begin{displaymath}\Delta W \propto [I + \tanh(\ubf)\ubf^T - \ubf\ubf^T]W.
\end{displaymath}

 

The superGaussian density is modeled as

 

\begin{displaymath}p(u) \propto [\Nc(0,1)] \sech^2(u)
\end{displaymath}

 

giving rise to

 

\begin{displaymath}\phi(u) = u + \tanh(u)
\end{displaymath}

 

and the learning rule

 

\begin{displaymath}\Delta W \propto [I - \tanh(\ubf)\ubf^T - \ubf\ubf^T]W.
\end{displaymath}

 

Combining these, we obtain

 

\begin{displaymath}\Delta W \propto
\begin{cases}
(I - \tanh(\ubf)\ubf^T - \...
...bf^T - \ubf\ubf^T) W
& \text{sub-Gaussian} \\
\end{cases}
\end{displaymath}

The decision between rules can be made on an element-by-element basis. A decision must be based on the available data.

Copyright 2008, by the Contributing Authors. Cite/attribute Resource . admin. (2006, May 17). Application of Information Theory to Blind Source Separation. Retrieved January 07, 2011, from Free Online Course Materials — USU OpenCourseWare Web site: http://ocw.usu.edu/Electrical_and_Computer_Engineering/Information_Theory/lecture4_5.htm. This work is licensed under a Creative Commons License Creative Commons License