# Application of Information Theory to Blind Source Separation

Introduction :: BSS :: Mackay's Approach :: Natural Gradient :: p(u)

## Mackay's approach

We consider the BSS problem as a ML estimation problem. To begin with, we state a lemma that we will need.

Given a sequence of observed data obtained from we write down the joint probability as

To do ML estimation of

*A*we examine

The ML solution is to find the

*A*that maximizes this likelihood function. We have

where the lemma we derived before has been used. The log likelihood function is

Let

*W*=

*A*

^{ -1 }. Then

Now we proceed as before, computing the derivative of the log likelihood with respect to

*W*. The first part is easy:

For the first part:

Let

and let . Then we have

Thus

Compare with what we had before!