# Maximum Entropy Estimation

Entropy Estimation :: Spectrum Estimation

## Maximum Entropy Estimation

The concept of entropy has been applied to estimation problems. Estimation is the art and science of computing a value when incomplete information is available. It is the incompleteness that makes the concept of maximum entropy useful. For many estimation problems, it is necessary to make assumptions about values which are not explicitly available. One possible choice is to assume that the values are such that the entropy is maximized.

Suppose we want to maximize
*
h
*
(
*
f
*
) over all densities with the following constraints:

We can "pseudo-solve" this as follows. Let

"Differentiate w.r.t.
*
f
*
(
*
x
*
)" (this is the part we are skimping on) and equate to zero:

which leads to

where the Lagrange multipliers are chosen to make
*
f
*
satisfy the constraints.

To show that this actually works, we will use an inequality approach. Let
*
g
*
be a density that satisfies the constraints. Then