# Differential Entropy

Differential Entropy :: AEP :: Discretization :: Other Entropy

## Joint, Conditional, and Relative Differential Entropy

More basic definitions:

is the

**joint differential entropy**.

is the

**conditional differential entropy**.

An important special case is the following:

The
**
relative
**
entropy is

The
**
mutual information
**
is

Some properties:

An important result is the following:

That is, for a given covariance, the normal (Gaussian) distribution
has the one which maximizes the entropy.