# Definitions and Basic Facts

Entropy Function :: Joint Entropy :: Relative Entropy :: Multivariable :: Convexity

## Binary Entropy Function

We saw last time that the entropy of a random variable
*
X
*
is

Suppose

*X*is a binary random variable,

Then the entropy of

*X*is

Since this depends on
*
p
*
, this is also written sometimes as
*
H
*
(
*
p
*
).
Plot. Observe:
**
concave
**
function of
*
p
*
. (What does this mean?)
*
H
*
(0) = 0,
*
H
*
(1) = 0. Why? Where is the max?
More generally, the entropy of a binary discrete random variable with
probability
*
p
*
is written as either
*
H
*
(
*
X
*
) or
*
H
*
(
*
p
*
).