##### Personal tools
•
You are here: Home Definitions and Basic Facts

# Definitions and Basic Facts

##### Document Actions

Entropy Function   ::   Joint Entropy   ::   Relative Entropy   ::   Multivariable   ::   Convexity

## Binary Entropy Function

We saw last time that the entropy of a random variable X is

Suppose X is a binary random variable,

Then the entropy of X is

Since this depends on p , this is also written sometimes as H ( p ). Plot. Observe: concave function of p . (What does this mean?) H (0) = 0, H (1) = 0. Why? Where is the max? More generally, the entropy of a binary discrete random variable with probability p is written as either H ( X ) or H ( p ).

Copyright 2008, Todd Moon. Cite/attribute Resource . admin. (2006, May 15). Definitions and Basic Facts. Retrieved January 07, 2011, from Free Online Course Materials — USU OpenCourseWare Web site: http://ocw.usu.edu/Electrical_and_Computer_Engineering/Information_Theory/lecture2_1.htm. This work is licensed under a Creative Commons License