Personal tools
You are here: Home Electrical and Computer Engineering Information Theory Differential Entropy

Differential Entropy

Document Actions
  • Content View
  • Bookmarks
  • CourseFeed

Differential Entropy   ::   AEP   ::   Discretization   ::   Other Entropy

Differential entropy

A continuous random variable (for the purpose of this class) is one in which the distribution function F ( x ) is continuous: there are no jumps (discrete outputs).

The {\bf differential entropy} $h(X)$\ of a continuous rando...
...displaymath}h(X) = -\int_{S} f(x)\log f(x) dx.

Let $X \sim \Uc(0,a)$. (Uniform). Then
\begin{displaymath}h(X) ...
...em differential} entropy, since entropy
should always be positive.

Normal: $X \sim \frac{1}{\sigma\sqrt{2\pi}}e^{-x^2/2\sigma^2} =...
...1}{2} \ln \pi e \sigma^2\text{ nats}

Having defined the differential entropy, we can now go through and define all the sorts of things we did before.

Copyright 2008, Todd Moon. Cite/attribute Resource . admin. (2006, May 15). Differential Entropy. Retrieved January 07, 2011, from Free Online Course Materials — USU OpenCourseWare Web site: This work is licensed under a Creative Commons License Creative Commons License