Some More Bounds
The Log Sum Inequality
In this section we introduce an inequality which will allow us to
deduce the concavity (or convexity) of some many useful
Note that the conditions on this theorem are much weaker than for
Jensen's inequality, since it is not necessary to have the sets of
numbers add up to 1.
Using this inequality, we can prove a convexity statement about the
relative entropy function.
The following theorem is important and will be used several times
throughout the quarter.