# The Asymptotic Equipartition Property

Introduction :: Convergence :: Data Compression :: Sets

## High probability sets and the typical set

We introduce here some notation that is useful in later chapters.

By these examples, we are in a better position to understand the
following definition.

Now, the typical set
is a fairly small set that
contains most of the probability, but it is not clear if it is the
smallest such set. We will show that any small but highly probable
set must contain a significant overlap with the typical set.

Then we have the following theorem:

To contrast, consider
*
X
*
_{
i
}
be Bernoulli(.9). Then a typical set
contains sequences in which the proportion of 1's is close to .9. But
it does
*
not
*
contain the single most probable sequence, the
sequence of all 1s. The set
(the high probability
set) will include all the most probable sequences, including the
sequence of all 1s. By the theorem,
*
A
*
and
*
B
*
must both contain the
sequences that are about 90% 1s, and that the two sets are almost
equal in size.