##### Personal tools
•
You are here: Home The Asymptotic Equipartition Property

# The Asymptotic Equipartition Property

##### Document Actions

Introduction   ::   Convergence   ::   Data Compression   ::   Sets

## High probability sets and the typical set

We introduce here some notation that is useful in later chapters.

By these examples, we are in a better position to understand the following definition.

Now, the typical set is a fairly small set that contains most of the probability, but it is not clear if it is the smallest such set. We will show that any small but highly probable set must contain a significant overlap with the typical set.

Then we have the following theorem:

To contrast, consider X i be Bernoulli(.9). Then a typical set contains sequences in which the proportion of 1's is close to .9. But it does not contain the single most probable sequence, the sequence of all 1s. The set (the high probability set) will include all the most probable sequences, including the sequence of all 1s. By the theorem, A and B must both contain the sequences that are about 90% 1s, and that the two sets are almost equal in size.

Copyright 2008, by the Contributing Authors. Cite/attribute Resource . admin. (2006, May 17). The Asymptotic Equipartition Property. Retrieved January 07, 2011, from Free Online Course Materials — USU OpenCourseWare Web site: http://ocw.usu.edu/Electrical_and_Computer_Engineering/Information_Theory/equi_3.htm. This work is licensed under a Creative Commons License