# The Asymptotic Equipartition Property

Introduction :: Convergence :: Data Compression :: Sets

## Where does this lead: data compression

Typical sequences occur most of the time. Therefore, one way to perform data compression is to only provide efficient codes for the typical sequences, and other (less efficient) codes for the non-typical sequences. Since there are sequences in , then we can encode every one of them with no more than bits. Let us prefix each typical-sequence code with a 0, so the total length of codes for typical sequences is .

Each sequence not in can be indexed with not more than bits. If these codes are prefixed with a 1, then we have a code for all sequences in . Even though we are doing a brute-force enumeration of the atypical set (overlooking the fact that we don't need to do those codes that are already in the typical set), we can still do a good job (on the average).

Notation: let
*
x
*
^{
n
}
denote the sequence
.

This theorem does
*
not
*
go so far yet as to say that this is about
the best that we can do.