# Some More Bounds

Log Sum Inequality :: Data Processing Inequality :: Fano's Inequality

## The data processing inequality

The data processing inequality is a simple but interesting theorem that states (in essence) the following: no matter what processing you do on some data, you cannot get more information out of a set of data than was there to begin with. In a sense, it provide a bound on how much can be accomplished with signal processing.

A Markov chain is at the heart of the "state" idea in differential equations and is used commonly in controls. The concept of a state is that
*
knowing the present state, the future of the system of independent of the past
*
. In other words, the state provides all the information necessary to move into the future: the necessary initial conditions of the differential equations.

The "conditional independence" idea means

Note that if

*Z*=

*f*(

*Y*) then .

Interpretation: If we think of

*Z*as being the result of some processing that is done on the data

*Y*, that is,

*Z*=

*f*(

*Y*) for some function, deterministic or random, then there is no function that can increase the amount of information that

*Y*tells about

*X*.