##### Personal tools
•
You are here: Home Some More Bounds

# Some More Bounds

##### Document Actions

Log Sum Inequality   ::   Data Processing Inequality   ::   Fano's Inequality

## The data processing inequality

The data processing inequality is a simple but interesting theorem that states (in essence) the following: no matter what processing you do on some data, you cannot get more information out of a set of data than was there to begin with. In a sense, it provide a bound on how much can be accomplished with signal processing.

A Markov chain is at the heart of the "state" idea in differential equations and is used commonly in controls. The concept of a state is that knowing the present state, the future of the system of independent of the past . In other words, the state provides all the information necessary to move into the future: the necessary initial conditions of the differential equations.

The "conditional independence" idea means

Note that if Z = f ( Y ) then .

Interpretation: If we think of Z as being the result of some processing that is done on the data Y , that is, Z = f ( Y ) for some function, deterministic or random, then there is no function that can increase the amount of information that Y tells about X .

Copyright 2008, by the Contributing Authors. Cite/attribute Resource . admin. (2006, May 17). Some More Bounds. Retrieved January 07, 2011, from Free Online Course Materials — USU OpenCourseWare Web site: http://ocw.usu.edu/Electrical_and_Computer_Engineering/Information_Theory/lecture3_2.htm. This work is licensed under a Creative Commons License