# Random Vectors

Vectors :: Covariance :: Functions :: Application :: Markov Model

## An Application: MMSE Prediction

Suppose we have a random sequence
, and we
observe the first
of them:

*Given*this data, we want to

*predict*the value of .

Our estimate of
will be denoted as
. Clearly, it could
be a function of all the observed data:

for some function . One thing we could try is to minimize the average of . That is we would like to solve

It is easy to see (HW!) that the best such is

That is, the best estimator (in a

**minimum mean-squared error**sense) is the conditional expectation!

Now let us take a specific distribution. Suppose
, and partition according to

Given , the variable is where

where

and

So is the conditional mean that we want and is the variance of the conditional distribution,

This is the minimum mean-squared error (MMSE).

Notationally, write

Then

This is just a digital filter!

We can also show that , so that incorporating information from measurements decreases our uncertainty.

Note: For a Gaussian r.v., the MMSE estimator is
**
linear
**
.