Programming Assignments
Introduction :: System Model :: Expectation :: Part 1 :: Filters :: Part 2 :: Submission
System Model
For brevity, we will employ an operator notation frequently used in the literature. If
we will use the notation
as a shorthand for the filtering operation
A known discretetime input signal is applied to a system which is assumed to be FIR, having transfer function
where we will also assume that the order of the filter is known. The filtered signal is corrupted by an additive noise signal which is assumed to be autoregressive (AR):
where is a zeromean, stationary, ergodic, whitenoise signal with variance , and has the allpole form
The measured output signal is thus
The system identification problem to be addressed here is this: given the input/output measurements , estimate and .
The AR Model and Its Prediction
The model for the noise can be written another way. The IIR filter could be written (say, using long division) as
with . (That is why we call it IIR  it has an infinite number of coefficients in its impulse response.) So
For the development below, we will find it convenient to develop a predictor for . Given the sequence of measurements for , what is the best estimate of ? We will denote this estimate by . If by ``best'' we mean best in the minimum meansquared error sense, we want to find an estimator which minimizes
We know that this will be the conditional mean:
Note that if we know all the data , we equivalently know all the data . Thus can equivalently be written
From the form
we see immediately that this conditional expectation is
since has zero mean. This can be written in our operator notation as
Since , we can write
Let the inverse transfer function be written in the form
where . (Note: It should be clear that in general.) In the case that is in fact an allpole filter as in ( 1 ), we have
so that . Substituting ( 3 ) into ( 2 ) we obtain in the general case
In the case that is an allpole filter, we obtain
That is: the best predictor of given previous measurements of has the form of an FIR filter, as shown in the block diagram here:
is as small as possible. We can write this in a vector form as follows. Let
Then we want to minimize

 Exercise 1:

Show that minimizing the expression in
(
4
) leads to the WidrowHopf normal equations
Also show that the minimum mean squared prediction error is
Prediction of and System Identification
An important step in the overall system identification process is a onestepahead predictor for . Suppose that is known for . What is the best estimate (prediction) that can be made for using all of this information? We will denote this predictor as . Since the input is known, this prediction must be made on the best prediction of given the information up to time , which we have denoted as :
We have seen above that can be written as . We thus have
or
The prediction error is

 Exercise 2

Show that the prediction error can be written as
Let us take the prediction error and write it in two ways. We have
We can also write
Let us consider what we can learn from each of these.
Representation 1
Suppose that
were known. Then in (
6
) we could form
the signal
With the input known and the output known, can be computed. Now rewrite ( 6 ) as
The identification problem at this point is then: Determine the coefficients of the linear predictor with coefficients that minimizes . The idea is suggested by the following figure.

 Exercise 3

Let
Representation 2
Now let us look at the representation in ( 7 ). Suppose that were known, and let
Then ( 7 ) can be written as
A system identification problem can be stated: Find to minimize
The idea is suggested by the following figure.

 Exercise 4

Let
Putting the pieces together
The above sections suggest that:
 Knowing , we can estimate , and
 Knowing , we can estimate .
Pick an initial .
 Solve for an updated using ( 9 ).
 Solve for an updated using ( 11 ).
 Repeat from step 1 until convergence.
A suggested is something like
(put a 0.1 in the middle of the impulse response).