Personal tools
  •  
You are here: Home Electrical and Computer Engineering Stochastic Processes Sequences and Limit Theorems

Sequences and Limit Theorems

Document Actions
  • Content View
  • Bookmarks
  • CourseFeed

Sequences  ::  Convergence  ::  Limit  ::  Central Limit

Modes of convergence of sequences of r.v.s

Suppose $X_1, X_2, \ldots$ is a sequence of random variables defined on $(\Omega,\Fc,P)$ . How can we define a limit of this sequence? As it turns out, there are several different (and inequivalent) ways of defining convergence.

Almost sure convergence

This is a very strong form of convergence, and usually quite difficult to prove.
\begin{definition}
A sequence of r.v.s $\{X_n\}_{n=1}^\infty$ converges {\bf a...
...ymath}This is also called convergence {\bf with probability 1.}
\end{definition}
One tool for showing a.s. convergence is the following fact:

$X_n \rightarrow X$ a.s. if and only if

\begin{displaymath}P(\lim_{n\rightarrow \infty} \sup_{m > n} \vert X_n - X_m\vert = 0) = 1.
\end{displaymath}


\begin{example}
Let $\Omega = [0,1]$, $\Fc = \Bc[0,1]$. Let $X_n(\omega) =
ne^...
...{0\}) >
0$ then $X_n$ doesn't converge in the almost sure sense.
\end{example}

Mean-square convergence

This is a strong mode of convergence which is usually easier to show than a.s. It is widely used in engineering.
\begin{definition}
The sequence $\{X_n\}_{n=1}^\infty$ converges to the r.v. $...
...\lim_{n\rightarrow \infty} E[(X_n - X)^2] = 0.
\end{displaymath}\end{definition}
We write $X_n \rightarrow X$ (m.s.) or $X_n \rightarrow X$ (q.m.) ''quadratic mode.''

There is a Cauchy criterion for m.s. convergence: If $E[X_n^2] <
\infty$ for all $n \in \Zbb^+$ , then $\{X_n\}$ converges in mean-square if and only if

\begin{displaymath}\lim_{n\rightarrow \infty} \sup_{m > n} E[(X_m - X_n)^2] = 0.
\end{displaymath}


\begin{example}
Let $\Omega = [0,1]$, $\Fc = \Bc[0,1]$, and $P$ is uniform:
$...
...rightarrow 0$ m.s.
\par
What about a.s. convergence in this case?
\end{example}
Here is an interesting fact: If $X_n \rightarrow X$ (m.s.) and $X_n
\rightarrow Y$ (a.s.), then $X = Y$ (a.s.).

Convergence in Probability


\begin{definition}
The sequence $\{X_n\}_{n=1}^\infty$ converges to $X$ {\bf i...
...ert X_n - X\vert \leq \epsilon) \rightarrow 1.
\end{displaymath}\end{definition}

\begin{example}
Let $\Omega = [0,1]$, $\Fc = \Bc[0,1]$, and $P$ is uniform. Le...
...1/n \rightarrow 0,
\end{displaymath}so $X_n \rightarrow 0$ (i.p.)
\end{example}

Convergence in Distribution


\begin{definition}
The sequence $\{X_n\}_{n=1}^\infty$ converges {\bf in distr...
..._n}(x)
\rightarrow F_X(x)$ at all continuous points of $F_X$.
\end{definition}

\begin{example}
$\Omega = [0,1]$, $\Fc = \Bc[0,1]$, $P$ is uniform. Let
\begin...
...\rightarrow X$, where $F_X(x) = u(x-1)$. (Draw the distributions.)
\end{example}
Note from this example that the $X_n$ values don't really ''approach'' any value -- the values are still 1 and 0. This is in distinction to the first three modes of convergence, in which $\vert X_n - X\vert \rightarrow
0$ in some sense.

By the definition of this mode of convergence, we don't have to worry about the points of discontinuity of $F_{X_n}$ .
\begin{example}
Let $X_n = 1/n$ for all $\omega \in \Omega$. (So it doesn't ma...
... discontinuity.
\par
Hence $X_n \rightarrow 0$ (in distribution).
\end{example}

Why and Which?

We have defined several different modes of convergence. Why so many? The basic answer is that they are inequivalent -- one does not meet all the analytical needs. Some are stronger than others.

\begin{displaymath}X_n \rightarrow X \text{ (m.s.)} \Rightarrow X_n \rightarrow X
\text{ (i.p.)}
\end{displaymath}


\begin{displaymath}X_n \rightarrow X \text{ (a.s.)} \Rightarrow X_n \rightarrow X
\text{ (i.p.)}
\end{displaymath}


\begin{displaymath}X_n \rightarrow X \text{ (i.p..)} \Rightarrow X_n \rightarrow X
\text{ (in distribution)}
\end{displaymath}

So convergence in distribution is weaker than i.p., m.s. or a.s.

In general, none of the implications can be reversed. And m.s. and a.s. do not imply each other. (Venn diagram - dist. on the outside, then i.p., with m.s. and a.s. overlapping inside.)


\begin{proof}of $X_n \rightarrow X \text{ (m.s.)} \Rightarrow X_n
\rightarrow X...
...P(\vert X_n - X\vert > \epsilon) \rightarrow 0$ for all $\epsilon$.
\end{proof}


\begin{proof}
of $X_n \rightarrow X \text{ (a.s.)} \Rightarrow X_n \rightarrow ...
...> \epsilon) \leq P(B_n)$, which we just showed $\rightarrow 0$.
\par
\end{proof}

\begin{proof}
of $X_n \rightarrow X \text{ (i.p.)} \Rightarrow X_n \rightarrow ...
...h}So $F_{X_n}(x) \rightarrow F_X(x)$. (Convergence in distribution.)
\end{proof}

Some examples of invalid implications

To see which modes are ''stronger'' than others, we can consider some counterexamples.
\begin{example}
Let $X_n \rightarrow X$ (i.p.) Can we say that $X_n \rightarro...
...ghtarrow 0$ (a.s.), we also see that a.s. $\not
\Rightarrow $m.s.
\end{example}

\begin{example}
Does i.p. imply a.s.? Define a sequence of r.v.s as follows on
...
...se the 2nd moment
is $P(X_n=1) \approx 1/\log_2(n) \rightarrow 0$.
\end{example}

\begin{example}
What about convergence in distribution and convergence i.p.?
\p...
...ution depends on marginals, which don't tell us
the whole picture.
\end{example}

Some other relationships:

  1. If $X_n \rightarrow X$ (i.p.) then there is a subsequence $\{
X_{n_k}\}_{k=1}^\infty$ such that $\lim_{k\rightarrow \infty}X_{n_k} =
X$ (a.s.)
  2. If $X_n \rightarrow X$ and there is a r.v. $Y$ with finite second moment such that $\vert X_n\vert \leq Y$ (a.s.) for every $n \in \Zbb^+$ , then $X_n \rightarrow X$ (m.s.).
  3. If $X_n \rightarrow C$ (in distribution), then $X_n \rightarrow C$ (i.p.)
Copyright 2008, by the Contributing Authors. Cite/attribute Resource . admin. (2006, May 31). Sequences and Limit Theorems. Retrieved January 07, 2011, from Free Online Course Materials — USU OpenCourseWare Web site: http://ocw.usu.edu/Electrical_and_Computer_Engineering/Stochastic_Processes/lec5_2.html. This work is licensed under a Creative Commons License Creative Commons License