Personal tools
  •  
You are here: Home Electrical and Computer Engineering Stochastic Processes Introduction; Review of Random Variables

Introduction; Review of Random Variables

Document Actions
  • Content View
  • Bookmarks
  • CourseFeed

Probability   ::   Set Theory   ::   Definition   ::   Conditional   ::   Random   ::   Distribution   ::   R.V.S.

Definition of probability

We now formally define what we mean by a probability space. A probability space has three components.

The first is the sample space , which is the collection of all possible outcomes of some experiment. The outcome space is frequently denoted by $\Omega$ .
\begin{example}
Suppose the experiment involves throwing a die.
\begin{displaymath}\Omega = \{1,2,3,4,5,6\}
\end{displaymath}\end{example}
We deal with subsets of $\Omega$ . For example, we might have an event which is "all even throws of the die'' or "all outcomes $\leq
4$ ''. We denote the collection of subsets of interest as $\Fc$ . The elements of $\Fc$ (that is, the subsets of $\Omega$ ) are called events . $\Fc$ is called the event class . We will restrict $\Fc$ to be a $\sigma$ -field .
\begin{example}
Let $\Omega = \{1,2,3,4,5,6\}$, and let $\Fc = \{
\{1,2,3,4\},...
...{2,4,6\},\{1,3,5\},...\}$ (What do we need to
finish this off?)
\end{example}

\begin{example}
$\Omega = \{1,2,3\}$. We could take $\Fc$ as the set of all su...
...s $\Fc = 2^\Omega$, and is called the {\bf power set} of $\Omega$.
\end{example}

\begin{example}
$\Omega = \Rbb$. $\Fc$ is restricted to something smaller than...
...intervals of $\Rbb$. This is called the {\bf Borel
field}, $\Bc$.
\end{example}

The tuple $(\Omega,\Fc)$ is called a pre-probability space (because we haven't assigned probabilities yet). This brings us to the third element of a probability space: Given a pre-probability space $(\Omega,\Fc)$ , a probability distribution or a measure on $(\Omega,\Fc)$ is a mapping $P$ from $\Fc$ to $\Rbb$ (which we will write: $P : \Fc \rightarrow \Rbb$ ) with the properties:

  • $P(\Omega) = 1$ (this is a normalization that is always applied for probabilities, but there are other measures which don't use this)
  • $P(A) \geq 0 \forall A \in \Fc$ . (Measures are nonnegative)
  • If $A_1, A_2, \ldots \in \Fc$ such that $A_iA_j = \emptyset$ for all $i \neq j$ (that is, the sets are "disjoint'' or "mutually exclusive'') then

    \begin{displaymath}P(\cup_{i=1}^\infty A_i) = \sum_{i=1}^\infty P(A_i).
\end{displaymath}

    This is called the $\sigma$ -additive, or additive, property.
These three properties are called the axioms of probability .

The triple $(\Omega,\Fc,P)$ is called a probability space .

  • $\Omega$ tells what individual outcomes are possible
  • $\Fc$ tells what sets of outcomes -- events -- are possible.
  • $P$ tells what the probabilities of these events are.

Some properties of probabilities (which follow from the axioms):

  1. $P(A^c) = 1-P(A)$ .

  2. $P(\emptyset) = 0$

  3. $A \subset B \Rightarrow P(A) \leq P(B)$ .

  4. $P(A \cup B) = P(A) + P(B) - P(AB)$

  5. If $A_1, A_2, \ldots, \in \Fc$ then $P(\cup_{i=1}^\infty A_i)
\leq \sum_{i=1}^\infty P(A_i)$

There is also the continuity of probability property. Suppose $A_1 \subset A_2 \subset A_3 \cdots$ (this is called an increasing sequence). Define

\begin{displaymath}\lim_{n\rightarrow \infty} A_n = \cup_{i=1}^\infty A_i \equiv A.
\end{displaymath}

We write this as $A_n \uparrow A$ ( $A_n$ converges up to $A$ ). Similarly, if

\begin{displaymath}A_1 \supset A_2 \supset A_3 \cdots
\end{displaymath}

(a decreasing sequence), define

\begin{displaymath}\lim_{n\rightarrow \infty} A_n = \cap_{i=1}^\infty A_i \equiv A_n
\downarrow A.
\end{displaymath}

If $A_n \uparrow A$ , then $P(A_n) \uparrow P(A)$ . If $A_n \downarrow A$ , then $P(A_n) \downarrow P(A)$ .
\begin{example}
Let $(\Omega,\Fc) = (\Rbb,\Bc)$.
\begin{enumerate}
\item Tak...
... \rightarrow 0^-} P((-\infty,x]).
\end{displaymath} \end{enumerate}\end{example}
We will introduce more properties later.

Some examples of probability spaces

  1. $\Omega = \{w_1,w_2,\ldots,w_n\}$ (a discrete set of outcomes). $\Fc = 2^\Omega$ . Let $p_1,p_2,\ldots, p_n$ be a set of nonnegative numbers satisfying $\sum_{i=1}^n p_i = 1$ . Define the function $P : \Fc \rightarrow \Rbb$ by

    \begin{displaymath}P(A) = \sum_{w_i \in A} p_i.
\end{displaymath}

    Then $(\Omega,\Fc,P)$ is a probability space.
  2. Uniform distribution: Let $\Omega = [0,1]$ , $\Fc = \Bc_{[0,1]}$ =smallest $\sigma$ -field containing all intervals in [0,1]. We can take (without proving that this actually works -- but it does)

    \begin{displaymath}P([a,b]) = \vert b-a\vert
\end{displaymath}

    for $0 \leq a \leq b \leq 1$ , and

    \begin{displaymath}P(\text{unions of disjoint intervals}) = \text{sum of probabilities
of individual intervals}.
\end{displaymath}

Copyright 2008, by the Contributing Authors. Cite/attribute Resource . admin. (2006, May 31). Introduction; Review of Random Variables. Retrieved January 07, 2011, from Free Online Course Materials — USU OpenCourseWare Web site: http://ocw.usu.edu/Electrical_and_Computer_Engineering/Stochastic_Processes/lec1_3.html. This work is licensed under a Creative Commons License Creative Commons License