Personal tools
You are here: Home → Electrical and Computer Engineering → Stochastic Processes → Introduction; Review of Random Variables

Introduction; Review of Random Variables

Document Actions
  • Send this
  • Print this
  • Content View
  • Bookmarks
  • CourseFeed

Probability   ::   Set Theory   ::   Definition   ::   Conditional   ::   Random   ::   Distribution   ::   R.V.S.

Definition of probability

We now formally define what we mean by a probability space. A probability space has three components.

The first is the sample space, which is the collection of all possible outcomes of some experiment. The outcome space is frequently denoted by $\Omega$.
\begin{example} Suppose the experiment involves throwing a die. \begin{displaymath}\Omega = \{1,2,3,4,5,6\} \end{displaymath}\end{example}
We deal with subsets of $\Omega$. For example, we might have an event which is "all even throws of the die'' or "all outcomes $\leq 4$''. We denote the collection of subsets of interest as $\Fc$. The elements of $\Fc$ (that is, the subsets of $\Omega$) are called events. $\Fc$ is called the event class. We will restrict $\Fc$ to be a $\sigma$-field.
\begin{example} Let $\Omega = \{1,2,3,4,5,6\}$, and let $\Fc = \{ \{1,2,3,4\},... ...{2,4,6\},\{1,3,5\},...\}$ (What do we need to finish this off?) \end{example}

\begin{example} $\Omega = \{1,2,3\}$. We could take $\Fc$ as the set of all su... ...s $\Fc = 2^\Omega$, and is called the {\bf power set} of $\Omega$. \end{example}

\begin{example} $\Omega = \Rbb$. $\Fc$ is restricted to something smaller than... ...intervals of $\Rbb$. This is called the {\bf Borel field}, $\Bc$. \end{example}

The tuple $(\Omega,\Fc)$ is called a pre-probability space (because we haven't assigned probabilities yet). This brings us to the third element of a probability space: Given a pre-probability space $(\Omega,\Fc)$, a probability distribution or a measure on $(\Omega,\Fc)$ is a mapping $P$ from $\Fc$ to $\Rbb$ (which we will write: $P : \Fc \rightarrow \Rbb$) with the properties:

  • $P(\Omega) = 1$ (this is a normalization that is always applied for probabilities, but there are other measures which don't use this)
  • $P(A) \geq 0 \forall A \in \Fc$. (Measures are nonnegative)
  • If $A_1, A_2, \ldots \in \Fc$ such that $A_iA_j = \emptyset$ for all $i \neq j$ (that is, the sets are "disjoint'' or "mutually exclusive'') then

    \begin{displaymath}P(\cup_{i=1}^\infty A_i) = \sum_{i=1}^\infty P(A_i). \end{displaymath}

    This is called the $\sigma$-additive, or additive, property.
These three properties are called the axioms of probability.

The triple $(\Omega,\Fc,P)$ is called a probability space.

  • $\Omega$ tells what individual outcomes are possible
  • $\Fc$ tells what sets of outcomes -- events -- are possible.
  • $P$ tells what the probabilities of these events are.

Some properties of probabilities (which follow from the axioms):

  1. $P(A^c) = 1-P(A)$.

  2. $P(\emptyset) = 0$

  3. $A \subset B \Rightarrow P(A) \leq P(B)$.

  4. $P(A \cup B) = P(A) + P(B) - P(AB)$

  5. If $A_1, A_2, \ldots, \in \Fc$ then $P(\cup_{i=1}^\infty A_i) \leq \sum_{i=1}^\infty P(A_i)$

There is also the continuity of probability property. Suppose $A_1 \subset A_2 \subset A_3 \cdots$ (this is called an increasing sequence). Define

\begin{displaymath}\lim_{n\rightarrow \infty} A_n = \cup_{i=1}^\infty A_i \equiv A. \end{displaymath}

We write this as $A_n \uparrow A$ ($A_n$ converges up to $A$). Similarly, if

\begin{displaymath}A_1 \supset A_2 \supset A_3 \cdots \end{displaymath}

(a decreasing sequence), define

\begin{displaymath}\lim_{n\rightarrow \infty} A_n = \cap_{i=1}^\infty A_i \equiv A_n \downarrow A. \end{displaymath}

If $A_n \uparrow A$, then $P(A_n) \uparrow P(A)$. If $A_n \downarrow A$, then $P(A_n) \downarrow P(A)$.
\begin{example} Let $(\Omega,\Fc) = (\Rbb,\Bc)$. \begin{enumerate} \item Tak... ... \rightarrow 0^-} P((-\infty,x]). \end{displaymath} \end{enumerate}\end{example}
We will introduce more properties later.

Some examples of probability spaces

  1. $\Omega = \{w_1,w_2,\ldots,w_n\}$ (a discrete set of outcomes). $\Fc = 2^\Omega$. Let $p_1,p_2,\ldots, p_n$ be a set of nonnegative numbers satisfying $\sum_{i=1}^n p_i = 1$. Define the function $P : \Fc \rightarrow \Rbb$ by

    \begin{displaymath}P(A) = \sum_{w_i \in A} p_i. \end{displaymath}

    Then $(\Omega,\Fc,P)$ is a probability space.
  2. Uniform distribution: Let $\Omega = [0,1]$, $\Fc = \Bc_{[0,1]}$ =smallest $\sigma$-field containing all intervals in [0,1]. We can take (without proving that this actually works -- but it does)

    \begin{displaymath}P([a,b]) = \vert b-a\vert \end{displaymath}

    for $0 \leq a \leq b \leq 1$, and

    \begin{displaymath}P(\text{unions of disjoint intervals}) = \text{sum of probabilities of individual intervals}. \end{displaymath}

Copyright 2008, by the Contributing Authors. Cite/attribute Resource. admin. (2006, May 31). Introduction; Review of Random Variables. Retrieved November 23, 2009, from Free Online Course Materials — USU OpenCourseWare Web site: http://ocw.usu.edu/Electrical_and_Computer_Engineering/Stochastic_Processes/lec1_3.html. This work is licensed under a Creative Commons License. Creative Commons License
Reuse Course
Download this course