Personal tools
  •  
You are here: Home Electrical and Computer Engineering Stochastic Processes Introduction; Review of Random Variables

Introduction; Review of Random Variables

Document Actions
  • Content View
  • Bookmarks
  • CourseFeed

Probability   ::   Set Theory   ::   Definition   ::   Conditional   ::   Random   ::   Distribution   ::   R.V.S.

Conditional probability and independence

Conditional probability is perhaps the most important probability concept from an engineering point of view. It allows us to describe mathematically how our information changes when we are "given'' a measurement.

We define conditional probability as follows: Suppose $(\Omega,\Fc,P)$ is a probability space and $A,B \in \Fc$ with $P(B) >
0$ . Define the conditional probability of $A$ given $B$ as

\begin{displaymath}P(A\vert B) = \frac{P(AB)}{P(B)}
\end{displaymath}

Essentially what we are saying is that the sample space is restricted from $\Omega$ down to $B$ . Dividing by $P(B)$ provides the correct normalization for this probability measure.

Some properties of conditional probability (consequences of the axioms of probability) are as follows:

  1. $P(A\vert B) \geq 0$ .
  2. $P(\Omega\vert B) = 1$
  3. For $A_1, A_2, \ldots \in \Fc$ with $A_iA_j = \emptyset$ for $i \neq j$ ,

    \begin{displaymath}P(\cup_{i=1}^\infty A_i\vert B) = \sum_{i=1}^\infty P(A_i\vert B)
\end{displaymath}

  4. $AB=\emptyset \Rightarrow P(A\vert B) = 0.$
  5. $P(B\vert B) = 1$
  6. $A \subset B \Rightarrow P(A\vert B) \geq P(A)$
  7. $B \subset A \Rightarrow P(A\vert B) = 1$ .


\begin{definition}
$A_1,A_2,\ldots, A_n \in \Fc$ is a {\bf partition} of $\Ome...
...$i \neq j$, and $\cup_{i=1}^n A_i =
\Omega$, and $P(A_i) > 0$.
\end{definition}

\begin{example}
Let $\Omega = \{1,2,3,4,5,6\}$, and $A_1= \{1\}$, $A_2 =
\{2,5,6\}$, $A_3 = \{3,4\}$.
\end{example}

The Law of Total Probability : If $A_1, \ldots, A_n$ is a partition of $\Omega$ and $A \in \Fc$ , then

\begin{displaymath}P(A) = \sum_{i=1}^n P(A\vert A_i) P(A_i)
\end{displaymath}

(Draw picture).

Bayes Formula is a simple formula for "turning around'' the conditioning. Because conditioning is so important in engineering, Bayes formula turns out to be a tremendously important tool (even though it is very simple). We will see applications of this throughout the semester.

Suppose $A,B \in \Fc$ , $P(A) \neq 0$ and $P(B) \neq 0$ . Then

\begin{displaymath}P(A\vert B) = \frac{P(B\vert A)P(A)}{P(B)}.
\end{displaymath}

Why?
\begin{definition}
The events $A$ and $B$ are {\bf independent} if
\begin{displaymath}P(AB) = P(A)P(B).
\end{displaymath}\end{definition}
Is independent the same as disjoint?

Note: For $P(B) >
0$ , if $A$ and $B$ are independent then $P(A\vert B) =
P(A)$ . (Since they are independent, $B$ can provide no information about $A$ , so the probability remains unchanged. If $P(B)= 0$ , then $B$ is independent of $A$ for any other event $A \in \Fc$ . (Why?).


\begin{definition}
$A_1,\ldots,A_n \in \Fc$ are {\bf independent} if for each ...
...p_{j=1}^k A_{i_j}) = \prod_{j=1}^k P(A_{i_j}).
\end{displaymath}\end{definition}

\begin{example}
Take $n=3$. Independent if:
\begin{displaymath}P(A_1A_2) = P(A_...
...in{displaymath}P(A_1A_2A_3) = P(A_1)P(A_2)P(A_3).
\end{displaymath}\end{example}

The next idea is important in a lot of practical problem of engineering interest.
\begin{definition}
$A_1$ and $A_2$ are {\bf conditionally independent} given ...
...P(A_1A_2\vert B) = P(A_1\vert B) P(A_2\vert B)
\end{displaymath}\end{definition}
(draw picture to illustrate the idea).

Copyright 2008, by the Contributing Authors. Cite/attribute Resource . admin. (2006, May 31). Introduction; Review of Random Variables. Retrieved January 07, 2011, from Free Online Course Materials — USU OpenCourseWare Web site: http://ocw.usu.edu/Electrical_and_Computer_Engineering/Stochastic_Processes/lec1_4.html. This work is licensed under a Creative Commons License Creative Commons License