Introduction; Review of Random Variables

Document Actions

Probability :: Set Theory :: Definition :: Conditional :: Random :: Distribution :: R.V.S.

Conditional probability and independence

Conditional probability is perhaps the most important probability concept from an engineering point of view. It allows us to describe mathematically how our information changes when we are "given'' a measurement.

We define conditional probability as follows: Suppose $(\Omega,\Fc,P)$ is a probability space and $A,B \in \Fc$ with $P(B) >
0$ 0$" align="middle" border="0" height="31" width="70" />. Define the conditional probability of given as

$\begin{displaymath}P(A\vert B) = \frac{P(AB)}{P(B)} \end{displaymath}$

Essentially what we are saying is that the sample space is restricted from $\Omega$ down to

. Dividing by

provides the correct normalization for this probability measure.

Some properties of conditional probability (consequences of the axioms of probability) are as follows:

$P(A\vert B) \geq 0$ .
$P(\Omega\vert B) = 1$
For $A_1, A_2, \ldots \in \Fc$ with $A_iA_j = \emptyset$ for $i \neq j$ ,

$\begin{displaymath}P(\cup_{i=1}^\infty A_i\vert B) = \sum_{i=1}^\infty P(A_i\vert B) \end{displaymath}$
$AB=\emptyset \Rightarrow P(A\vert B) = 0.$
$P(B\vert B) = 1$
$A \subset B \Rightarrow P(A\vert B) \geq P(A)$
$B \subset A \Rightarrow P(A\vert B) = 1$ .

$\begin{definition} $A_1,A_2,\ldots, A_n \in \Fc$ is a {\bf partition} of $\Ome... ...$i \neq j$, and $\cup_{i=1}^n A_i = \Omega$, and $P(A_i) > 0$. \end{definition}$ 0$. \end{definition}" align="bottom" border="0" height="37" width="555" />

$\begin{example} Let $\Omega = \{1,2,3,4,5,6\}$, and $A_1= \{1\}$, $A_2 = \{2,5,6\}$, $A_3 = \{3,4\}$. \end{example}$

The Law of Total Probability: If $A_1, \ldots, A_n$ is a partition of $\Omega$ and $A \in \Fc$ , then

$\begin{displaymath}P(A) = \sum_{i=1}^n P(A\vert A_i) P(A_i) \end{displaymath}$

(Draw picture).

Bayes Formula is a simple formula for "turning around'' the conditioning. Because conditioning is so important in engineering, Bayes formula turns out to be a tremendously important tool (even though it is very simple). We will see applications of this throughout the semester.

Suppose $A,B \in \Fc$ , $P(A) \neq 0$ and $P(B) \neq 0$ . Then

$\begin{displaymath}P(A\vert B) = \frac{P(B\vert A)P(A)}{P(B)}. \end{displaymath}$

Why?
$\begin{definition} The events $A$ and $B$ are {\bf independent} if \begin{displaymath}P(AB) = P(A)P(B). \end{displaymath}\end{definition}$
Is independent the same as disjoint?

Note: For $P(B) >
0$ 0$" align="middle" border="0" height="31" width="70" />, if and are independent then $P(A\vert B) = P(A)$ . (Since they are independent, can provide no information about , so the probability remains unchanged. If , then is independent of for any other event $A \in \Fc$ . (Why?).

$\begin{definition} $A_1,\ldots,A_n \in \Fc$ are {\bf independent} if for each ... ...p_{j=1}^k A_{i_j}) = \prod_{j=1}^k P(A_{i_j}). \end{displaymath}\end{definition}$

$\begin{example} Take $n=3$. Independent if: \begin{displaymath}P(A_1A_2) = P(A_... ...in{displaymath}P(A_1A_2A_3) = P(A_1)P(A_2)P(A_3). \end{displaymath}\end{example}$

The next idea is important in a lot of practical problem of engineering interest.
$\begin{definition} $A_1$ and $A_2$ are {\bf conditionally independent} given ... ...P(A_1A_2\vert B) = P(A_1\vert B) P(A_2\vert B) \end{displaymath}\end{definition}$
(draw picture to illustrate the idea).

Copyright 2008, by the Contributing Authors. Cite/attribute Resource. admin. (2006, May 31). Introduction; Review of Random Variables. Retrieved November 24, 2009, from Free Online Course Materials — USU OpenCourseWare Web site: http://ocw.usu.edu/Electrical_and_Computer_Engineering/Stochastic_Processes/lec1_4.html. This work is licensed under a Creative Commons License.

Course Contents Stochastic Processes Home About the Professor Syllabus Schedule Homework Assignments Homework Solutions Programming Assignments	Personal tools Log in You are here: Home → Electrical and Computer Engineering → Stochastic Processes → Introduction; Review of Random Variables
	Info Introduction; Review of Random Variables Document Actions Probability :: Set Theory :: Definition :: Conditional :: Random :: Distribution :: R.V.S. Conditional probability and independence Conditional probability is perhaps the most important probability concept from an engineering point of view. It allows us to describe mathematically how our information changes when we are "given'' a measurement. We define conditional probability as follows: Suppose $(\Omega,\Fc,P)$ is a probability space and $A,B \in \Fc$ with 0$" align="middle" border="0" height="31" width="70" />. Define the conditional probability of given as $\begin{displaymath}P(A\vert B) = \frac{P(AB)}{P(B)} \end{displaymath}$ Essentially what we are saying is that the sample space is restricted from $\Omega$ down to . Dividing by provides the correct normalization for this probability measure. Some properties of conditional probability (consequences of the axioms of probability) are as follows: $P(A\vert B) \geq 0$ . $P(\Omega\vert B) = 1$ For $A_1, A_2, \ldots \in \Fc$ with $A_iA_j = \emptyset$ for $i \neq j$ , $\begin{displaymath}P(\cup_{i=1}^\infty A_i\vert B) = \sum_{i=1}^\infty P(A_i\vert B) \end{displaymath}$ $AB=\emptyset \Rightarrow P(A\vert B) = 0.$ $P(B\vert B) = 1$ $A \subset B \Rightarrow P(A\vert B) \geq P(A)$ $B \subset A \Rightarrow P(A\vert B) = 1$ . $\begin{definition} $A_1,A_2,\ldots, A_n \in \Fc$ is a {\bf partition} of $\Ome... ...$i \neq j$, and $\cup_{i=1}^n A_i = \Omega$, and $P(A_i) > 0$. \end{definition}$ 0$. \end{definition}" align="bottom" border="0" height="37" width="555" /> $\begin{example} Let $\Omega = \{1,2,3,4,5,6\}$, and $A_1= \{1\}$, $A_2 = \{2,5,6\}$, $A_3 = \{3,4\}$. \end{example}$ The Law of Total Probability: If $A_1, \ldots, A_n$ is a partition of $\Omega$ and $A \in \Fc$ , then $\begin{displaymath}P(A) = \sum_{i=1}^n P(A\vert A_i) P(A_i) \end{displaymath}$ (Draw picture). Bayes Formula is a simple formula for "turning around'' the conditioning. Because conditioning is so important in engineering, Bayes formula turns out to be a tremendously important tool (even though it is very simple). We will see applications of this throughout the semester. Suppose $A,B \in \Fc$ , $P(A) \neq 0$ and $P(B) \neq 0$ . Then $\begin{displaymath}P(A\vert B) = \frac{P(B\vert A)P(A)}{P(B)}. \end{displaymath}$ Why? $\begin{definition} The events $A$ and $B$ are {\bf independent} if \begin{displaymath}P(AB) = P(A)P(B). \end{displaymath}\end{definition}$ Is independent the same as disjoint? Note: For 0$" align="middle" border="0" height="31" width="70" />, if and are independent then $P(A\vert B) = P(A)$ . (Since they are independent, can provide no information about , so the probability remains unchanged. If , then is independent of for any other event $A \in \Fc$ . (Why?). $\begin{definition} $A_1,\ldots,A_n \in \Fc$ are {\bf independent} if for each ... ...p_{j=1}^k A_{i_j}) = \prod_{j=1}^k P(A_{i_j}). \end{displaymath}\end{definition}$ $\begin{example} Take $n=3$. Independent if: \begin{displaymath}P(A_1A_2) = P(A_... ...in{displaymath}P(A_1A_2A_3) = P(A_1)P(A_2)P(A_3). \end{displaymath}\end{example}$ The next idea is important in a lot of practical problem of engineering interest. $\begin{definition} $A_1$ and $A_2$ are {\bf conditionally independent} given ... ...P(A_1A_2\vert B) = P(A_1\vert B) P(A_2\vert B) \end{displaymath}\end{definition}$ (draw picture to illustrate the idea). Copyright 2008, by the Contributing Authors. Cite/attribute Resource. admin. (2006, May 31). Introduction; Review of Random Variables. Retrieved November 24, 2009, from Free Online Course Materials — USU OpenCourseWare Web site: http://ocw.usu.edu/Electrical_and_Computer_Engineering/Stochastic_Processes/lec1_4.html. This work is licensed under a Creative Commons License.	Reuse Course Download this course

Sections

Personal tools

Introduction; Review of Random Variables

Document Actions

Conditional probability and independence