Personal tools
You are here: Home → Electrical and Computer Engineering → Stochastic Processes → Homework Solutions

Homework Solutions

Document Actions
  • Send this
  • Print this
  • Content View
  • Bookmarks
  • CourseFeed

Utah State University
ECE 6010
Stochastic Processes
Homework # 1 Solutions

  1. Create a list of all the stochastic processes you can think of that might occur in the real world (not just examples from the textbook). Be creative!
  2. We defined a field to be a collection of sets that is closed under complementation and finite unions. Show that such a collection is also closed under finite intersections.

    Given the field $ \Fc$ we know that for $ A, B \in \Fc$ , we have $ A \cup B \in \Fc$ and $ A^c, B^c \in \Fc$ . Generalizing, when $ A_i \in \Fc, i=1,\ldots, n$ , then $ \cup_{i=1}^n A_i \in \Fc$ . Furthermore, $ \cup_{i=1}^n A^c_i \in \Fc$ , as must be the complementary event,

    $\displaystyle \left[\cup_{i=1}^n A^c_i\right]^c $

    Hence, by DeMorgan's theorem,

    $\displaystyle \cap_{i=1}^n A_i \in \Fc. $

  3. Using the axioms of probability, prove the following properties of probability:
    1. $ P(A^c) = 1-P(A)$

      $ A^c \cap A = \emptyset$ and $ A^c \cup A = \Omega$ .

      $ P(A^c \cup A) = P(\Omega) = 1$ but also $ P(A^c \cup A) = P(A^c) + P(A)$ (by additivity), so $ 1 = P(A^c) + P(A)$ .

    2. $ P(\emptyset) = 0$

      $ 1 = P(\Omega) = P(\Omega \cup \emptyset) = P(\Omega) + P(\emptyset)$ .

    3. $ A \subset B \Rightarrow P(A) \leq P(B)$

      If $ A \subset B$ then $ B = A \cup (B \setminus A)$ . So

      $\displaystyle P(B) = P(A \cup (B \setminus A)) = P(A) + P(B \setminus A) \geq P(A). $

    4. $ P(A \cup B) = P(A) + P(B) - P(AB)$

      $\displaystyle P(A \cup B) = P(AB^c \cup A^cB \cup AB) = P(AB^c) + P(A^cB) + P(AB).$ (*)

      Now

      $\displaystyle A = AB \cup AB^c \qquad \qquad B = AB \cup A^cB $

      so

      $\displaystyle P(A) = P(AB) + P(AB^c) \qquad \qquad P(B) = P(AB) + P(A^cB) $

      and so

      $\displaystyle P(AB^c) = P(A) - P(AB) \qquad \qquad P(A^cB) = P(B) P(AB). $

      Substituting these in (**) gives the answer.

    5. $ A_1,A_2,\ldots \in \Fc \Rightarrow P(\cup_{i=1}^\infty A_i) \leq \sum_{i=1}^\infty P(A_i)$

      This is true for $ n=2$ events: $ P(A_1 \cup A_2) \leq P(A_1) + P(A_2)$ . Proof by induction: Assume true for $ n$ :

      $\displaystyle P(A_1 \cup \cdots \cup A_n) \leq \sum_{i=1}^n P(A_i), $

      and show that it holds for $ n+1$ :

      $\displaystyle P(A_1 \cup A_2 \cup \cdots \cup A_{n+1}) = P([A_1 \cup A_2 \cup \cdots \cup A_n] \cup A_{n+1}) $

      But the set $ [A_1 \cup A_2 \cup \cdots \cup A_n]$ describes a single set, which we will call $ B$ . We then have $ P(B \cup A_{n+1}) \leq P(A) + P(A_{n+1})$ . But by the inductive hypothesis, we have a bound on $ P(B)$ :

      $\displaystyle P(B) = P(A_1 \cup \cdots \cup A_n) \leq \sum_{i=1}^n P(A_i), $

      so that

      $\displaystyle P(A_1 \cup \cdots \cup A_n\cup A_{n+1}) \leq \sum_{i=1}^n P(A_i) + P(A_{n+1}) $

  4. Suppose $ P(B) > 0$ 0$" align="middle" border="0" height="37" width="84" /> . Prove the following properties of conditional probability:
    1. $ P(A\vert B) \geq 0$ .

      $ P(A\vert B) = P(AB)/P(B)$ . But since $ P(AB) \geq 0$ , the result follows.

    2. $ P(\Omega\vert B) = 1$

      $ P(\Omega\vert B) = P(\Omega B)/P(B) = P(B)/P(B) = 1$ .

    3. For $ A_1,A_2,\ldots \in \Fc$ with $ A_i A_j = \emptyset$ for $ i \neq j$ , $ P(\cup_{i=1}^\infty A_i\vert B) = \sum_{i=1}^\infty P(A_i\vert B) $

      $ P(\cup_{i=1}^\infty A_i\vert B) = P((\cup_{i=1}^\infty A_i)B)/P(B) = P(\cup_{i... ...y A_iB)/P(B) = \sum_{i=1}^\infty P(A_iB)/P(B) = \sum_{i=1}^\infty P(A_i\vert B)$ , since the $ A_iB$ sets are disjoint.

    4. $ AB=\emptyset \Rightarrow P(A\vert B) = 0.$

      $ P(A\vert B) = P(AB)/P(B) = P(\emptyset)/P(B) = 0.$

    5. $ P(B\vert B) = 1$

      $ P(B\vert B) = P(BB)/P(B) = P(B)/P(B) = 1$

    6. $ A \subset B \Rightarrow P(A\vert B) \geq P(A)$

      Since $ A \subset B$ , $ AB = A$ . Then

      $\displaystyle P(A\vert B) = P(AB)/P(B) = P(A)/P(B) \geq P(A) $

      since $ P(B) \leq 1$ .

    7. $ B \subset A \Rightarrow P(A\vert B) = 1$ .

      Since $ A \subset B$ , $ AB = B$ . Then

      $\displaystyle P(A\vert B) = P(AB)/P(B) = P(B)/P(B) = 1. $

  5. Prove the law of total probability.

    Let $ \{A_i\}$ be a partition of $ A$ . Note that $ A = \cup_{i=1}^n A_i A$ . Note also that $ P(AA_i) = P(A\vert A_i)P(A_i)$ .

    $\displaystyle P(A) = P(\cup_{i=1}^n AA_i) = \sum_{i=1}^n P(AA_i) = \sum_{i=1}^n P(A\vert A_i)P(A_i). $

  6. Prove Bayes rule

    $\displaystyle P(A\vert B) = P(AB)/P(B) = P(B\vert A) P(A)/P(B) $

  7. Suppose $ A$ and $ B$ are independent events. Show that $ A$ and $ B^c$ are also independent.

    By independence, $ P(AB) = P(A) P(B)$ .

    $\displaystyle \begin{aligned} P(A) = P(A(B \cup B^c)) = P(AB \cup AB^c) = P(AB) + P(AB^c) = P(A)P(B) + P(AB^c) \end{aligned}$

    so

    $\displaystyle P(AB^c) = P(A) - P(A)P(B) = P(A)(1-P(B)) = P(A)P(B^c). $

    which means $ A$ and $ B^c$ are independent.
Copyright 2008, Todd Moon. Cite/attribute Resource. admin. (2006, June 13). Homework Solutions. Retrieved November 23, 2009, from Free Online Course Materials — USU OpenCourseWare Web site: http://ocw.usu.edu/Electrical_and_Computer_Engineering/Stochastic_Processes/hw1sol.html. This work is licensed under a Creative Commons License. Creative Commons License
Reuse Course
Download this course