Personal tools
You are here: Home → Electrical and Computer Engineering → Stochastic Processes → Homework Assignments

Homework Assignments

Document Actions
  • Send this
  • Print this
  • Content View
  • Bookmarks
  • CourseFeed

Homework 11

These problems come from the Leon-Garcia text.

  1. Let $ M_n$ denote the sequence of sample means from an i.i.d. random process $ X_n$ :

    $\displaystyle M_n = \frac{X_1 + x_2 + \cdots + X_n}{n} $

    1. Is $ M_n$ a Markov process?
    2. If so, find the state transition p.m.f. $ f_{M_n}(x\vert M_{n-1}=y)$
  2. An urn initially contains five black balls and five white balls. The following experiment is repeated indefinitely. A ball is drawn from the urn; if the ball is white it is put back in the urn, otherwise it is left out. Let $ X_n$ be the number of black balls remaining in the urn after $ n$ draws from the urn.
    1. Is $ X_n$ a Markov process? If so, find the appropriate transition probabilities.
    2. Do the transition probabilities depend on $ n$ ?
  3. Let $ X_n$ be the Bernoulli i.i.d. process and let $ Y_n$ be

    $\displaystyle Y_n = X_n + X_{n-1}. $

    1. Show that $ Y_n$ is not a Markov process.

    2. Now consider the vector process $ \Zbf_n = (X_n,X_{n-1})$ . Show that $ \Zbf_n$ is a Markov process.
    3. Find the state transition diagram for $ \Zbf_n$ .
  4. Show that the following autoregressive process is a Markov process:

    $\displaystyle Y_n = r Y_{n-1} + X_n, $

    with $ Y_n = 0$ , where $ X_n$ is an i.i.d. process.
  5. Let $ X_n$ be the Markov chain in problem 2.
    1. Find the one-step transition probability matrix $ P$ for $ X_n$ .
    2. Find the two-step transition probability matrix $ P^2$ . Check your answer by computing $ p_{54}(2)$ and comparing it to the corresponding entry of $ P^2$
    3. What happens to $ X_n$ as $ n\rightarrow \infty$ ? Use your answer to guess the limit of $ P^n$ as $ n\rightarrow \infty$ .

  6. Two players play the following game. A fair coin is flipped; if the outcome is heads, player A pays player B $1, and if th outcome is tails, player B pays player A $1. The game is continued until one of the players goes broke. Suppose initially that player A has $1 and player B has $2, so a total of $3 is up for grabs. Let $ X_n$ be the number of dollars held by player A after $ n$ rounds.
    1. Show that $ X_n$ is a Markov chain.
    2. Draw the state transition diagram for $ X_n$ and give the one-step transition probability matrix $ P$ .
    3. Use the state transition diagram to help you show that for $ n$ even,

      $\displaystyle p_{ii}(n) = (1/2)^n \qquad i=1,2 $

      $\displaystyle p_{10}(n) = (2/3)(1-(1/4)^(n/2)) = p_{23}(n) $

    4. Find the $ n$ -step transition probability matrix for $ n$ even using part c.
    5. Find the limit of $ P^n$ as $ n\rightarrow \infty$ .
    6. Find the probability that player A eventually wins.
  7. A machine consists of two parts that fail and are repaired independently. A working part fails during any given day with probability $ a$ . A part that is not working is repaired by the next day with probability $ b$ . Let $ X_n$ be the number of working parts in day $ n$ .
    1. Show that $ X_n$ is a three-state Markov chain and give its one-step transition probability matrix $ P$ .
    2. Show that the steady-state pmf $ \pibf$ is binomial with parameter $ p = b/(a+b)$
    3. What do you expect is the steady-state pmf for a machine that consists of $ n$ parts?

Copyright 2008, Todd Moon. Cite/attribute Resource. admin. (2006, June 13). Homework Assignments. Retrieved November 23, 2009, from Free Online Course Materials — USU OpenCourseWare Web site: http://ocw.usu.edu/Electrical_and_Computer_Engineering/Stochastic_Processes/hw11.html. This work is licensed under a Creative Commons License. Creative Commons License
Reuse Course
Download this course