Homework Assignments
Homework 11
These problems come from the LeonGarcia text.

Let
denote the sequence of sample means from an
i.i.d. random process
:
 Is a Markov process?
 If so, find the state transition p.m.f.

An urn initially contains five black balls and five white
balls. The following experiment is repeated indefinitely. A ball
is drawn from the urn; if the ball is white it is put back in the
urn, otherwise it is left out. Let
be the number of black
balls remaining in the urn after
draws from the urn.
 Is a Markov process? If so, find the appropriate transition probabilities.
 Do the transition probabilities depend on ?

Let
be the Bernoulli i.i.d. process and let
be

Show that
is not a Markov process.
 Now consider the vector process . Show that is a Markov process.
 Find the state transition diagram for .

Show that
is not a Markov process.

Show that the following autoregressive process is a Markov
process:

Let
be the Markov chain in problem 2.
 Find the onestep transition probability matrix for .
 Find the twostep transition probability matrix . Check your answer by computing and comparing it to the corresponding entry of
 What happens to as ? Use your answer to guess the limit of as .

Two players play the following game. A fair coin is flipped; if
the outcome is heads, player A pays player B $1, and if th outcome
is tails, player B pays player A $1. The game is continued until
one of the players goes broke. Suppose initially that player A has
$1 and player B has $2, so a total of $3 is up for grabs. Let
be the number of dollars held by player A after
rounds.
 Show that is a Markov chain.
 Draw the state transition diagram for and give the onestep transition probability matrix .

Use the state transition diagram to help you show that for
even,
 Find the step transition probability matrix for even using part c.
 Find the limit of as .
 Find the probability that player A eventually wins.

A machine consists of two parts that fail and are repaired
independently. A working part fails during any given day with
probability
. A part that is not working is repaired by the next
day with probability
. Let
be the number of working parts
in day
.
 Show that is a threestate Markov chain and give its onestep transition probability matrix .
 Show that the steadystate pmf is binomial with parameter
 What do you expect is the steadystate pmf for a machine that consists of parts?
Copyright 2008,
Todd Moon.
Cite/attribute Resource
.
admin. (2006, June 13). Homework Assignments. Retrieved January 07, 2011, from Free Online Course Materials — USU OpenCourseWare Web site: http://ocw.usu.edu/Electrical_and_Computer_Engineering/Stochastic_Processes/hw11.html.
This work is licensed under a
Creative Commons License