##### Personal tools
•
You are here: Home Homework Assignments

# Homework Assignments

## Homework 11

These problems come from the Leon-Garcia text.

1. Let denote the sequence of sample means from an i.i.d. random process :

1. Is a Markov process?
2. If so, find the state transition p.m.f.
2. An urn initially contains five black balls and five white balls. The following experiment is repeated indefinitely. A ball is drawn from the urn; if the ball is white it is put back in the urn, otherwise it is left out. Let be the number of black balls remaining in the urn after draws from the urn.
1. Is a Markov process? If so, find the appropriate transition probabilities.
2. Do the transition probabilities depend on ?
3. Let be the Bernoulli i.i.d. process and let be

1. Show that is not a Markov process.

2. Now consider the vector process . Show that is a Markov process.
3. Find the state transition diagram for .
4. Show that the following autoregressive process is a Markov process:

with , where is an i.i.d. process.
5. Let be the Markov chain in problem 2.
1. Find the one-step transition probability matrix for .
2. Find the two-step transition probability matrix . Check your answer by computing and comparing it to the corresponding entry of
3. What happens to as ? Use your answer to guess the limit of as .

6. Two players play the following game. A fair coin is flipped; if the outcome is heads, player A pays player B $1, and if th outcome is tails, player B pays player A$1. The game is continued until one of the players goes broke. Suppose initially that player A has $1 and player B has$2, so a total of \$3 is up for grabs. Let be the number of dollars held by player A after rounds.
1. Show that is a Markov chain.
2. Draw the state transition diagram for and give the one-step transition probability matrix .
3. Use the state transition diagram to help you show that for even,

4. Find the -step transition probability matrix for even using part c.
5. Find the limit of as .
6. Find the probability that player A eventually wins.
7. A machine consists of two parts that fail and are repaired independently. A working part fails during any given day with probability . A part that is not working is repaired by the next day with probability . Let be the number of working parts in day .
1. Show that is a three-state Markov chain and give its one-step transition probability matrix .
2. Show that the steady-state pmf is binomial with parameter
3. What do you expect is the steady-state pmf for a machine that consists of parts?