##### Personal tools
•
You are here: Home Markov Processes

# Markov Processes

##### Document Actions

Concepts  ::  Discrete  ::  Continuous  ::  States

## Classes of States

Let . Then state is recurrent if .

If , then state is said to be transient .

• If started in a transient state, then the state does not recur an infinite number of times.
• If in a recurrent state, then the state recurs an infinite number of times.
Let denote the Markov chain with . Let

Then

We see that recurrent means that .

Transient means that .

Observation: The states of an irreducible, finite-state Markov chain are all recurrent.

## Limiting probabilities

If all states are transient, then all the state probabilities approach 0 as . If a M.C. has some transient classes and some recurrent classes, then eventually the process enters and remains in one of the recurrent classes. For limiting purposes, we can focus on individual recurrent classes.

Suppose a M.C. starts in a recurrent state at time 0. Let denote the times when the process returns to state , where is the time that elapses between the
Copyright 2008, by the Contributing Authors. Cite/attribute Resource . admin. (2006, June 08). Markov Processes. Retrieved January 07, 2011, from Free Online Course Materials — USU OpenCourseWare Web site: http://ocw.usu.edu/Electrical_and_Computer_Engineering/Stochastic_Processes/lecture10_4.htm. This work is licensed under a Creative Commons License