Skip to content
Probability II / Discrete-Time Markov Chains

Recurrence, Transience, and Stationary Distributions

The previous lesson classified states qualitatively — transient or recurrent — using the graph of the chain. This note answers the quantitative question that motivated Examples 3–4 of the first lesson: do the long-run fractions of time spent in each state settle down, and to what? The answer is the stationary distribution π\pi, the unique probability vector left unchanged by one application of the transition matrix — and this note shows how to compute it for chains both small (by hand) and structured (doubly stochastic).
Learning objectives. By the end of this note you should be able to:
  • give the precise definition of a recurrent state in terms of the return probability, and state the ∑npn(y,y)=∞\sum_n p^n(y,y)=\infty criterion;
  • define a stationary distribution and set up the linear system that computes it;
  • recognise and exploit the doubly-stochastic special case;
  • state the mean-return-time formula π(y)=1/Ey[Ty]\pi(y)=1/\mathbb{E}_y[T_y] and use it to interpret a stationary probability.

In this lesson

  1. Recurrence, precisely
  2. Stationary distributions
  3. Mean return time

The full lesson, with worked examples and exercise solutions, is for members.

Read the full lesson

New to SelfMadeQuant.com? See membership options or explore the curriculum.