Classification of States: Communicating Classes, Irreducibility, and Periodicity

Not every state in a Markov chain plays the same role. In gambler's ruin, states $1,\ldots,N-1$ are eventually abandoned forever while $0$ and $N$ are visited forever once reached — a distinction the previous lessons have used informally but not yet defined. This note makes it precise with two tools: the communicates-with relation, which partitions the state space into classes the chain cannot escape or cannot avoid, and periodicity, which detects a subtler obstruction — a chain that returns to a state only at multiples of some fixed number of steps. Both classifications are prerequisites for the convergence theorem two lessons ahead: a Markov chain converges to a stationary distribution only if it is irreducible (one class) and aperiodic (period 1).

Learning objectives. By the end of this note you should be able to: