Continuous-Time Markov Chains: The Generator Matrix and the Embedded Jump Chain

Discrete-time Markov chains move at fixed ticks; the Poisson process moves in continuous time but only ever counts upward by 1. This topic's opening lesson merges the two ideas: a continuous-time Markov chain (CTMC) is a process that, like a discrete chain, can jump between arbitrary states, but does so at random continuous-time instants governed by exponential holding times — combining the Markov chain's state-dependent transition structure with the Poisson process's memoryless timing. The generator matrix $Q$, built from state-dependent jump rates, plays exactly the role the transition matrix $p$ played in discrete time, and the embedded jump chain shows every CTMC can be built from an ordinary discrete chain plus a sequence of independent exponential clocks.

Learning objectives. By the end of this note you should be able to: