Stationary Distribution of a Markov Chain
Introduction
Suppose we are given a Markov chain $\left\{ X_{n} \right\}$ with state space $S = \left\{ 1, 2, \dots \right\}$ and its transition probability matrix $P = \begin{bmatrix} p_{ji} \end{bmatrix}$. Here $p_{ij} = P(X_{n+1} = j \mid X_{n} = i)$ is the probability of moving from state $i$ to state $j$. Hence each column of $P$ sums to $1$. Since the state of a Markov chain changes randomly at every step, which state the chain occupies at time $n$ can only be described in terms of probabilities. Let us write this as the following column vector.
$$ \pi^{(n)} := \begin{bmatrix} P(X_{n} = 1) \\ P(X_{n} = 2) \\ \vdots \end{bmatrix} $$
The probability of being in state $j$ at the next time $n+1$ is obtained by multiplying the probability of currently being in state $i$ by the probability of moving from $i$ to $j$, and summing over all possible $i$.
$$ \pi^{(n+1)}_{j} = P(X_{n+1} = j) = \sum_{i} P(X_{n+1} = j \mid X_{n} = i) P(X_{n} = i) = \sum_{i} p_{ij} \pi^{(n)}_{i} $$
In matrix form this is $\pi^{(n+1)} = P \pi^{(n)}$, and iterating it gives the following. In other words, multiplying the initial distribution $\pi^{(0)}$ by the transition probability matrix $n$ times yields the distribution after $n$ steps.
$$ \pi^{(n)} = P^{n} \pi^{(0)} $$
As a concrete example, consider modeling the probability of rain with a Markov chain. Suppose that, regardless of whether it rained yesterday, tomorrow’s probability of rain depends only on whether it rains today. Let the state space be $\left\{0, 1\right\}$, where state $0$ means it rained and state $1$ means it did not. Suppose the state transition probabilities are as follows.
$$\begin{matrix} p_{00} = 0.7 & p_{01} = 0.3 \\ p_{10} = 0.4 & p_{11} = 0.6 \end{matrix}$$
This means that if it rains today, the probability that it rains tomorrow is $70 \%$ and the probability that it does not is $30 \%$; if it does not rain today, the probability that it does not rain tomorrow is $60 \%$ and the probability that it rains is $40 \%$. If it rained today then $\pi^{(0)} = [1, 0]^{\mathsf{T}}$, and if it did not then $\pi^{(0)} = [0, 1]^{\mathsf{T}}$; but whichever we start from, as $n$ grows, $\pi^{(n)}$ approaches the same distribution $[4/7, 3/7]^{\mathsf{T}}$. This means that today’s weather has almost no influence on the weather in the distant future, which is intuitively natural. If we call the distribution reached in this way $\pi$, then applying the transition once more must still give $\pi$, so $\pi = P \pi$ holds. A distribution that is left unchanged by the transition matrix in this way is called a stationary distribution.
Definition
Let $P = [p_{ji}]$ be the transition probability matrix of a Markov chain on a countable state space $S$, where $p_{ij} = P(X_{n+1} = j \mid X_{n} = i)$. If a column vector $\pi = [\pi_{j}]_{j \in S}$ satisfying $\pi_{j} \ge 0$ and $\sum_{j \in S} \pi_{j} = 1$ satisfies the following, then $\pi$ is called a stationary distribution of this Markov chain.
$$ \pi_{j} = \sum_{i \in S} p_{ij} \pi_{i} \quad (j \in S) \qquad \text{i.e.} \qquad \pi = P \pi $$
Explanation
Simply put, a stationary distribution is a distribution that does not change no matter how far the Markov chain is run. It is “stationary” in the sense of remaining fixed, not “normal” in the sense of being regular.
Note that what remains unchanged here is not the state but the distribution of the state. Suppose there are $700$ people following the chain of the rain example, of whom $400$ are in state $0$ (rain) and $300$ are in state $1$ (clear). After one step, $400 \times 0.3 = 120$ people move from state $0$ to state $1$, and $300 \times 0.4 = 120$ people move from state $1$ to state $0$. Each individual keeps moving between states, but since the number leaving equals the number entering, the counts in the two states remain $400$ and $300$. This ratio $[4/7, 3/7]^{\mathsf{T}}$ is precisely the stationary distribution.
On the other hand, a given transition probability matrix does not always have exactly one stationary distribution, nor does a Markov chain always approach a stationary distribution. Consider the following two cases.
$P = \begin{bmatrix} 1 & 0 \\ 0 & 1 \end{bmatrix}$
The state never moves. Whatever distribution we start from stays the same, so every distribution is a stationary distribution.
$P = \begin{bmatrix} 0 & 1 \\ 1 & 0 \end{bmatrix}$
The state changes at every step without fail. The only stationary distribution is $[1/2, 1/2]^{\mathsf{T}}$, but starting from state $1$ the distribution merely alternates as $[1, 0]^{\mathsf{T}}, [0, 1]^{\mathsf{T}}, [1, 0]^{\mathsf{T}}, \dots$ and does not approach $[1/2, 1/2]^{\mathsf{T}}$.
