📂Probability Theory

Limit of Transition Probabilities

Definition

When the current state is $i$, after going through $k$ steps to reach $j$, the transition probability is denoted as $p_{ij}^{(k)}$. The transition probability after an infinite number of steps is represented as follows: $$\pi_{j}:= \lim_{n \to \infty} p_{ij}^{ ( n ) }$$

Description

Whether it’s statistics or applied mathematics, the main interest usually lies in predicting the future. What’s intriguing in the theory of stochastic processes is not just what happens next, but what happens in the distant future. And this representation often uses infinity.

As expressed by $\displaystyle \pi_{j} = \sum_{i} \pi_{i} p_{ij}$ in the definition, when represented as a matrix, for $\pi:= \begin{bmatrix} \pi_{1} & \cdots & \pi_{k} \end{bmatrix}$ and $P:= \begin{bmatrix} p_{1j} & \cdots & p_{1j} \\ \vdots & \ddots & \vdots \\ p_{nj} & \cdots & p_{nj} \end{bmatrix}$, it can be presented as $\pi = \pi P$. Taking the transpose on both sides yields $\pi^{T} = P^{T} \pi^{T}$.

To summarize, it becomes a simple problem of finding $\mathbf{x} : = \pi^{T} = \begin{bmatrix} \pi_{1} \\ \vdots \\ \pi_{n} \end{bmatrix}$ that satisfies $\left( P^{T} - I \right) \mathbf{x} = 0$. If a limit exists, the problem is usually solved in this manner.

Example

Let’s calculate the limit for the following stochastic process.

The transition matrix can be represented as $P:= \begin{bmatrix} 3/4 & 1/4 & 0 \\ 1/2 & 0 & 1/2 \\ 0 & 0 & 1 \end{bmatrix}$, and we need to solve $\left( P^{T} - I \right) \begin{bmatrix} \pi_{A} \\ \pi_{B} \\ \pi_{C} \end{bmatrix} = \mathbb{0}$.

Since $\left( P^{T} - I \right) = \begin{bmatrix} -1/4 & 1/2 & 0 \\ 1/4 & -1 & 0 \\ 0 & 1/2 & 0 \end{bmatrix}$, we obtain the system of equations $\begin{cases} - 1/4 \pi_{A} + 1/2 \pi_{B} = 0 \\ 1/4 \pi_{A} - \pi_{B} = 0 \\ \pi_{B} = 0 \end{cases}$.

Solving the system of equations gives us $\pi_{A} = \pi_{B} = 0$, and since $\pi_{A} + \pi_{B} + \pi_{C} = 1$, we also get up to $\pi_{C} = 1$.

This result is not significantly different from intuition, mathematically showing that if the process is repeated infinitely, it will eventually end at C.