📂Probability Theory

Transition Probabilities of Stochastic Processes

Definition

Let us assume there is a stochastic process $\left\{ X_{t} \right\}$ with a countable set as its state space.

1. For two points in time $t_{1} < t_{2}$, the transition probability $p_{ij} \left( t_{1} , t_{2} \right)$ is defined as follows: $$p_{ij} \left( t_{1} , t_{2} \right) := P \left( X_{t_{2}} = j \mid X_{t_{1}} = i \right)$$
   Here, the (current) state represented by $i$ is referred to as the source state, and the target state represented by $j$ is referred to as the target state. In particular, for a discrete stochastic process $\left\{ X_{t} \right\}_{t \in \mathbb{N}}$ where $t_{1} = n \in \mathbb{N}$ and $t_{2} = n + k \in \mathbb{N}$, the transition probability can be simply represented as: $$\begin{align*} p_{ij}^{(k)} :=& P \left(n + k = j \mid X_{n} = i \right) \\ p_{ij} :=& p_{ij}^{(1)} \end{align*}$$
2. If the transition probabilities do not depend on the time points but solely on the interval $\Delta t = t_{2} - t_{1}$, in other words, if the following condition is satisfied, then it is called a stationary or homogeneous transition probability: $$p_{ij} (\Delta t) := \left( X_{t_{2} - t_{1}} = j \mid X_{0} = i \right)$$
3. For stationary transition probabilities, the matrix function defined as $P(t)$ and $P^{(k)}$ is referred to as the transition probability matrix: $$\begin{align*} \left( P(t) \right)_{ij} :=& \left( p_{ij} (t) \right) \\ \left( P^{(k)} \right)_{ij} :=& \left( p_{ij}^{(k)} \right) \end{align*}$$
4. Let the transition probability matrix $P(t)$ of a continuous stochastic process be a differentiable matrix function. The following matrix $$Q := P’ (0)$$
   is called the infinitesimal generator matrix, and its elements $\left( Q \right)_{ij}$ are called transition rates.

See also

• Chapman-Kolmogorov equation