Transition Probabilities of Stochastic Processes
Definition
Let us assume there is a stochastic process $\left\{ X_{t} \right\}$ with a countable set as its state space.
- For two points in time $t_{1} < t_{2}$, the transition probability $p_{ij} \left( t_{1} , t_{2} \right)$ is defined as follows:
$$
p_{ij} \left( t_{1} , t_{2} \right) := P \left( X_{t_{2}} = j \mid X_{t_{1}} = i \right)
$$
Here, the (current) state represented by $i$ is referred to as the source state, and the target state represented by $j$ is referred to as the target state. In particular, for a discrete stochastic process $\left\{ X_{t} \right\}_{t \in \mathbb{N}}$ where $t_{1} = n \in \mathbb{N}$ and $t_{2} = n + k \in \mathbb{N}$, the transition probability can be simply represented as: $$ \begin{align*} p_{ij}^{(k)} :=& P \left(n + k = j \mid X_{n} = i \right) \\ p_{ij} :=& p_{ij}^{(1)} \end{align*} $$ - If the transition probabilities do not depend on the time points but solely on the interval $\Delta t = t_{2} - t_{1}$, in other words, if the following condition is satisfied, then it is called a stationary or homogeneous transition probability: $$ p_{ij} (\Delta t) := \left( X_{t_{2} - t_{1}} = j \mid X_{0} = i \right) $$
- For stationary transition probabilities, the matrix function defined as $P(t)$ and $P^{(k)}$ is referred to as the transition probability matrix: $$ \begin{align*} \left( P(t) \right)_{ij} :=& \left( p_{ij} (t) \right) \\ \left( P^{(k)} \right)_{ij} :=& \left( p_{ij}^{(k)} \right) \end{align*} $$
- Let the transition probability matrix $P(t)$ of a continuous stochastic process be a differentiable matrix function. The following matrix
$$
Q := P’ (0)
$$
is called the infinitesimal generator matrix, and its elements $\left( Q \right)_{ij}$ are called transition rates.