A vector that satisfies the following condition p=[p1⋯pn]T is called a probability vector.
0≤pi≤1(1≤i≤n)andi=1∑npi=1
Explanation
A probability vector is a vector that represents the probabilities of each state when there are n states. Conceptually, it is analogous to a probability mass function. If the probability mass function of a discrete random variableX is pX, then the probability vector is as follows.
p=p1⋮pn=pX(1)⋮pX(n)=p(1)⋮p(n)
If the probability that the j-th state changes to the i-th state is qij=q(i∣j), then the probability vector q(j)=[q1j⋯qnj]T represents the probability that the j-th state changes to other states. The matrix of these column vectors becomes a transition matrix.
If p is the probability vector for the current state, then p′=Qp is the probability vector for the next state, given that the probabilities for the current state are provided as p.
If we let the probability row vector for the current state be π and the transition matrix be P=−p(1)−⋮−p(n)−, the following notation is frequently used.
πP=π′
In this case, Pij=P(j∣i), where the element in the i-th row and the j-th column represents the probability of changing from the i-th state to the j-th state. The reason for using a row vector notation is not entirely clear. A notation utilizing the letter t for transition, such as T, is also commonly used for the transition matrix.