Expectation of Random Vectors
📂Mathematical Statistics Expectation of Random Vectors Definition E ( X ) : = [ E ( X 1 ) ⋮ E ( X n ) ]
E \left( X \right) := \begin{bmatrix} E \left( X_{1} \right) \\ \vdots \\ E \left( X_{n} \right) \end{bmatrix}
E ( X ) := E ( X 1 ) ⋮ E ( X n ) expectation of a random vector X = ( X 1 , ⋯ , X n ) X = \left( X_{1} , \cdots , X_{n} \right) X = ( X 1 , ⋯ , X n ) vector of the expectations of its components, as shown above. Similarly, the matrix X = [ X i j ] \mathbf{X} = \left[ X_{ij} \right] X = [ X ij ] random variable of size m × n m \times n m × n E ( X ) : = [ E ( X i j ) ] E \left( \mathbf{X} \right) := \left[ E \left( X_{ij} \right) \right] E ( X ) := [ E ( X ij ) ]
Properties [1] Linearity : If X 1 \mathbf{X}_{1} X 1 X 2 \mathbf{X}_{2} X 2 m × n m \times n m × n constant matrices A 1 , A 2 ∈ R k × m A_{1}, A_{2} \in \mathbb{R}^{k \times m} A 1 , A 2 ∈ R k × m B ∈ R n × l B \in \mathbb{R}^{n \times l} B ∈ R n × l E ( A 1 X 1 + A 2 X 2 ) = A 1 E ( X 1 ) + A 2 E ( X 2 ) E ( A 1 X 1 B ) = A 1 E ( X 1 ) B
\begin{align*}
E \left( A_{1} \mathbf{X}_{1} + A_{2} \mathbf{X}_{2} \right) =& A_{1} E \left( \mathbf{X}_{1} \right) + A_{2} E \left( \mathbf{X}_{2} \right)
\\ E \left( A_{1} \mathbf{X}_{1} B \right) =& A_{1} E \left( X_{1} \right) B
\end{align*}
E ( A 1 X 1 + A 2 X 2 ) = E ( A 1 X 1 B ) = A 1 E ( X 1 ) + A 2 E ( X 2 ) A 1 E ( X 1 ) B [2] Trace : E ( tr ( X ) ) = tr ( E ( X ) ) E(\tr(\mathbf{X})) = \tr(E(\mathbf{X})) E ( tr ( X )) = tr ( E ( X )) Proof [1] We will only show E ( A X ) = A E ( X ) E \left( A \mathbf{X} \right) = A E \left( \mathbf{X} \right) E ( A X ) = A E ( X )
Let’s denote A = [ a i k ] A = \begin{bmatrix} a_{ik}\end{bmatrix} A = [ a ik ] m × p m \times p m × p X = [ X k j ] \mathbf{X} = \begin{bmatrix} X_{kj}\end{bmatrix} X = [ X kj ] p × n p \times n p × n
E ( A X ) = E ( [ ∑ k = 1 p a i k X k j ] ) = [ E ( ∑ k = 1 p a i k X k j ) ] = [ ∑ k = 1 p a i k E ( X k j ) ] by linearity of E = A E ( X )
\begin{align*}
E(A \mathbf{X})
&= E \left( \begin{bmatrix} \sum\limits_{k=1}^{p} a_{ik}X_{kj} \end{bmatrix} \right) \\
&= \begin{bmatrix} E \left( \sum\limits_{k=1}^{p} a_{ik}X_{kj} \right) \end{bmatrix} \\
&= \begin{bmatrix} \sum\limits_{k=1}^{p} a_{ik} E \left( X_{kj} \right)
\end{bmatrix} & \text{by linearity of E E E E ( A X ) = E ( [ k = 1 ∑ p a ik X kj ] ) = [ E ( k = 1 ∑ p a ik X kj ) ] = [ k = 1 ∑ p a ik E ( X kj ) ] = A E ( X ) by linearity of E
■
[2] X = [ X i j ] \mathbf{X} = \begin{bmatrix} X_{ij} \end{bmatrix} X = [ X ij ] n × n n \times n n × n 행렬 이라고 하자.
E ( tr ( A ) ) = E ( ∑ i = 1 n X i i ) = ∑ i = 1 n E ( X i i ) by linearity of E = tr [ E ( X 11 ) ⋯ E ( X 1 n ) ⋮ ⋱ ⋮ E ( X n 1 ) ⋯ E ( X n n ) ] by definition of trace = tr ( E ( X ) )
\begin{align*}
E(\tr(A))
&= E \left( \sum\limits_{i=1}^{n} X_{ii} \right) \\
&= \sum\limits_{i=1}^{n} E(X_{ii}) & \text{by linearity of E E E E ( tr ( A )) = E ( i = 1 ∑ n X ii ) = i = 1 ∑ n E ( X ii ) = tr E ( X 11 ) ⋮ E ( X n 1 ) ⋯ ⋱ ⋯ E ( X 1 n ) ⋮ E ( X nn ) = tr ( E ( X ) ) by linearity of E by definition of trace
■
