📂Mathematical Statistics

Expected Value of the Quadratic Form of a Random Vector

Formula

Let the population mean vector $\mu \in \mathbb{R}^{n}$ and the covariance matrix $\Sigma \in \mathbb{R}^{n \times n}$ be given such that the random vector $\mathbf{X}$ is $\mathbf{X} \sim \left( \mu , \Sigma \right)$. For a symmetric matrix $A$, the expected value of the quadratic form of a random vector is as follows. $$E (Q) = \operatorname{tr} A \Sigma + \mu^{T} A \mu$$ Here, $\mu^{T}$ is the transpose matrix of $\mu$, and $\operatorname{tr}$ is the trace.

Derivation ¹

Cyclic property of the trace: $$\operatorname{tr} (ABC) = \operatorname{tr} (BCA) = \operatorname{tr} (CAB)$$

Expectation and trace of a random vector: $E(\tr(\mathbf{X})) = \tr(E(\mathbf{X}))$

Covariance matrix: Given as $\mathbf{\mu} \in \mathbb{R}^{p}$ is $\mathbf{\mu} := \left( EX_{1} , \cdots , EX_{p} \right)$ $$\operatorname{Cov} (\mathbf{X}) = E \left[ \mathbf{X} \mathbf{X}^{T} \right] - \mathbf{\mu} \mathbf{\mu}^{T}$$

$$\begin{align*} & E (Q) \\ =& E \left( \operatorname{tr} \mathbf{X}^{T} A \mathbf{X} \right) \\ =& E \left( \operatorname{tr} A \mathbf{X} \mathbf{X}^{T} \right) \\ =& \operatorname{tr} A E \left( \mathbf{X} \mathbf{X}^{T} \right) \\ =& \operatorname{tr} A \left( \Sigma + \mu \mu^{T} \right) \\ =& \operatorname{tr} A \Sigma + \operatorname{tr} A \mu \mu^{T} \\ =& \operatorname{tr} A \Sigma + \operatorname{tr} \mu^{T} A \mu \\ =& \operatorname{tr} A \Sigma + \mu^{T} A \mu \end{align*}$$

■