📂Mathematical Statistics

Conditions for Equivalence of Chi-Squared Nature in Quadratic Forms of Normal Distribution Random Vectors

Theorem

Let sample $\mathbf{X} = \left( X_{1} , \cdots , X_{n} \right)$ follow a normal distribution as iid as $X_{1} , \cdots , X_{n} \overset{\text{iid}}{\sim} N \left( 0, \sigma^{2} \right)$. For a symmetric matrix $A \in \mathbb{R}^{n \times n}$ with rank $r \le n$, define the quadratic form of a random vector as $Q = \sigma^{-2} \mathbf{X}^{T} A \mathbf{X}$, then the following holds. $$Q \sim \chi^{2} (r) \iff A^{2} = A$$ In other words, the equivalent condition for $Q$ to follow a chi-squared distribution $\chi^{2} (r)$ is that $A$ is an idempotent matrix.

Explanation

This theorem is used in proof of Hogg-Craig theorem and proof of Cochran’s theorem.

Proof

Moment generating function of quadratic form of normal distribution random vector: Let sample $\mathbf{X} = \left( X_{1} , \cdots , X_{n} \right)$ follow a normal distribution as iid as $X_{1} , \cdots , X_{n} \overset{\text{iid}}{\sim} N \left( 0, \sigma^{2} \right)$. For a symmetric matrix $A \in \mathbb{R}^{n \times n}$ with rank $r \le n$, the moment generating function of the quadratic form of a random vector $Q = \sigma^{-2} \mathbf{X}^{T} A \mathbf{X}$ is as follows. $$M_{Q} (t) = \prod_{i=1}^{r} \left( 1 - 2 t \lambda_{i} \right)^{-1/2} = \det \left( I_{n} - 2 t A \right)^{-1/2} \qquad , | t | < 1 / 2 \lambda_{1}$$ Here $I_{n} \in \mathbb{R}^{n \times n}$ is the identity matrix, and $\det$ is the determinant. $\lambda_{1} \ge \cdots \ge \lambda_{r}$ is the eigenvalues of $A$ that are not $0$, listed in descending order without loss of generality.

$$M_{Q} (t) = \prod_{i=1}^{r} \left( 1 - 2 t \lambda_{i} \right)^{-1/2}$$ The moment generating function of $Q$ $M_{Q} (t)$ is as above.

$(\implies)$

Moment generating function of chi-squared distribution: The moment generating function of a chi-squared distribution with degrees of freedom $r$ is as follows. $$m(t) = (1-2t)^{-r/2} \qquad , t < {{ 1 } \over { 2 }}$$

Assuming $Q$ follows $\chi^{2} (r)$, the moment generating function of $Q$ near $0$ has two forms at $t$. $$M_{Q} (t) = \prod_{i=1}^{r} \left( 1 - 2 t \lambda_{i} \right)^{-1/2} = (1-2t)^{-r/2}$$ Taking the power of $-1/2$ on both sides, we obtain the following. $$\prod_{i=1}^{r} \left( 1 - 2 t \lambda_{i} \right) = (1-2t)^{r}$$

Symmetric real matrix with eigenvalues only $0$ and $1$: If all eigenvalues of the symmetric matrix $A \in \mathbb{R}^{n \times n}$ are $0$ or $1$, then $A$ is an idempotent matrix.

Since the factorization of polynomials with complex coefficients is unique, it is $\lambda_{1} = \cdots = \lambda_{r} = 1$. All the other eigenvalues are $0$, making the symmetric matrix $A \in \mathbb{R}^{n \times n}$ an idempotent matrix.

$(\impliedby)$

Eigenvalues of idempotent matrix: The eigenvalues of an idempotent matrix are only $0$ or $1$.

Assume $A$ is an idempotent matrix. The eigenvalues of an idempotent matrix are either $0$ or $1$, and given $\lambda_{1}, \cdots , \lambda_{r}$ are eigenvalues not equal to $0$, they must be all $1$. Since the moment generating function of $Q$ is as follows, $Q$ follows a chi-squared distribution with degrees of freedom $r$. $$\begin{align*} M_{Q} (t) =& \prod_{i=1}^{r} \left( 1 - 2 t \lambda_{i} \right)^{-1/2} \\ =& \prod_{i=1}^{r} \left( 1 - 2 t \right)^{-1/2} \\ =& \left( 1 - 2 t \right)^{-r/2} \end{align*}$$

■