📂Matrix Algebra

Eigenvalue Diagonalization of Hermitian Matrices: Proof of the Spectral Theorem

Theorem

Let $\Lambda : = \text{diag} ( \lambda_{1} , \cdots , \lambda_{m} )$ be the diagonal matrix composed of the eigenvalues $\lambda_{k}$ of an invertible matrix $A \in \mathbb{C}^{m \times m}$, and let $Q$ be the orthogonal matrix composed of the orthonormal eigenvectors $\mathbf{q}_{k}$ corresponding to those eigenvalues.

[1] Spectral Theorem

$A$ is a normal matrix if and only if $A$ is unitarily diagonalizable. $$ A A^{\ast} = A^{\ast} A \iff A = Q \Lambda Q^{\ast} $$

Corollary

If $A$ is a symmetric matrix, then $A$ is orthogonally diagonalizable.

[2] Under the Hermitian Matrix Condition

If $A$ is a Hermitian matrix, then it is unitarily diagonalizable: $$ A = A^{\ast} \implies A = Q \Lambda Q^{\ast} $$ Furthermore, the diagonal entries of $\Lambda$ are all real numbers.

---

• $A^{\ast} = \left( \overline{A} \right)^{T}$ is the transpose of the matrix obtained by taking the complex conjugate of $A$.

Explanation

The fact that an invertible matrix can be decomposed was itself confirmed in the process of eigenvalue diagonalization. The spectral theorem is quite important because it presents the conditions under which its converse holds. An area where it can be applied right away is statistics, where it forms the theoretical foundation of principal component analysis.

Meanwhile, expressing $A = Q \Lambda Q^{\ast}$ from the spectral theorem in the following series form of eigenpairs $\left\{ \left( \lambda_{k} , e_{k} \right) \right\}_{k=1}^{m}$ is called spectral decomposition. $$ A = \sum_{k=1}^{m} \lambda_{k} e_{k} e_{k}^{\ast} $$

Proof

Since a square matrix $A$ is Schur decomposable, there exist a unitary matrix $Q$ and an upper triangular matrix $T$ satisfying the following. $$ A = Q T Q^{\ast} $$ This notation is shared throughout the proof below. $O$ denotes the zero matrix.

[1] ¹

$A$ being a normal matrix is equivalent to $T$ being a normal matrix: $$ \begin{align*} & A A^{\ast} = A^{\ast} A \\ \iff & Q T Q^{\ast} \left( Q T Q^{\ast} \right)^{\ast} = \left( Q T Q^{\ast} \right)^{\ast} Q T Q^{\ast} \\ \iff & Q T T^{\ast} Q^{\ast} = Q^{\ast} T^{\ast} T Q \\ \iff & Q \left[ T T^{\ast} - T^{\ast} T \right] Q^{\ast} = O \\ \iff & T T^{\ast} = T^{\ast} T \end{align*} $$

Equivalent condition for triangular normal matrices: Let $T$ be a square matrix. A triangular matrix $T$ is a normal matrix if and only if $T$ is a diagonal matrix: $$ T T^{\ast} = T^{\ast} T \iff \left( T \right)_{ij} = 0 , \forall i \ne j $$

Meanwhile, the upper triangular matrix $T$ being a normal matrix is equivalent to $T$ being a diagonal matrix, which can be summarized as follows. $$ \begin{align*} & A A^{\ast} = A^{\ast} A \\ \iff & T T^{\ast} = T^{\ast} T \\ \iff & A = Q T Q^{\ast} \end{align*} $$ Now it only remains to show that $T = \Lambda := \diag \left( \lambda_{1} , \cdots , \lambda_{m} \right)$ is the diagonal matrix composed of the eigenvalues of $A$. Multiplying both sides of $A = Q \Lambda Q^{\ast}$ by $Q$ on the right gives $$ A Q = Q \Lambda $$ and here, since $Q:= \begin{bmatrix} \mathbf{q}_{1} & \cdots & \mathbf{q}_{m} \end{bmatrix}$ is a unitary matrix, we have $A \mathbf{q}_{k} = \lambda_{k} \mathbf{q}_{k}$ for $k = 1 , \cdots , m$. Therefore, $\lambda_{k}$ is an eigenvalue of $A$.

[2] ²

We have $A^{\ast} = Q T^{\ast} Q^{\ast}$, and since $A^{\ast} = A$, it follows that $Q T Q^{\ast} = Q T^{\ast} Q^{\ast}$, that is, $T = T^{\ast}$. An upper triangular matrix satisfying this is a diagonal matrix, and by the same method as above, we can show that $T = \Lambda$ is the diagonal matrix composed of the eigenvalues of $A$. In particular, in this case, since $A$ is a Hermitian matrix, its eigenvalues are all real.

■