📂Matrix Algebra

Eigenvalue Diagonalization of Hermitian Matrices: Proof of Spectral Theory

Summary

Let’s define the invertible matrix $A \in \mathbb{C}^{m \times m}$, the diagonal matrix composed of its eigenvalues $\lambda_{k}$ as $\Lambda : = \text{diag} ( \lambda_{1} , \cdots , \lambda_{m} )$, and the orthogonal matrix composed of the corresponding orthonormal eigenvectors $\mathbf{q}_{k}$ as $Q$.

[1] Spectral Theory

The necessary and sufficient condition for $A$ to be a normal matrix is that $A$ is unitarily diagonalizable. $$ A A^{\ast} = A^{\ast} A \iff A = Q \Lambda Q^{\ast} $$

[2] Under the Condition of Hermitian Matrix

If $A$ is a Hermitian matrix, then it is unitarily diagonalizable: $$ A = A^{\ast} \implies A = Q \Lambda Q^{\ast} $$ Furthermore, the diagonal elements of $\Lambda$ are all real numbers.

---

• $A^{\ast} = \left( \overline{A} \right)^{T}$ is the matrix obtained by taking the complex conjugate transpose of $A$, also known as the Hermitian matrix.

Explanation

The fact that the invertible matrix can be decomposed was confirmed in the process of eigenvalue diagonalization. Spectral theory provides the conditions for its reverse process, making it quite significant. One immediate field where this can be applied is statistics, forming the theoretical foundation for principal component analysis.

Meanwhile, expressing $A = Q \Lambda Q^{\ast}$ in spectral theory as a series of eigenpairs $\left\{ \left( \lambda_{k} , e_{k} \right) \right\}_{k=1}^{m}$ is called spectral decomposition. $$ A = \sum_{k=1}^{m} \lambda_{k} e_{k} e_{k}^{\ast} $$

Proof

Since the square matrix $A$ is Schur decomposable, there exist a unitary matrix $Q$ and an upper triangular matrix $T$ that satisfy the following: $$ A = Q T Q^{\ast} $$ This notation will be shared in the proof below. $O$ represents the zero matrix.

[1] ¹

The fact that $A$ is a normal matrix is equivalent to saying that $T$ is a normal matrix: $$ \begin{align*} & A A^{\ast} = A^{\ast} A \\ \iff & Q T Q^{\ast} \left( Q T Q^{\ast} \right)^{\ast} = \left( Q T Q^{\ast} \right)^{\ast} Q T Q^{\ast} \\ \iff & Q T T^{\ast} Q^{\ast} = Q^{\ast} T^{\ast} T Q \\ \iff & Q \left[ T T^{\ast} - T^{\ast} T \right] Q^{\ast} = O \\ \iff & T T^{\ast} = T^{\ast} T \end{align*} $$

Equivalence Condition for Triangular Normal Matrix: Let $T$ be a square matrix. The necessary and sufficient condition for the triangular matrix $T$ to be a normal matrix is that $T$ is a diagonal matrix: $$ T T^{\ast} = T^{\ast} T \iff \left( T \right)_{ij} = 0 , \forall i \ne j $$

Meanwhile, the fact that the upper triangular matrix $T$ is a normal matrix is equivalent to saying that $T$ is a diagonal matrix, which can be summarized as follows. $$ \begin{align*} & A A^{\ast} = A^{\ast} A \\ \iff & T T^{\ast} = T^{\ast} T \\ \iff & A = Q T Q^{\ast} \end{align*} $$ It remains to show that $T = \Lambda := \diag \left( \lambda_{1} , \cdots , \lambda_{m} \right)$ is a diagonal matrix composed of the eigenvalues of $A$. Multiplying both sides of $A = Q \Lambda Q^{\ast}$ on the right by $Q$ gives $$ A Q = Q \Lambda $$ where $Q:= \begin{bmatrix} \mathbf{q}_{1} & \cdots & \mathbf{q}_{m} \end{bmatrix}$ is a unitary matrix, so for $k = 1 , \cdots , m$, we have $A \mathbf{q}_{k} = \lambda_{k} \mathbf{q}_{k}$. Therefore, $\lambda_{k}$ is an eigenvalue of $A$.

[2] ²

It is $A^{\ast} = Q T^{\ast} Q^{\ast}$, and since $A^{\ast} = A$, it follows that $Q T Q^{\ast} = Q T^{\ast} Q^{\ast}$, i.e., $T = T^{\ast}$. The upper triangular matrix that satisfies this is a diagonal matrix, and using the same method as above, it can be shown that $T = \Lambda$ is a diagonal matrix consisting of the eigenvalues of $A$. In particular, in this case, $A$ is a Hermitian matrix whose eigenvalues are all real.

■