Matrix Algebra: Projections
Definition
If a projection $P \in \mathbb{C}^{m \times m}$ satisfies $\mathcal{C} (P) ^{\perp} = \mathcal{N} (P)$, then $P$ is called an orthogonal projection.
Explanation
According to the property of projection $\mathbb{C}^{m } = \mathcal{C} (P) \oplus \mathcal{N} (P)$, $P$ exactly divides $\mathbb{C}^{m}$ into two subspaces $\mathcal{C} (P)$ and $\mathcal{N} (P)$.
This division satisfies condition $\mathcal{N} (P) = \mathcal{C} (P) ^{\perp}$, meaning the null space $\mathcal{N} (P)$ of the linear transformation $P$ is the orthogonal complement of the column space $\mathcal{C} (P)$, indicating the division is not just any division but one that involves orthogonality. In that sense, the definition of an orthogonal projection is quite reasonable.
Furthermore, a necessary and sufficient condition for the linear transformation $P$ to be an orthogonal projection is that $P$ is Hermitian.
The proof of this is more difficult and messy than one might think, so it is recommended to simply know this fact for study purposes.
Theorem
$$ \mathcal{C} (P) ^{\perp} = \mathcal{N} (P) \iff P = P^{\ast} $$
Proof
$(\Longrightarrow)$
When the orthonormal basis of $\mathbb{C}^{m}$ is $\left\{ \mathbf{q}_{1} , \cdots , \mathbf{q}_{m} \right\}$, let’s say $\dim \mathcal{C} (P) = r$, then the orthonormal basis of $\mathcal{C} (P)$ can be set as $\left\{ \mathbf{q}_{1} , \cdots , \mathbf{q}_{r} \right\}$. Since $\left\{ \mathbf{q}_{1} , \cdots , \mathbf{q}_{r} \right\}$ is a basis of $\mathcal{C} (P)$, there exists some $\mathbf {v}$ that satisfies $\mathbf{q}_{i} = P \mathbf{v}$, and by multiplying $P$ to this equation,
$$ P \mathbf{q}_{i} = PP \mathbf{v} = P \mathbf{v} = \mathbf{q}_{i} $$
Meanwhile, since $\mathbb{C}^{m} = \mathcal{C} (P) \oplus \mathcal{N} (P)$, the orthonormal basis of $\mathcal{N} (P)$ will be $\left\{ \mathbf{q}_{r +1} , \cdots , \mathbf{q}_{m} \right\}$. By constructing the matrix $Q : = \begin{bmatrix} \mathbf{q}_{1} & \cdots & \mathbf{q}_{r} & \mathbf{q}_{r+1} & \cdots & \mathbf{q}_{m} \end{bmatrix}$ with vectors of $\left\{ \mathbf{q}_ {1} , \cdots , \mathbf{q}_{r} \right\}$, $Q$ becomes a unitary matrix, and by calculating $PQ$,
$$ \begin{align*} PQ =& P\begin{bmatrix} \mathbf{q}_{1} & \cdots & \mathbf{q}_{r} & \mathbf{q}_{r+1} & \cdots & \mathbf{q}_{m} \end{bmatrix} \\ =& \begin{bmatrix} P \mathbf{q}_{1} & \cdots & P \mathbf{q}_{r} & P \mathbf{q}_{r+1} & \cdots & P \mathbf{q}_{m} \end{bmatrix} \\ &= \begin{bmatrix} \mathbf{q}_{1} & \cdots & \mathbf{q}_{r} & \mathbb{0} & \cdots & \mathbb{0} \end{bmatrix} \end{align*} $$
For convenience, let’s say $\widehat{Q} := \begin{bmatrix} \mathbf{q}_{1} & \cdots & \mathbf{q}_{r} \end{bmatrix}$, then it can be expressed as $PQ = \begin{bmatrix} \widehat{Q} & O \end{bmatrix}$. By multiplying $Q^{\ast}$ to the equation obtained above,
$$ \begin{align*} Q^{\ast} P Q =& \begin{bmatrix} \widehat{Q}^{\ast} \\ \mathbf{q}_{r+1} \\ \vdots \\ \mathbf{q}_{m} \end{bmatrix} \begin{bmatrix} \widehat{Q} & O \end{bmatrix} \\ =& \begin{bmatrix} \widehat{Q}^{\ast} \widehat{Q} & O \\ O & O \end{bmatrix} \\ =& \begin{bmatrix} I_{r} & O \\ O & O \end{bmatrix} \end{align*} $$
Calculating the inner product of $P$ 에 대해서 정리하면
$$ P = Q \begin{bmatrix} I_{r} & O \\ O & O \end{bmatrix} Q^{\ast} $$
이고
$$ P^{\ast} = \left( Q \begin{bmatrix} I_{r} & O \\ O & O \end{bmatrix} Q^{\ast} \right)^{\ast} = Q \begin{bmatrix} I_{r} & O \\ O & O \end{bmatrix} Q^{\ast} = P $$
이므로 $P$ 는 에르미트 행렬이다.
$(\Longleftarrow)$
$\mathbb{C}^{m } = \mathcal{C} (P) \oplus \mathcal{N} (P)$ 에서 $\mathcal{N} (P) = \mathcal{C} (I-P)$ 이다. 두 벡터 $P \mathbb{x} \in \mathcal{C} (P)$ 와 $(I - P) \mathbb{y} \in \mathcal{C} (I - P)$,
$$ \begin{align*} ( P \mathbb{x} )^{\ast} (I - P) \mathbb{y} =& \mathbb{x}^{\ast} P^{\ast} ( I - P ) \mathbb{y} \\ =& \mathbb{x}^{\ast} P ( I - P ) \mathbb{y} \\ =& \mathbb{x}^{\ast} ( P - P^2 ) \mathbb{y} \\ =& \mathbb{x}^{\ast} ( P - P ) \mathbb{y} \\ =& \mathbb{0} \end{align*} $$
Therefore,
$$ \mathcal{C} (P) = \mathcal{C} (I-P)^{\perp} = \mathcal{N} (P)^{\perp} $$
■