📂Matrix Algebra

Matrix Algebra: Projections

Definition

The projection $P \in \mathbb{C}^{m \times m}$ is called an orthogonal projection if it satisfies $\mathcal{C} (P) ^{\perp} = \mathcal{N} (P)$ and $P$.

Explanation

According to the property of projection $\mathbb{C}^{m } = \mathcal{C} (P) \oplus \mathcal{N} (P)$, it can be seen that $P$ divides $\mathbb{C}^{m}$ into exactly two subspaces, $\mathcal{C} (P)$ and $\mathcal{N} (P)$.

The fact that this division satisfies the condition $\mathcal{N} (P) = \mathcal{C} (P) ^{\perp}$ means that the null space $\mathcal{N} (P)$ of the linear transformation $P$ is the orthogonal complement of the column space $\mathcal{C} (P)$. Thus, it is a partition that includes orthogonality, making the definition of orthogonal projection quite valid.

On the other hand, the necessary and sufficient condition for the linear transformation $P$ to be an orthogonal projection is that $P$ is a Hermitian matrix.

The proof is rather difficult and messy, so it is recommended to know it as a fact when studying.

Theorem

$$\mathcal{C} (P) ^{\perp} = \mathcal{N} (P) \iff P = P^{\ast}$$

Proof

$(\Longrightarrow)$

When the orthonormal basis of $\mathbb{C}^{m}$ is $\left\{ \mathbf{q}_{1} , \cdots , \mathbf{q}_{m} \right\}$ and we let $\dim \mathcal{C} (P) = r$, the orthonormal basis of $\mathcal{C} (P)$ can be set to $\left\{ \mathbf{q}_{1} , \cdots , \mathbf{q}_{r} \right\}$. Since $\left\{ \mathbf{q}_{1} , \cdots , \mathbf{q}_{r} \right\}$ is the basis of $\mathcal{C} (P)$, there will exist some $\mathbf {v}$ that satisfies $\mathbf{q}_{i} = P \mathbf{v}$, and if $P$ is multiplied by this equation

$$P \mathbf{q}_{i} = PP \mathbf{v} = P \mathbf{v} = \mathbf{q}_{i}$$

Meanwhile, since it is $\mathbb{C}^{m} = \mathcal{C} (P) \oplus \mathcal{N} (P)$, the orthonormal basis of $\mathcal{N} (P)$ will be $\left\{ \mathbf{q}_{r +1} , \cdots , \mathbf{q}_{m} \right\}$. When the matrix $Q : = \begin{bmatrix} \mathbf{q}_{1} & \cdots & \mathbf{q}_{r} & \mathbf{q}_{r+1} & \cdots & \mathbf{q}_{m} \end{bmatrix}$ is constructed with the vectors of $\left\{ \mathbf{q}_ {1} , \cdots , \mathbf{q}_{r} \right\}$, $Q$ becomes a unitary matrix, and when $PQ$ is calculated

$$\begin{align*} PQ =& P\begin{bmatrix} \mathbf{q}_{1} & \cdots & \mathbf{q}_{r} & \mathbf{q}_{r+1} & \cdots & \mathbf{q}_{m} \end{bmatrix} \\ =& \begin{bmatrix} P \mathbf{q}_{1} & \cdots & P \mathbf{q}_{r} & P \mathbf{q}_{r+1} & \cdots & P \mathbf{q}_{m} \end{bmatrix} \\ &= \begin{bmatrix} \mathbf{q}_{1} & \cdots & \mathbf{q}_{r} & \mathbb{0} & \cdots & \mathbb{0} \end{bmatrix} \end{align*}$$

For convenience, if we let $\widehat{Q} := \begin{bmatrix} \mathbf{q}_{1} & \cdots & \mathbf{q}_{r} \end{bmatrix}$ be $PQ = \begin{bmatrix} \widehat{Q} & O \end{bmatrix}$, the equation can be represented as $PQ = \begin{bmatrix} \widehat{Q} & O \end{bmatrix}$. When $Q^{\ast}$ is multiplied with the equation obtained above

$$\begin{align*} Q^{\ast} P Q =& \begin{bmatrix} \widehat{Q}^{\ast} \\ \mathbf{q}_{r+1} \\ \vdots \\ \mathbf{q}_{m} \end{bmatrix} \begin{bmatrix} \widehat{Q} & O \end{bmatrix} \\ =& \begin{bmatrix} \widehat{Q}^{\ast} \widehat{Q} & O \\ O & O \end{bmatrix} \\ =& \begin{bmatrix} I_{r} & O \\ O & O \end{bmatrix} \end{align*}$$

By arranging it with respect to $P$

$$P = Q \begin{bmatrix} I_{r} & O \\ O & O \end{bmatrix} Q^{\ast}$$

and

$$P^{\ast} = \left( Q \begin{bmatrix} I_{r} & O \\ O & O \end{bmatrix} Q^{\ast} \right)^{\ast} = Q \begin{bmatrix} I_{r} & O \\ O & O \end{bmatrix} Q^{\ast} = P$$

Therefore, $P$ is a Hermitian matrix.

$(\Longleftarrow)$

From $\mathbb{C}^{m } = \mathcal{C} (P) \oplus \mathcal{N} (P)$ it follows that $\mathcal{N} (P) = \mathcal{C} (I-P)$. When the inner product of the two vectors $P \mathbf{x} \in \mathcal{C} (P)$ and $(I - P) \mathbf{y} \in \mathcal{C} (I - P)$ is calculated

$$\begin{align*} ( P \mathbf{x} )^{\ast} (I - P) \mathbf{y} =& \mathbf{x}^{\ast} P^{\ast} ( I - P ) \mathbf{y} \\ =& \mathbf{x}^{\ast} P ( I - P ) \mathbf{y} \\ =& \mathbf{x}^{\ast} ( P - P^2 ) \mathbf{y} \\ =& \mathbf{x}^{\ast} ( P - P ) \mathbf{y} \\ =& \mathbb{0} \end{align*}$$

Thus

$$\mathcal{C} (P) = \mathcal{C} (I-P)^{\perp} = \mathcal{N} (P)^{\perp}$$

■