logo

Matrix Projection in Linear Algebra 📂Matrix Algebra

Matrix Projection in Linear Algebra

Definition

A square matrix $P \in \mathbb{C}^{m \times m}$ is a projector if $P^2 = P$.

Explanation

In algebraic terms, this is referred to as an idempotent, which similarly refers to an element like $a^2 = a$.

If $P$ is a projection, then $(I-P)^2 = I - 2P + P^2 = I - 2P + P = (I-P)$, so it can be known that $(I-P)$ is also a projection.

Such a projector $(I - P)$ is called the complementary projector of $P$.

Thinking of projection as a linear transformation geometrically, it’s like casting light on a geometric figure and obtaining its shadow. For example, a function like $f(x,y,z) := (x,y,0)$ shoots light towards the direction of the $z$ axis and represents the shadow that forms on the $xy$ plane. Projecting the shadow again yields the same result, hence the definition of $P^2 = P$ as a projector is sensible.

Properties

Projection $P \in \mathbb{C}^{m \times m}$ and its complementary projection $I-P$ satisfy the following properties:

(a) $\mathcal{C} (I-P) = \mathcal{N} (P)$

(b) $\mathcal{N} (1-P) = \mathcal{C} (P)$

(c) $\mathcal{N} (1-P) \cap \mathcal{N} (P) = \left\{ 0 \right\}$

(d) $\mathcal{C} (P) \cap \mathcal{N} (P) = \left\{ 0 \right\}$

(e) $\mathcal{C} (P) \oplus \mathcal{N} (P) = \mathbb{C}^{m}$

Proof

(a)(b)

It suffices to show that $\mathcal{C} ( I - P)$ and $\mathcal{N}(P)$ are inclusive of each other. For any vector $\mathbf{v} \in \mathbb{C}^{m}$,

$$ (I - P) \mathbf{v} = \mathbf{v} - P \mathbf{v} $$

If $\mathbf{v} \in \mathcal{N} (P)$, then $P \mathbf{v} = \mathbb{0}$, hence $(I - P) \mathbf{v} = \mathbf{v}$, i.e., $\mathbf{v} \in \mathcal{C} (I - P)$, thus

$$ \mathcal{N} (P) \subset \mathcal{C} (I - P) $$

If we let $\mathbf{w} \in \mathcal{C} (I-P)$, then any $\mathbf{v}$ satisfying $\mathbf{w} = (I - P) \mathbf{v}$ is also in $\mathcal{C} (I-P)$. Taking the projection $P$ on $\mathbf{w} = (I - P) \mathbf{v}$,

$$ P \mathbf{w} = P \mathbf{v} - P^2 \mathbf{v} = P \mathbf{v} - P \mathbf{v} = \mathbb{0} $$

Thus $\mathbf{w} \in \mathcal{N} (P)$, and therefore

$$ \mathcal{C} (I - P) \subset \mathcal{N} (P) $$

Hence (1) is proven, and since $P = I - (I- P)$, by regarding the complementary projection $(I - P)$ of projection $P$, (2) is immediately proven.

(c)(d)

Assume that $\mathbf{v} \in \mathcal{N} (I - P) \cap \mathcal{N} (P)$ is not the zero vector. However, since $\mathbf{v} \in \mathcal{N} (I - P)$, $(I-P) \mathbf{v} = \mathbb{0}$, and $\mathbf{v} \in \mathcal{N} (P)$,

$$ P \mathbf{v} = \mathbb{0} $$

Adding both sides yields $(I-P) \mathbf{v} + P \mathbf{v} = \mathbf{v} = \mathbb{0}$, which is a contradiction to the assumption.

Thus, (3) is proven, and (4) is immediately proven by (1) and (2).

(e)

By the definition of direct sum, it suffices to show existence, exclusivity, and uniqueness.

  • (i) Existence

    By (a), $\mathcal{N} (P) = \mathcal{C} (I - P)$, and for any vector $\mathbf{s} \in \mathbb{C}^{m}$, $P \mathbf{s} \in \mathcal{C}(P)$, and $(I - P)\mathbf{s} \in \mathcal{C} (I - P)$.

However, since $P \mathbf{s} + (I - P) \mathbf{s} = P \mathbf{s} + \mathbf{s} - P \mathbf{s} = s \in \mathbb{C}^{m}$, $\mathbf{s}$ can always be represented as the sum of $\mathcal{C}(P)$ and $\mathcal{C} (I - P)$.

  • (ii) Exclusivity

    Already proven in (d).

  • (iii) Uniqueness

    From the above (ii), for $s \in \mathbb{C}^{m}$,

    $$ \mathbf{s} = \mathbf{c}_{1} + \mathbf{n}_{1} = \mathbf{c}_{2} + \mathbf{n}_{2} $$

    There exist $\mathbf{c}_{1} , \mathbf{c}_{2} \in \mathcal{C}(P)$ and $\mathbf{n}_{1}, \mathbf{n}_{2} \in \mathcal{N}(P)$ satisfying it.

Let’s assume $\mathbf{c}_{1} \ne \mathbf{c}_{2} $ and $\mathbf{n}_{1} \ne \mathbf{n}_{2}$.

Multiplying both sides of $\mathbf{c}_{1} + \mathbf{n}_{1} = \mathbf{c}_{2} + \mathbf{n}_{2}$ by $P$,

$$ P\mathbf{c}_{1} + P\mathbf{n}_{1} = P\mathbf{c}_{2} + P\mathbf{n}_{2} $$

Meanwhile, since $\mathbf{n}_{1}, \mathbf{n}_{2} \in \mathcal{N} (P)$,

$$ P\mathbf{c}_{1} = P\mathbf{c}_{2} $$

Thus $P( \mathbf{c}_{1} - \mathbf{c}_{2} ) = \mathbb{0}$. By the definition of the null space, $( \mathbf{c}_{1} - \mathbf{c}_{2}) \in \mathcal{N}(P)$, and by the property of vector spaces, $( \mathbf{c}_{1} - \mathbf{c}_{2}) \in \mathcal{C}(P)$, which by (exclusivity) must mean $\mathbf{c}_{1} - \mathbf{c}_{2} = \mathbb{0}$.

This contradicts the assumption $\mathbf{c}_{1} \ne \mathbf{c}_{2}$, and similarly, $\mathbf{n}_{1} = \mathbf{n}_{2}$ being true can be shown.

See also