📂Matrix Algebra

Matrix Projection in Linear Algebra

Definition

A square matrix $P \in \mathbb{C}^{m \times m}$ is a projector if $P^2 = P$.

Explanation

In algebraic terms, this is referred to as an idempotent, which similarly refers to an element like $a^2 = a$.

If $P$ is a projection, then $(I-P)^2 = I - 2P + P^2 = I - 2P + P = (I-P)$, so it can be known that $(I-P)$ is also a projection.

Such a projector $(I - P)$ is called the complementary projector of $P$.

Thinking of projection as a linear transformation geometrically, it’s like casting light on a geometric figure and obtaining its shadow. For example, a function like $f(x,y,z) := (x,y,0)$ shoots light towards the direction of the $z$ axis and represents the shadow that forms on the $xy$ plane. Projecting the shadow again yields the same result, hence the definition of $P^2 = P$ as a projector is sensible.

Properties

Projection $P \in \mathbb{C}^{m \times m}$ and its complementary projection $I-P$ satisfy the following properties:

(a) $\mathcal{C} (I-P) = \mathcal{N} (P)$

(b) $\mathcal{N} (1-P) = \mathcal{C} (P)$

(c) $\mathcal{N} (1-P) \cap \mathcal{N} (P) = \left\{ 0 \right\}$

(d) $\mathcal{C} (P) \cap \mathcal{N} (P) = \left\{ 0 \right\}$

(e) $\mathcal{C} (P) \oplus \mathcal{N} (P) = \mathbb{C}^{m}$

Proof

(a)(b)

It suffices to show that $\mathcal{C} ( I - P)$ and $\mathcal{N}(P)$ are inclusive of each other. For any vector $\mathbf{v} \in \mathbb{C}^{m}$,

$$(I - P) \mathbf{v} = \mathbf{v} - P \mathbf{v}$$

If $\mathbf{v} \in \mathcal{N} (P)$, then $P \mathbf{v} = \mathbb{0}$, hence $(I - P) \mathbf{v} = \mathbf{v}$, i.e., $\mathbf{v} \in \mathcal{C} (I - P)$, thus

$$\mathcal{N} (P) \subset \mathcal{C} (I - P)$$

If we let $\mathbf{w} \in \mathcal{C} (I-P)$, then any $\mathbf{v}$ satisfying $\mathbf{w} = (I - P) \mathbf{v}$ is also in $\mathcal{C} (I-P)$. Taking the projection $P$ on $\mathbf{w} = (I - P) \mathbf{v}$,

$$P \mathbf{w} = P \mathbf{v} - P^2 \mathbf{v} = P \mathbf{v} - P \mathbf{v} = \mathbb{0}$$

Thus $\mathbf{w} \in \mathcal{N} (P)$, and therefore

$$\mathcal{C} (I - P) \subset \mathcal{N} (P)$$

Hence (1) is proven, and since $P = I - (I- P)$, by regarding the complementary projection $(I - P)$ of projection $P$, (2) is immediately proven.

■

(c)(d)

Assume that $\mathbf{v} \in \mathcal{N} (I - P) \cap \mathcal{N} (P)$ is not the zero vector. However, since $\mathbf{v} \in \mathcal{N} (I - P)$, $(I-P) \mathbf{v} = \mathbb{0}$, and $\mathbf{v} \in \mathcal{N} (P)$,

$$P \mathbf{v} = \mathbb{0}$$

Adding both sides yields $(I-P) \mathbf{v} + P \mathbf{v} = \mathbf{v} = \mathbb{0}$, which is a contradiction to the assumption.

Thus, (3) is proven, and (4) is immediately proven by (1) and (2).

■

(e)

By the definition of direct sum, it suffices to show existence, exclusivity, and uniqueness.

• (i) Existence

  By (a), $\mathcal{N} (P) = \mathcal{C} (I - P)$, and for any vector $\mathbf{s} \in \mathbb{C}^{m}$, $P \mathbf{s} \in \mathcal{C}(P)$, and $(I - P)\mathbf{s} \in \mathcal{C} (I - P)$.

However, since $P \mathbf{s} + (I - P) \mathbf{s} = P \mathbf{s} + \mathbf{s} - P \mathbf{s} = s \in \mathbb{C}^{m}$, $\mathbf{s}$ can always be represented as the sum of $\mathcal{C}(P)$ and $\mathcal{C} (I - P)$.

• (ii) Exclusivity

  Already proven in (d).

• (iii) Uniqueness

  From the above (ii), for $s \in \mathbb{C}^{m}$,

  $$\mathbf{s} = \mathbf{c}_{1} + \mathbf{n}_{1} = \mathbf{c}_{2} + \mathbf{n}_{2}$$

  There exist $\mathbf{c}_{1} , \mathbf{c}_{2} \in \mathcal{C}(P)$ and $\mathbf{n}_{1}, \mathbf{n}_{2} \in \mathcal{N}(P)$ satisfying it.

Let’s assume $\mathbf{c}_{1} \ne \mathbf{c}_{2}$ and $\mathbf{n}_{1} \ne \mathbf{n}_{2}$.

Multiplying both sides of $\mathbf{c}_{1} + \mathbf{n}_{1} = \mathbf{c}_{2} + \mathbf{n}_{2}$ by $P$,

$$P\mathbf{c}_{1} + P\mathbf{n}_{1} = P\mathbf{c}_{2} + P\mathbf{n}_{2}$$

Meanwhile, since $\mathbf{n}_{1}, \mathbf{n}_{2} \in \mathcal{N} (P)$,

$$P\mathbf{c}_{1} = P\mathbf{c}_{2}$$

Thus $P( \mathbf{c}_{1} - \mathbf{c}_{2} ) = \mathbb{0}$. By the definition of the null space, $( \mathbf{c}_{1} - \mathbf{c}_{2}) \in \mathcal{N}(P)$, and by the property of vector spaces, $( \mathbf{c}_{1} - \mathbf{c}_{2}) \in \mathcal{C}(P)$, which by (exclusivity) must mean $\mathbf{c}_{1} - \mathbf{c}_{2} = \mathbb{0}$.

This contradicts the assumption $\mathbf{c}_{1} \ne \mathbf{c}_{2}$, and similarly, $\mathbf{n}_{1} = \mathbf{n}_{2}$ being true can be shown.

■

See also

• Idempotents in abstract algebra