📂Hilbert Space

Orthogonal Decomposition Theorem Proof

Theorem¹

Let $\left( H, \left\langle \cdot,\cdot \right\rangle \right)$ be a Hilbert space. Then for the closed subspace $M$ of $H$,

$$ H = M \oplus M^{\perp} $$

Corollary

$$ \left( M^{\perp} \right)^{\perp} = M $$

This fact can be demonstrated for $\left( M^{\perp} \right)^{\perp} := \left\{ \mathbf{x} \in H \mid \left\langle \mathbf{x} , \mathbf{m}^{\perp} \right\rangle = 0 , \mathbf{m}^{\perp} \in M^{\perp} \right\}$ as a corollary.

Explanation

$M^{\perp } := \left\{ \mathbf{x} \in H \mid \left\langle \mathbf{x} , \mathbf{m} \right\rangle = 0 , \mathbf{m} \in M \right\}$ is called the orthogonal complement of $M$. Being orthogonal is as useful a property as any. That Hilbert spaces guarantee this means they are indeed nice spaces.

On the other hand, since the proof involves the shortest vector theorem, it does not hold for inner spaces that are not Hilbert spaces.

Proof

Strategy: The proof simply shows the conditions to be represented as a direct sum.

---

If $\mathbf{x} \in M$, then since $\mathbf{x} = \mathbf{x} + \mathbb{0}$, there’s nothing to prove. Hence, suppose $\mathbf{x} \in H$ and $\mathbf{x} \notin M$. Then $M \lneq H$ and since $M$ is a closed set, the shortest vector theorem can be applied.

Shortest Vector Theorem: Let $H$ be a Hilbert space. Suppose $M \lneq H$ is a nonempty, closed, convex subset. Then for $\mathbf{x} \in ( H \setminus M)$

$$ \delta := \| \mathbf{x} - \mathbf{m}_{0} \| = \inf_{\mathbf{m} \in M} \| \mathbf{x} - \mathbf{m} \| > 0 $$

there uniquely exists $\mathbf{m}_{0} \in M$ satisfying it.

Consider some $\mathbf{m} \in M$ for which $t := \left\langle \mathbf{x} - \mathbf{m}_{0 } , \mathbf{m} \right\rangle \in \mathbb{C}$.

• Case 1. $t = 0$

  Since $\left\langle \mathbf{x} - \mathbf{m}_{0 } , \mathbf{m} \right\rangle = t = 0$, then $( \mathbf{x} - \mathbf{m}_{0} ) \in M^{ \perp }$.

• Case 2. $t \ne 0$

  For all $\lambda \in \mathbb{C}$,

  $$ \begin{align*} \delta^2 \le & \| \mathbf{x}_{0} - ( \mathbf{m}_{0} - \lambda \mathbf{m} ) \|^2 \\ =& \left\langle (\mathbf{x} - \mathbf{m}_{0}) + \lambda \mathbf{m} , (\mathbf{x} - \mathbf{m}_{0}) + \lambda \mathbf{m} \right\rangle \\ =& \| \mathbf{x} - \mathbf{m}_{0} \|^2 + \overline{ \lambda } \left\langle \mathbf{x} - \mathbf{m}_{0} , \mathbf{m} \right\rangle + \lambda \left\langle \mathbf{m} , \mathbf{x} - \mathbf{m}_{0} \right\rangle + | \lambda |^2 \| \mathbf{m} \|^2 \\ =& \| \mathbf{x} - \mathbf{m}_{0} \|^2 + \overline{ \lambda } \left\langle \mathbf{x} - \mathbf{m}_{0} , \mathbf{m} \right\rangle + \overline{ \overline{\lambda} \left\langle \mathbf{x} - \mathbf{m}_{0} , \mathbf{m} \right\rangle } + | \lambda |^2 \| \mathbf{m} \|^2 \\ =& \| \mathbf{x} - \mathbf{m}_{0} \|^2 + \overline{ \lambda } t + \overline{ \overline{\lambda} t } + | \lambda |^2 \| \mathbf{m} \|^2 \\ =& \delta^2 + 2 \operatorname{Re} ( \overline{ \lambda } t ) + | \lambda |^2 \| \mathbf{m} \|^2 \end{align*} $$

  Subtracting $\delta^2$ from both sides yields

  $$ 0 \le 2 \operatorname{Re} ( \overline{ \lambda } t ) + | \lambda |^2 \| \mathbf{m} \|^2 $$.

  Here, $\operatorname{Re} ( \overline{ \lambda } t )$ denotes the real part of $\overline{ \lambda } t$.

  • Case 2-1. $\| \mathbf{m} \| = 1$

    $$ 0 \le 2 \operatorname{Re} ( \overline{ \lambda } t ) + | \lambda |^2 $$

    This inequality holds for all $\lambda \in \mathbb{C}$, so if we set $\lambda = -t$,

    $$ 0 \le 2 \operatorname{Re} ( \overline{ -t } \cdot t ) + | t |^2 = - 2 | t |^2 + | t |^2 = -| t |^2 $$

    That is, $|t| = 0$, which should be $t = 0$, contradicting the assumption of Case 2.

  • Case 2-2. $\| \mathbf{m} \| \ne 1$

    $$ \begin{align*} t =& \left\langle \mathbf{x} - \mathbf{m}_{0} , \mathbf{m} \right\rangle \\ =& \| \mathbf{m} \| \left\langle \mathbf{x} - \mathbf{m}_{0} , {{\mathbf{m}} \over { \| \mathbf{m} \| }} \right\rangle \\ =& \| \mathbf{m} \| \cdot 0 \\ =& 0 \end{align*} $$

    That is, again $t = 0$, which also contradicts the assumption of Case 2.

Ultimately, by Case 1., it has to be $t = 0$. This means that for the $\mathbf{m}_{0} \in M$ whose existence is guaranteed by the shortest vector theorem, $(\mathbf{x} - \mathbf{m}_{0}) \in M^{\perp}$ is true. Hence, any $\mathbf{x} \in H$ can be represented as follows.

$$ \mathbf{x} = \mathbf{m}_{0} + (\mathbf{x} - \mathbf{m}_{0}) \in M + M^{ \perp } $$

To demonstrate uniqueness, consider $\mathbf{m}_{1} , \mathbf{m}_{2} \in M$ and $z_{1} , z_{2} \in M^{\perp}$ and set

$$ \mathbf{x} = \mathbf{m}_{1} + z_{1} = \mathbf{m}_{2} + z_{2} $$

Then,

$$ \mathbf{m}_{1} - \mathbf{m}_{2} = z_{2} - z_{1} \in \left( M \cap M^{\perp} \right) = \left\{ \mathbb{0} \right\} $$

In other words,

$$ \mathbf{m}_{1} - \mathbf{m}_{2} = \mathbb{0} \implies \mathbf{m}_{1} = \mathbf{m}_{2} \implies z_{2} = z_{1} $$

And, the method to represent $\mathbf{x}$ is unique.

■

Corollary

• $( \subset )$

  If $y \in \left( M^{\perp} \right)^{\perp}$, then $\left\langle y , \mathbf{m}^{\perp} \right\rangle = 0$, and either $y = \mathbb{0}$ or $y \notin M^{\perp}$. However, by the orthogonal decomposition theorem, since $H = M \oplus M^{\perp}$, it must be $y \in M$,

  $$ \left( M^{\perp} \right)^{\perp} \subset M $$

• $( \supset )$

  If $\mathbf{m} \in M$, then $\left\langle \mathbf{m} , \mathbf{m}^{\perp} \right\rangle = 0$, and since $\mathbf{m} \in \left( M^{\perp} \right)^{\perp}$,

  $$ M \subset \left( M^{\perp} \right)^{\perp} $$

■