Gram-Schmidt Orthonormalization
Theorem
Every finite-dimensional inner space has an orthonormal basis.
Description
As with the proof of existence, which often appears neither long nor significant, it’s an incredibly important theorem. This is because numerous logics that support linear algebra depend on the existence of an orthonormal basis.
Proof
Let’s consider one of the bases that generates the inner space $(V, \left\langle \cdot , \cdot \right\rangle)$, named $\left\{ \mathbf{x}_{1} , \cdots , \mathbf{x}_{n} \right\}$. Define a new vector,
$$ \mathbf{v}_{1} := {{ \mathbf{x}_{1} } \over {||\mathbf{x}_{1}||}} $$
and
$$ {\color{blue} \mathbf{v}_{2}} := {{ {\color{red} \mathbf{x}_{2} - ( \mathbf{x}_{2} \cdot \mathbf{v}_{1} ) \mathbf{v}_{1} } } \over { || \mathbf{x}_{2} - ( \mathbf{x}_{2} \cdot \mathbf{v}_{1} ) \mathbf{v}_{1} || }} = {{ \mathbf{x}_{2} - \text{Proj} ( \mathbf{x}_{1} ) } \over { || \mathbf{x}_{2} - \text{Proj} ( \mathbf{x}_{1} ) || }} $$.
As shown in the figure, $\mathbf{v}_{1} \perp \mathbf{v}_{2}$ holds. Following the above process,
$$ \mathbf{v}_{k} := {{ \mathbf{x}_{k} - \text{Proj} ( \mathbf{x}_{k-1} ) } \over { || \mathbf{x}_{k} - \text{Proj} ( \mathbf{x}_{k-1} ) || }} $$ can be repeated; thus, we can find an orthogonal basis $\left\{ \mathbf{v}_{1} , \cdots , \mathbf{v}_{n} \right\}$. By definition, since $\mathbf{v}_{1} , \cdots , \mathbf{v}_{n}$ all have the magnitude $1$, $\left\{ \mathbf{v}_{1} , \cdots , \mathbf{v}_{n} \right\}$ are orthonormal vectors.
■
In fact, normalization is simply dividing by the norm to make the size 1, and the key point is to find orthogonal vectors using projection.