Inner product in Euclidean space
Definition
Let’s say $V = \mathbb{R}^n$ for a vector space, and also $\mathbf{x}, \mathbf{y}, \mathbf{z} \in V$ and $k \in \mathbb{R}$.
$\left< \cdot , \cdot \right> : V^2 \to \mathbb{R}$ is defined as the inner product on $V$ when it satisfies the following four conditions:
(1) Symmetry: $\left< \mathbf{x} , \mathbf{y} \right> = \left< \mathbf{y}, \mathbf{x} \right>$
(2) Additivity: $\left< \mathbf{x} + \mathbf{y} , \mathbf{z} \right> = \left< \mathbf{x}, \mathbf{z} \right> + \left< \mathbf{y}, \mathbf{z} \right>$
(3) Homogeneity: $\left< k \mathbf{x} , \mathbf{y} \right> = k \left< \mathbf{x}, \mathbf{y} \right>$
(4) Positive-definiteness: $\left< \mathbf{x} , \mathbf{x} \right> \ge 0$ and $\left< \mathbf{x} , \mathbf{x} \right> =0 \iff \mathbf{x}=\mathbb{0}$
In particular, when $\mathbf{x} = (x_{1}, x_{2}, \cdots , x_{n})$ and $\mathbf{y} = (y_{1}, y_{2}, \cdots , y_{n})$,
$$ \left< \mathbf{x} , \mathbf{y}\right> = \mathbf{x} \cdot \mathbf{y} = x_{1} y_{1} + x_{2} y_{2} + \cdots + x_{n} y_{n} = \mathbf{x}^T \mathbf{y} $$
is defined as the dot product or Euclidean inner product.
Explanation
The concept of vector spaces itself can be generalized for any field $\mathbb{F}$. Naturally, the inner product can also be generalized, but in the basic level of linear algebra, it is common to deal only with Euclidean spaces.
However, the reason why inner product is confusing when learned in university is that it is sufficiently generalized compared to high school level. When considering the inner product by itself, if there exists a mapping that satisfies the conditions, there’s no particular reason to multiply components. The difference starts from ’the inner product we knew’ becoming ‘one of the inner products we learn in university’, which is the dot product. Not only that, it is generalized for $n$ dimensions, and loses its geometric properties, which can bring great confusion to the definitions of size or angle between vectors.
- [1]: $\left\| \mathbf{x} \right\| = \sqrt{ \mathbf{x} \cdot \mathbf{x} }$ is defined as the size or length of $\mathbf{x}$.
[2] $d(\mathbf{x}, \mathbf{y} ) = \left\| \mathbf{x} - \mathbf{y} \right\|$ is defined as the distance between $\mathbf{x}$ and $\mathbf{y}$.
[3] For $\theta \in [0 , \pi]$, $\displaystyle \cos \theta = {{ \mathbf{x} \cdot \mathbf{y} } \over { \left\| \mathbf{x} \right\| \left\| \mathbf{y} \right\|}}$ is defined as the angle between $\mathbf{x}$ and $\mathbf{y}$.
[4] When $\mathbf{x} \cdot \mathbf{y} = 0$, $\mathbf{x}$ and $\mathbf{y}$ are perpendicular is defined.
While up to 3 dimensions, it’s possible to manually calculate and visualize to see these definitions align with intuition, from 4 dimensions it becomes impossible. However, this kind of transcendental generalization is exactly the charm and strength of mathematics, and these definitions alone make it easy to generalize several theorems. See the two examples below.
Generalized Cauchy-Schwarz Inequality
$$ \left\| \mathbf{x} \right\| \left\| \mathbf{y} \right\| \ge \left< \mathbf{x} , \mathbf{y} \right> $$
The Cauchy-Schwarz Inequality originally holds for four unknowns but can be easily generalized using the inner product.
Proof
For an arbitrary scalar $t \in \mathbb{R}$,
$$ ( t \mathbf{x} + \mathbf{y} ) ^2 = t^2 (\mathbf{x} \cdot \mathbf{x}) + 2t (\mathbf{x} \cdot \mathbf{y} )+ ( \mathbf{y} \cdot \mathbf{y} ) $$
Since $t$ is a real number, by the quadratic formula it must be $(\mathbf{x} \cdot \mathbf{y})^2 - ( \mathbf{x} \cdot \mathbf{x} )( \mathbf{y} \cdot \mathbf{y} ) \le 0$.
■
Generalized Pythagorean Theorem
Suppose two vectors $\mathbf{x}$ and $\mathbf{y}$ are perpendicular, then $\left\| \mathbf{x} \right\|^2 + \left\| \mathbf{y} \right\|^2 = \left\| \mathbf{x} +\mathbf{y} \right\|^2$
The Pythagorean Theorem also was originally valid for triangles on the plane. To generalize it, one needs to use the Pythagorean Theorem of a lower dimension step by step as we go higher in dimension, but using inner product makes it much easier and concise.
Proof
$$ \begin{align*} \left\| \mathbf{x} +\mathbf{y} \right\|^2 =& ( \mathbf{x} + \mathbf{y} ) ^2 \\ =& (\mathbf{x} \cdot \mathbf{x}) + 2 (\mathbf{x} \cdot \mathbf{y} )+ ( \mathbf{y} \cdot \mathbf{y} ) \\ &= \left\| \mathbf{x} \right\|^2 + 2 (\mathbf{x} \cdot \mathbf{y} ) + \left\| \mathbf{y} \right\|^2 \end{align*} $$
By the definition of the inner angle, $(\mathbf{x} \cdot \mathbf{y} ) = \cos{\theta} \left\| \mathbf{x} \right\|^2 \left\| \mathbf{y} \right\|^2$ and since $\mathbf{x}$ and $\mathbf{y}$ are perpendicular to each other $\cos{\theta} = 0$
■
A More Generalized Pythagorean Theorem
Let’s assume $\mathbf{a}_1,\ \mathbf{a}_2,\ \cdots,\ \mathbf{a}_n$ are vectors perpendicular to each other. Then, the following formula holds:
$$ \left\| \mathbf{a}_1+\mathbf{a}_2+\cdots+\mathbf{a}_n \right\|^2 = \left\| \mathbf{a}_1 \right\|^2 + \left\| \mathbf{a}_2 \right\|^2 +\cdots+ \left\| \mathbf{a}_n \right\|^2 $$
This theorem is a generalization of the above for $n$ vectors.
Proof
According to the above definition, the following holds:
$$ \left\| \mathbf{a}_1+\mathbf{a}_2+\cdots+\mathbf{a}_n \right\| ^2=\left< \mathbf{a}_1 + \mathbf{a}_2 + \cdots + \mathbf{a}_n,\ \mathbf{a}_1 + \mathbf{a}_2+\cdots+\mathbf{a}_n \right> $$
By the additivity of the inner product, breaking down the above inner products results in:
$$ \begin{array} {l} \left< \mathbf{a}_1,\ \mathbf{a}_1 \right>+\left< \mathbf{a}_1,\ \mathbf{a}_2 \right>+\cdots+\left< \mathbf{a}_1,\ \mathbf{a}_n \right> \\ + \left< \mathbf{a}_2,\ \mathbf{a}_1 \right> + \left< \mathbf{a}_2,\ \mathbf{a}_2 \right>+\cdots+\left< \mathbf{a}_2,\ \mathbf{a}_n \right> \\ +\cdots \\ + \left< \mathbf{a}_n,\ \mathbf{a}_1 \right>+\left< \mathbf{a}_n,\ \mathbf{a}_2 \right>+\cdots +\left< \mathbf{a}_n,\ \mathbf{a}_n \right> \end{array} $$
Because of the assumption, $\left< \mathbf{a}_i,\ \mathbf{a}_j \right>=\delta_{ij}$, so only the inner products of the same vectors remain, and the rest are $0$. Therefore,
$$ \begin{align*} \left\| \mathbf{a}_1+\mathbf{a}_2+\cdots+\mathbf{a}_n \right\|^2 =& \left< \mathbf{a}_1,\ \mathbf{a}_1 \right> + \left< \mathbf{a}_2,\ \mathbf{a}_2 \right>+\cdots+\left< \mathbf{a}_n,\ \mathbf{a}_n \right> \\ =& \left\| \mathbf{a}_1 \right\|^2 + \left\| \mathbf{a}_2 \right\|^2 +\cdots+\left\| \mathbf{a}_n \right\|^2 \end{align*} $$
■