Exponential Function Set and Trigonometric Function Set are Orthogonal Bases
Theorem
Both sets $\left\{ e^{inx} \right\}_{n=-\infty}^\infty$ and $\left\{ \cos nx\ \right\}_{n=0}^\infty \cup \left\{ \sin nx \right\}_{n=1}^\infty$ are the orthonormal bases of $L^{2}(-\pi,\ \pi)$. Furthermore, $\left\{ \cos nx \right\}_{n=0}^{\infty}$ and $\left\{ \sin nx \right\}_{n=1}^{\infty}$ are the orthonormal bases of $L^{2}(0,\ \pi)$.
Explanation
This fact explains the reason why it is valid to express a given function as a series of trigonometric functions in Fourier series.
Proof
Let $\phi_{n}(x)=e^{inx}$. And assume $f \in L^{2}(-\pi,\ \pi)$ and $\epsilon$ to be a very small arbitrary positive number.
Lemma
For any $f \in L^{2}(a,\ b)$, there exists a sequence of smooth functions $\left\{ f_{n} \right\}$ that satisfies $| f_{n} - f | \rightarrow 0$ on $[a,\ b]$.
Then, according to the lemma, there exists $\tilde{f}$ that satisfies the following equation.
$$ \begin{equation} | f-\tilde{f} | < \frac{\epsilon}{3} \label{eq1} \end{equation} $$
Let’s say $c_{n}=\frac{1}{2\pi}\langle f,\ \phi_{n} \rangle$ and $\tilde{c}_{n}=\frac{1}{2\pi}\langle \tilde{f},\ \phi_{n} \rangle$ are the Fourier coefficients of $f$ and $\tilde{f}$. Then, the Fourier series of $\tilde{f}$, $\sum \tilde{c}_{n}\phi_{n}$, converges uniformly to $\tilde{f}$, and since it converges uniformly, it converges in norm. Hence, the following equation holds.
$$ \begin{equation} \left\| \tilde{f} -\sum \limits_{-N}^{N} \tilde{c}_{n}\phi_{n} \right\| < \dfrac{\epsilon}{3} \label{eq2} \end{equation} $$
And the following equation holds.
$$ \begin{align} \left\| \sum\limits_{-N}^{N}\tilde{c}_{n}\phi_{n} - \sum\limits_{-N}^{N}c_{n}\phi_{n}\right\| ^{2} &= \left\| \sum\limits_{-N}^{N} (\tilde{c}_{n}-c_{n}) \phi_{n} \right\| ^{2} \nonumber \\ &= \sum \limits_{-N}^{N} | \tilde{c}_{n} -c_{n} |^{2} | \phi_{n} |^{2} \nonumber \\ &= \sum \limits_{-N}^{N} \dfrac{1}{2\pi} | \langle \tilde{f}-f,\ \phi_{n} \rangle |^{2} | \phi_{n}|^{2} \nonumber \\ &= \sum \limits_{-N}^{N} \dfrac{1}{2\pi} | \langle \tilde{f}-f,\ \phi_{n} \rangle |^{2} \nonumber \\ &\le \sum \limits_{-\infty}^{\infty} \dfrac{1}{2\pi} | \langle \tilde{f}-f,\ \phi_{n} \rangle |^{2} \nonumber \\ &\le | \tilde{f} - f | ^{2} \nonumber \\ &< \left( \dfrac{\epsilon}{3} \right)^{2} \label{eq3} \end{align} $$
The fourth equality holds because of $ | \phi_{n} |=1$. The sixth line holds due to the Bessel’s inequality. The last line holds due to the assumption $\eqref{eq1}$. Now, using $(1)$, $(2)$, $(3)$, we show that $| f -\sum c_{n}\phi_{n} | \rightarrow 0$, and the proof is complete.
$$ f-\sum \limits_{-N}^{N}c_{n}\phi_{n} = (f-\tilde{f}) + \left( \tilde{f} -\sum _{-N}^{N} \tilde{c}_{n}\phi_{n} \right) + \left( \sum _{-N}^{N} \tilde{c}_{n}\phi_{n} - \sum _{-N}^{N}c_{n}\phi_{n}\right) $$
Since the above equation holds, by the triangle inequality,
$$ \begin{align*} \left\| f-\sum\limits_{-N}^{N}c_{n}\phi_{n} \right\| &\le \| f-\tilde{f} \| + \left\| \tilde{f} -\sum _{-N}^{N} \tilde{c}_{n}\phi_{n} \right\| + \left\| \sum _{-N}^{N} \tilde{c}_{n}\phi_{n} - \sum _{-N}^{N}c_{n}\phi_{n} \right\| \\ &= \dfrac{\epsilon}{3} + \dfrac{\epsilon}{3} + \dfrac{\epsilon}{3} \\ &= \epsilon \end{align*} $$
Therefore,
$$ \left\| f- \sum \limits_{-\infty}^{\infty} c_{n}\phi_{n} \right\| \rightarrow 0 $$
And according to condition (b), $\left\{ \phi_{n} \right\}$ is a complete orthonormal set.
■
The rest of the cases are essentially the same or can be proven by almost the same process, so they are omitted.