Taylor's Theorem Rest Term
Definition1 2
For a differentiable function $f$, the $P_{k}$ defined below is called the Taylor polynomial of $f$ at point $a$.
$$ P_{k} (x) := f(a) + f^{\prime}(a) (x-a) + \dfrac{f^{\prime \prime}(a)}{2!}(x-a)^{2} + \cdots + \dfrac{f^{(k)}(a)}{k!}(x-a)^{k} $$
The difference between $f$ and $P_{k}$ is called the remainder term.
$$ R_{k}(x) = f(x) - P_{k}(x) $$
Explanation
$$ f(x) = P_{k}(x) + R_{k}(x) = \sum \limits_{n=0}^{k} \dfrac{f^{(n)}(a)}{n!} (x-a)^{n} + R_{k}(x) $$
If we rearrange $f$ in terms of the Taylor polynomial $P_{k}$ and the remainder, then the remainder $R_{k}$ becomes the error when approximating $f$ with its derivatives up to the $k$st.
Peano Form
The remainder such as $R_{k}(x) = \mathcal{o}((x-a)^{k})$ is called the Peano form of the remainder.
$$ f(x) = \sum \limits_{n=0}^{k} \dfrac{f^{(n)}(a)}{n!} (x-a)^{n} + \mathcal{o}((x-a)^{k}) $$
Here, $\mathcal{o}((x-a)^{k})$ means any function $g$ that satisfies $\lim \limits_{x \to a} \dfrac{g(x)}{(x-a)^{k}} = 0$. This is usually used when one wants to roughly specify the remainder without stating it explicitly.
Lagrange Form
The remainder such as $R_{k}(x) = \dfrac{f^{(k+1)}(\xi)}{(k+1)!} (x-a)^{k+1}$ is called the Lagrange form of the remainder.
$$ f(x) = \sum \limits_{n=0}^{k} \dfrac{f^{(n)}(a)}{n!} (x-a)^{n} + \dfrac{f^{(k+1)}(\xi)}{(k+1)!} (x-a)^{k+1} \quad \text{for some } \xi \in (x,a) $$
It is one of the commonly used forms along with the Peano form. Substituting $k=0$ results in the Mean Value Theorem.
$$ f(x) = f(a) + f^{\prime}(\xi)(x-a) \implies \dfrac{f(x) - f(a)}{x-a} = f^{\prime}(\xi) $$
The following alternative equations can be derived:
For $n$ $$ f(x + p) = \sum\limits_{k=0}^{n-1} \dfrac{f^{(k)}(x)}{k!}p^{n} + \dfrac{1}{n!}f^{(n)}(x + \xi p) p^{n} \quad \text{for some } \xi \in (0,1) $$
$n=1$ $$ f(x + p) = f(x) + pf^{\prime}(x + \xi p) \quad \text{for some } \xi \in (0,1) $$
$n=2$ $$ f(x + p) = f(x) + pf^{\prime}(x) + \dfrac{1}{2!}p^{2} f^{\prime \prime}(x + \xi p) \quad \text{for some } \xi \in (0,1) $$
Derivation3
Since the method of proof is the same, only the case for $n=2$ is shown. When we perform a Taylor expansion of the remainder in the Lagrange form for a suitable $g$,
$$ g(t_{1}) = g(t_{0}) + g^{\prime}(t_{0}) (t_{1} - t_{0}) + \dfrac{1}{2!}g^{\prime \prime}(\xi) (t_{1} - t_{0})^{2} \quad \text{for some } \xi \in (t_{0},t_{1}) $$
By substituting $t_{0}=0$ and $t_{1}=1$,
$$ g(1) = g(0) + g^{\prime}(0) + \dfrac{1}{2!}g^{\prime \prime}(\xi) \quad \text{for some } \xi \in (0,1) $$
Now, if we set as $g(\xi) = f(x + \xi p)$, then since $g^{\prime}(\xi) = pf^{\prime}(x + \xi p)$ and $g^{\prime \prime}(\xi) = p^{2}f^{\prime \prime}(x + \xi p)$,
$$ f(x + p) = f(x) + pf^{\prime}(x) + \dfrac{1}{2!}p^{2} f^{\prime \prime}(x + \xi p) \quad \text{for some } \xi \in (0,1) $$
■
Cauchy Form
The remainder such as $R_{k}(x) = \dfrac{f^{(k+1)}(\xi)}{k!} (x-\xi)^{k} (x-a)$ is called the Cauchy form of the remainder.
$$ f(x) = \sum \limits_{n=0}^{k} \dfrac{f^{(n)}(a)}{n!} (x-a)^{n} + \dfrac{f^{(k+1)}(\xi)}{k!} (x-\xi)^{k} (x-a) \quad \text{for some } \xi \in (x,a) $$
Integral Form
$\displaystyle R_{k}(x) = \int_{a}^{x} \dfrac{f^{(k+1)}(t)}{k!} (x-t)^{k} dt$ is called the integral form of the remainder.
$$ \begin{equation} f(x) = \sum \limits_{n=0}^{k} \dfrac{f^{(n)}(a)}{n!} (x-a)^{n} + \int_{a}^{x} \dfrac{f^{(k+1)}(t)}{k!} (x-t)^{k} dt \end{equation} $$
$$ \begin{equation} f(x + p) = \sum \limits_{n=0}^{k} \dfrac{f^{(n)}(x)}{n!} p^{n} + \int_{0}^{1} \dfrac{f^{(k+1)}(x + tp)}{k!} (1-t)^{k} dt p^{k+1} \end{equation} $$
Derivation
(1)
By the Fundamental Theorem of Calculus,
$$ \begin{equation} f(x) - f(a) = \int_{a}^{x} f^{\prime}(t_{1})dt_{1} \implies f(x) = f(a) + \int_{a}^{x} f^{\prime}(t_{1})dt_{1} \end{equation} $$
When this is applied to $f^{\prime}(t_{1})$,
$$ \begin{equation} f^{\prime}(t_{1}) = f^{\prime}(a) + \int_{a}^{t_{1}} f^{\prime \prime}(t_{2})dt_{2} \end{equation} $$
By substituting $(4)$ into $(3)$,
$$ \begin{align*} f(x) &= f(a) + \int_{a}^{x} \left( f^{\prime}(a) + \int_{a}^{t_{1}} f^{\prime \prime}(t_{2})dt_{2} \right) dt_{1} \\ &= f(a) + f^{\prime}(a) (x-a) + \int_{a}^{x} \int_{a}^{t_{1}} f^{\prime \prime}(t_{2})dt_{2} dt_{1} \\ &= f(a) + f^{\prime}(a) (x-a) + \int_{a}^{x} \int_{a}^{t_{1}} f^{\prime \prime}(t_{2})dt_{2} dt_{1} \\ \end{align*} $$
$f^{\prime \prime}(t_{2})$ is then as follows.
$$ f^{\prime \prime}(t_{2}) = f^{\prime \prime}(a) + \int_{a}^{t_{2}} f^{\prime \prime \prime}(t_{3})dt_{3} $$
By substituting this into the above equation,
$$ \begin{align*} f(x) &= f(a) + f^{\prime}(a) (x-a) + \int_{a}^{x} \int_{a}^{t_{1}} f^{\prime \prime}(t_{2})dt_{2} dt_{1} \\ &= f(a) + f^{\prime}(a) (x-a) + \int_{a}^{x} \int_{a}^{t_{1}} \left( f^{\prime \prime}(a) + \int_{a}^{t_{2}} f^{\prime \prime \prime}(t_{3})dt_{3} \right)dt_{2} dt_{1} \\ &= f(a) + f^{\prime}(a) (x-a) + \int_{a}^{x} \int_{a}^{t_{1}} f^{\prime \prime}(a) dt_{2} dt_{1} + \int_{a}^{x}\int_{a}^{t_{1}}\int_{a}^{t_{2}} f^{\prime \prime \prime}(t_{3})dt_{3} dt_{2} dt_{1} \\ &= f(a) + f^{\prime}(a) (x-a) + \frac{f^{\prime \prime}(a)}{2}(x-a)^{2} + \int_{a}^{x}\int_{a}^{t_{1}}\int_{a}^{t_{2}} f^{\prime \prime \prime}(t_{3})dt_{3} dt_{2} dt_{1} \\ \end{align*} $$
Repeating this,
$$ f(x) = \sum \limits_{n=0}^{k} \dfrac{f^{(n)}(a)}{n!} (x-a)^{n} + \int_{a}^{x} \cdots \int_{a}^{t_{k}} f^{(k+1)}(t_{k+1})dt_{k+1} \cdots dt_{1} \\ $$
Now looking at the latter integral, when we examine the case with just two integrals, we can change the order and limits of integration as follows.
$$ \int_{a}^{x} \int_{a}^{t_{1}} f^{\prime \prime}(t_{2})dt_{2} dt_{1} = \int_{a}^{x} \int_{t_{2}}^{x} f^{\prime \prime}(t_{2})dt_{1} dt_{2} $$
$$ \begin{cases} a \lt t_{2} \lt t_{1} \\ a \lt t_{1} \lt x \end{cases} = \begin{cases} a \lt t_{2} \lt x \\ t_{2} \lt t_{1} \lt x \end{cases} $$
Therefore, the latter integral term is,
$$ \begin{align*} &\int_{a}^{x} \int_{a}^{t_{1}} \cdots \int_{a}^{t_{k-1}} \int_{a}^{t_{k}} f^{(k+1)}(t_{k+1})dt_{k+1}dt_{k} \cdots dt_{2} dt_{1} \\ &=\int_{a}^{x} \int_{t_{k+1}}^{x} \cdots \int_{t_{3}}^{x} \int_{t_{2}}^{x} f^{(k+1)}(t_{k+1})dt_{1}dt_{2} \cdots dt_{k} dt_{k+1} \\ &=\int_{a}^{x} f^{(k+1)}(t_{k+1}) \int_{t_{k+1}}^{x} \cdots \int_{t_{3}}^{x} \int_{t_{2}}^{x} dt_{1}dt_{2} \cdots dt_{k} dt_{k+1} \\ &=\int_{a}^{x} f^{(k+1)}(t_{k+1}) \dfrac{(x-t_{k+1})^{k}}{k!} dt_{k+1} \\ &=\int_{a}^{x} f^{(k+1)}(t) \dfrac{(x-t)^{k}}{k!} dt \end{align*} $$
■
(2)
By using $(1)$ for $\gamma$,
$$ \gamma (s_{1}) = \sum \limits_{n=0}^{k} \dfrac{\gamma^{(n)}(s_{0})}{n!} (s_{1}-s_{0})^{n} + \int_{s_{0}}^{s_{1}} \dfrac{\gamma^{(k+1)}(t)}{k!} (s_{1}-t)^{k} dt $$
By substituting $s_{1} = 1$, $s_{0}=0$,
$$ \gamma (1) = \sum \limits_{n=0}^{k} \dfrac{\gamma^{(n)}(0)}{n!} + \int_{0}^{1} \dfrac{\gamma^{(k+1)}(t)}{k!} (1-t)^{k} dt $$
Now, setting as $\gamma (t) = f(x + tp)$, since $\gamma ^{(n)}(t) = \frac{d^{n} \gamma (t)}{d t^{n}} = p^{n}f^{(n)}(x +tp)$,
$$ f(x + p) = \sum \limits_{n=0}^{k} \dfrac{f^{(n)}(x)}{n!} p^{n} + \int_{0}^{1} \dfrac{f^{(k+1)}(x + tp)}{k!} (1-t)^{k} dt p^{k+1} $$
■