📂Numerical Analysis

Complex Step Derivative Approximation

Formula

The derivative $f ' (x)$ of the function $f : \mathbb{R} \to \mathbb{R}$ can be approximated using complex numbers as follows. $$f ' (x) \approx \frac{\im \left( f \left( x + i h \right) \right)}{h}$$ Here, $\im$ represents the imaginary part of the complex number, and $h$ is a sufficiently small real number.

Explanation

Complex analysis continues to be fascinating the more you learn about it. Unlike the finite difference method, which approximates derivatives through multiple function values, this method approximates the derivative of $f$ by formally treating it like a complex function, even if it wasn’t a complex function.

Derivation ¹

There are two methods, both remarkably intriguing. Let’s represent the real number $x, y \in \mathbb{R}$ as the complex number $z = x + i y$, and the real function $f : \mathbb{R} \to \mathbb{R}$ as $f(z) = u(z) + i v(z)$ in terms of $u, v : \mathbb{C} \to \mathbb{R}$.

Cauchy-Riemann Equations

Cauchy-Riemann Equations: Suppose the function $f: A \subseteq \mathbb{C} \to \mathbb{C}$ is analytic at $\mathscr{R}$. If the real function $u,v$ satisfies $$f(z) = f(x+iy) = u(x,y) + iv(x,y)$$ then the partial derivatives of $u,v$ with respect to $x,y$ exist and satisfy the following system of differential equations at all points on $\mathscr{R}$. $$\begin{cases} u_{x} (x,y) = v_{y} (x,y) \\ u_{y} (x,y) = -v_{x} (x,y) \end{cases}$$

If $f$ is analytic, then $f = u + i v$ must satisfy the Cauchy-Riemann equations. $$\begin{align*} {\frac{ \partial u }{ \partial x }} =& {\frac{ \partial v }{ \partial y }} \\ {\frac{ \partial u }{ \partial y }} =& - {\frac{ \partial v }{ \partial x }} \end{align*}$$ Rewriting the right-hand side of $u_{x}$, which is $v_{y}$, in limit form gives the following. $${\frac{ \partial u }{ \partial x }} = \lim_{h \to 0} {\frac{ v \left( x + i \left( y + h \right) \right) - v \left( x + i y \right) }{ h }}$$ Though $f$ is formally treated like a complex function, it is inherently a real function. Thus, $y = 0$ and $v(x) = 0$ are valid, and $u(x)$ is indeed $u(x) = f(x)$. Substituting them one by one, with $y = 0$, we have $${\frac{ \partial u }{ \partial x }} = \lim_{h \to 0} {\frac{ v \left( x + i h \right) - v \left( x \right) }{ h }}$$ and with $v(x) = 0$, we have $${\frac{ \partial u }{ \partial x }} = \lim_{h \to 0} {\frac{ v \left( x + i h \right) }{ h }}$$ Finally, $f = u + i v$ leads to $\im f = v$, and for a sufficiently small $h$, we obtain the following formula. $${\frac{ \partial f }{ \partial x }} = \lim_{h \to 0} {\frac{ \im f \left( x + i h \right) }{ h }} \approx {\frac{ \im f \left( x + i h \right) }{ h }}$$

■

Taylor Expansion

Taylor Expansion: If the function $f(x)$ is continuous at $[a,b]$ and $n$ differentiable at $(a,b)$, then there exists $\xi \in (a,b)$ satisfying $$\begin{align*} f(b) =& \sum_{k=0}^{n-1} {{(b-a)^{k}\over{k!}}{f^{(k)}( a )}} + {(b-a)^{n}\over{n!}}{f^{(n)}(\xi)} \\ =& {f(a)} + {(b-a)f ' (a)} + \cdots + {(b-a)^{n-1}\over{(n-1)!}}{f^{(n-1)}(a)} + {(b-a)^{n}\over{(n)!}}{f^{(n)}(\xi)} \end{align*}$$

Performing a Taylor expansion from $f(x)$ to $i h$ yields the following. $$f \left( x + i h \right) = f (x) + i h f ' (x) - h^{2} {\frac{ f '' (x) }{ 2! }} - i h^{3} {\frac{ f ''' (x) }{ 3!s }} + \cdots$$ Since we are interested in $f ' (x)$, focusing on the imaginary part gives $$\im f \left( x + i h \right) = h f ' (x) - h^{3} {\frac{ f ''' (x) }{ 3!s }} + \cdots$$ and canceling $h$ results in $${\frac{ \im f \left( x + i h \right) }{ h }} = f ' (x) + O \left( h^{2} \right)$$ Similarly, if $h$ is sufficiently small, $h^{2}$ becomes negligible, providing the following approximation. $${\frac{ \im f \left( x + i h \right) }{ h }} \approx f ' (x)$$

■