logo

Situations Where Implicit Methods Are Preferred Over Explicit Methods in Solving Ordinary Differential Equations 📂Numerical Analysis

Situations Where Implicit Methods Are Preferred Over Explicit Methods in Solving Ordinary Differential Equations

Overview

y˙=f(y) \dot{y} = f(y) The numerical solver used to solve the given ordinary differential equation above is categorized into explicit methods and implicit methods based on their calculation techniques. Generally, explicit methods are easy to implement and use, while implicit methods—although more complex to solve—tend to be stable under various conditions. While this is not always the case, there are situations where implicit methods are necessary regardless of their pros and cons.

Example

tn+k=tn+khyn=y(yn) \begin{align*} t_{n+k} =& t_{n} + k h \\ y_{n} =& y \left( y_{n} \right) \end{align*} Let’s assume a time step tnt_{n} and an approximate solution yny_{n} as outlined above for h>0h > 0.

As an example, the Euler’s method can be derived using a forward difference as follows: f(y)=y˙=limh0y(t+h)y(t)h    f(yn)1h[y(tn+1)y(tn)]    yn+1=yn+hf(yn) \begin{align*} & f(y) = \dot{y} = \lim_{h \to 0} {\frac{ y (t + h) - y(t) }{ h }} \\ \implies & f \left( y_{n} \right) \approx {\frac{ 1 }{ h }} \left[ y \left( t_{n+1} \right) - y \left( t_{n} \right) \right] \\ \implies & y_{n+1} = y_{n} + h f \left( y_{n} \right) \end{align*} However, from the definition of derivative, such a derivation should also be possible using a backward difference as follows: f(y)=y˙=limh0y(t)y(th)h    f(yn)1h[y(tn)y(tn1)]    yn=yn1+hf(yn) \begin{align*} & f(y) = \dot{y} = \lim_{h \to 0} {\frac{ y (t) - y(t - h) }{ h }} \\ \implies & f \left( y_{n} \right) \approx {\frac{ 1 }{ h }} \left[ y \left( t_{n} \right) - y \left( t_{n-1} \right) \right] \\ \implies & y_{n} = y_{n-1} + h f \left( y_{n} \right) \end{align*} The update rule ynyn1+hf(yn)y_{n} \gets y_{n-1} + h f \left( y_{n} \right) obtained in this way is known as the implicit Euler method. Although there is no issue with the mathematical derivation, there is a problem when it comes to practical use: to compute the left-hand side yny_{n}, one already needs to know yny_{n} from the right-hand side. Unlike the explicit Euler method, which has a straightforward sequential calculation technique, the implicit Euler method requires finding yny_{n} that satisfies the equation at each step, dramatically increasing computational cost, often necessitating techniques such as the Newton-Raphson method.

At this juncture, it may seem unnecessary to replace explicit methods with implicit methods.

Problems That Can Arise with Explicit Methods

However, explicit methods suffer from a lack of stability, especially when dealing with stiff differential equations, where explicit methods might not converge. Consider the following Dahlquist problem concerning λ>0\lambda > 0. y˙=λy \dot{y} = - \lambda y This equation has a non-trivial solution y(t)=y0eλty (t) = y_{0} e^{-\lambda t} simple enough for high school students to solve, but explicit methods can illuminate its weaknesses numerically. The explicit Euler method for this problem is given by: yn+1=ynhλyn=(1hλ)yn y_{n+1} = y_{n} - h \lambda y_{n} = \left( 1 - h \lambda \right) y_{n} Theoretically, we know the non-trivial solution of this equation is y(t)=y0eλt>0y (t) = y_{0} e^{-\lambda t} > 0 when y0>0y_{0} > 0, but when λ0\lambda \gg 0—that is, when yy varies too drastically—the standard assumption of a ‘sufficiently small hh’ might not hold. In such cases, 1hλ<01 - h \lambda < 0 occurs, causing yny_{n} to oscillate and show behavior entirely different from the true solution, and it might not converge at all if (1hλ)<1\left( 1 - h \lambda \right) < -1.

Stability of Implicit Methods

In contrast, the implicit Euler method for the same problem is as follows: yn=yn1hλyn    (1+hλ)yn=yn1    yn=11+hλyn1 \begin{align*} y_{n} =& y_{n-1} - h \lambda y_{n} \\ \implies \left( 1 + h \lambda \right) y_{n} =& y_{n-1} \\ \implies y_{n} =& {\frac{ 1 }{ 1 + h \lambda }} y_{n-1} \end{align*} Disregarding the numerical computation aspect, implicit methods update by always multiplying the initially given y0y_{0} by a positive (1+hλ)1\left( 1 + h \lambda \right)^{-1}, hence there is no sign reversal.

On deeper inspection, these phenomena occur due to fundamental characteristics inherent to the methods:

  • Explicit methods proceed assuming the system conditions are satisfied, adding some term to yny_{n} without verifying whether that ‘some term’ truly satisfies the given conditions, which can lead to issues.
  • Implicit methods, on the other hand, advance by ensuring that yny_{n} satisfies the given approximation within a certain tolerance at each step, naturally resulting in greater stability.