📂Statistical Analysis

CINDy for Overcoming Fractional Functions: SINDyPI

Algorithm ^{1 2}

SINDy algorithm: Consider a dynamical system whose state space is $\mathbb{R}^{n}$ and which is given by a smooth function $f : \mathbb{R}^{n} \to \mathbb{R}^{n}$ as follows. $$ \dot{\mathbf{x}} = f \left( \mathbf{x} \right) $$ Construct the matrix $\Theta \left( X \right) \in \mathbb{R}^{m \times p}$ of derived variables obtained by taking some nonlinear functions of the independent variables $X$, and apply STLSQ to the following matrix equation to recover the governing equation. This algorithm is called SINDy. $$ \dot{X} = \Theta \left( X \right) \Xi $$

Implicit differential equations: In particular, for ordinary differential equations, a differential equation that is expressed implicitly as follows is called an implicit ordinary differential equation. $$ f \left( x , \dot{x} \right) = 0 $$

As a variant of SINDy, algorithms that circumvent the problem by using implicit differential equations to recover rational functions are known as implicit SINDy or SINDy-PI.

Description

$$ \dot{x} = {\frac{ x }{ x + a }} $$ Because SINDy fundamentally recovers the governing equation as a linear combination of nonlinear terms, it has the drawback that it is difficult to discover even simple rational functions of the form above. Simply having $1/x$ multiplied can be remedied by enlarging the library, but to identify $a$ one would need infinitely many candidate terms, which is practically impossible.

The idea here is to express the problem as an implicit differential equation. Multiplying both sides by $x + a$ yields a differential equation that contains the rational function and can be written as follows. $$ \begin{align*} & (x + a) \dot{x} = x \ \\ \implies & a \dot{x} = x - x \dot{x} \ \\ \implies & \dot{x} = {\frac{ 1 }{ a }} x - {\frac{ 1 }{ a }} x \dot{x} \end{align*} $$ As a result, these algorithms target equations of the following form. $$ \dot{\mathbf{x}} = {\frac{ f_{N} \left( \mathbf{x} \right) }{ f_{D} \left( \mathbf{x} \right) }} $$

From a numerical-analysis viewpoint this effectively transfers the cost to solvers for implicit differential equations (so one cannot use some widely known solvers), but being able to derive an equation where one previously could not recover a rational function at all is itself a significant advance.

Implicit SINDy

Implicit SINDy is implemented by multiplying the original library once by $\dot{x}$ and adding that to the library. As illustrated in the figure, one may approach it via the null space, but one can also simply pass the matrix that has only $\dot{x}$ on the left-hand side and perform STLSQ.

SINDy-PI

According to the authors, SINDy-PI is more robust to noise and has a higher success probability than implicit SINDy. Setting up the equations as in the figure and searching for $\Xi$, there exists the trivial solution given by the identity matrix; to exclude it they reformulate the problem as an optimization problem with the constraint that $\diag \Xi$ is the zero vector.

In one sense this can be much more general than implicit SINDy, but having to find a $\Xi$ of size $p \times p$ can be excessive depending on the problem. In my personal experience I have seen libraries like $\dot{x_{2}} g(x)$ included even to predict $x_{1}$.