📂Statistical Analysis

Reversibility of ARMA Models

Definition ¹

In the ARMA model, having invertibility means that $AR(p)$ and $MA(q)$ can represent each other.

Examples

Although it is not a formula development for the general $ARMA ( p , q)$, let’s examine the examples of $AR(1)$ and $MA(1)$.

Autoregressive Model $AR(1) \implies MA( \infty )$

Considering the following autoregressive model $AR(1)$ for $| \phi | < 1$: $$ Y_{t} = \phi Y_{t-1} + e_{t} $$ $Y_{t-1}$ can also be represented as $Y_{t-1} = \phi Y_{t-2} + e_{t-1}$, thus $$ \begin{align*} Y_{t} =& \phi ( \phi Y_{t-2} + e_{t-1} ) + e_{t} \\ =& \phi^2 Y_{t-2} + e_{t} + \phi e_{t-1} \\ =& \phi^2 ( \phi Y_{t-3} + e_{t-2} ) + e_{t} + \phi e_{t-1} \\ =& \phi^3 Y_{t-3} + e_{t} + \phi e_{t-1} + \phi^2 e_{t-2} \end{align*} $$ If this process is repeated recursively indefinitely, it results in $\displaystyle \lim_{q \to \infty} \phi^{q} = 0$, thus $$ Y_{t} = e_{t} + \phi e_{t-1} + \phi^2 e_{t-2} + \cdots $$ In other words, it is $AR(1) \implies MA( \infty )$.

Moving Average Model $MA(1) \implies AR( \infty )$

Considering the following moving average model $MA(1)$ for $| \theta | < 1$: $$ Y_{t} = e_{t} - \theta e_{t-1} $$ $e_{t-1}$ can be represented as $e_{t-1} = Y_{t-1} + \theta e_{t-2}$, thus $$ \begin{align*} e_{t} =& Y_{t} + \theta ( Y_{t-1} + \theta e_{t-2}) \\ =& Y_{t} + \theta Y_{t-1} + \theta^2 e_{t-2} \\ =& Y_{t} + \theta Y_{t-1} + \theta^2 ( Y_{t-2} + \theta e_{t-3}) \\ =& Y_{t} + \theta Y_{t-1} + \theta^2 Y_{t-2} + \theta^3 e_{t-3} \end{align*} $$ If this process is repeated recursively indefinitely, it results in $\displaystyle \lim_{p \to \infty} \theta^{p} = 0$, thus $$ Y_{t} = e_{t} - \theta Y_{t-1} - \theta^2 Y_{t-2} - \cdots $$ In other words, it is $MA(1) \implies AR( \infty )$.

Theorem

Following this development, the ARMA model can in fact be represented as an autoregressive model, and the ARIMA model is nothing but an ARMA model that includes differencing, meaning that the ARIMA model itself can be represented as an autoregressive model. The reason to consider $AR ( \infty )$ rather than $MA ( \infty )$ is because the time series data we can obtain in reality is based on $y_{1} , \cdots , y_{t}$. This is why functions in time series related packages often only include ‘ar’.

Although invertibility itself may not be considered a very important condition when analyzing time series, knowing or not knowing these formulas is essential in understanding and diagnosing the model. $$ Y_{t} = e_{t} - \theta e_{t-1} $$

$$ Y_{t} = e_{t} - \theta Y_{t-1} - \theta^2 Y_{t-2} - \cdots $$ That the above formula is represented by the below formula is particularly noteworthy. At first glance, $Y_{t}$ might seem to be influenced only by the previous white noise, but in fact, it demonstrates that it reflects all previous data through the formula.