📂Mathematical Statistics

Heavy-tail distribution and long-tail distribution in probability theory

Definition ¹

Let a random variable $X$ have a cumulative distribution function $F = F(x)$. For convenience, throughout this post we assume that $X$ has a right tail and has a probability density function $f$.

Tail of a Distribution

The function $\overline{F}$ defined as follows is called the tail of $F$. Here $(x, \infty)$ is an interval, and $f \left( x, \infty \right)$ corresponds to the range of the interval. By definition, $\overline{F}$ is a non-decreasing function. $$ \overline{F} (x) := F \left( x , \infty \right) = P (X > x) $$ For any $x_{0}$, a property of $F$ that depends only on the set $\left\{ \overline{F}(x) : x \ge x_{0} \right\}$ is called a tail property.

Heavy-tailed

If the following holds for all $\lambda > 0$, then $X$ is said to follow a heavy-tailed distribution. $$ \int_{-\infty}^{\infty} e^{\lambda x} f(x) dx = \infty $$

Long-tailed

If the following holds for any $\delta > 0$, then $X$ is said to follow a long-tailed distribution. $$ \lim_{x \to \infty} {\frac{ \overline{F} \left( x + \delta \right) }{ \overline{F} (x) }} = 1 $$

Explanation

Whether heavy-tailed or long-tailed, the reason distributions with such prominent tails matter in applied mathematics is that there are cases where the probability of events such as ‘abnormally large’ ones is far from ordinary. For example, the Pareto distribution, Cauchy distribution, log-normal distribution, and Weibull distribution can all possess the heavy-tail property, and in all of them the probability of so-called ‘outlying data’ appearing at enormous scales is too large to be ignored.

The definition of heavy tail can be understood, from the formula itself, as $f(x)$ failing to overwhelm $e^{\lambda x}$ so that it ultimately diverges; to spell it out once more, it means that the tail is so thick that the rate at which $f$ decreases is not even exponential.

The definition of long tail is likewise: one would expect that the larger $x$ becomes, the more the tail should tend to shorten so as to converge easily, yet the very fact that it keeps holding on no matter how far back one goes is what it means for the tail to be long. A long-tailed distribution is also a heavy-tailed distribution.