📂Mathematical Statistics

Heavy-tail distribution and long-tail distribution in probability theory

정의 ¹

Random variable $X$ is assumed to have cumulative distribution function $F = F(x)$. For convenience, in this post we assume $X$ has a right tail and a probability density function $f$.

분포의 꼬리

다음과 같이 정의되는 $\overline{F}$ 를 $F$ 의 tail라 한다. 여기서 $(x, \infty)$ 는 interval이고, $f \left( x, \infty \right)$ 는 range of the interval에 해당한다. 정의에 따라, $\overline{F}$ 는 nondecreasing function다. $$ \overline{F} (x) := F \left( x , \infty \right) = P (X > x) $$ 아무 $x_{0}$ 에 대해 set $\left\{ \overline{F}(x) : x \ge x_{0} \right\}$ 에만 의존하는 $F$ 의 성질을 tail property이라 한다.

헤비테일

모든 $\lambda > 0$ 에 대해 다음을 만족하면 $X$ 가 heavy-tailed 분포를 따른다고 한다. $$ \int_{-\infty}^{\infty} e^{\lambda x} f(x) dx = \infty $$

롱테일

아무 $\delta > 0$ 에 대해 다음을 만족하면 $X$ 가 long-tailed 분포를 따른다고 한다. $$ \lim_{x \to \infty} {\frac{ \overline{F} \left( x + \delta \right) }{ \overline{F} (x) }} = 1 $$

설명

Whether heavy-tailed or long-tailed, such distributions with pronounced tails are important in applied mathematics because there can be non-negligible probabilities of events like “extremely large” outcomes. For example, the Pareto distribution, Cauchy distribution, log-normal distribution, and Weibull distribution can exhibit heavy-tail behavior; all of these have non-negligible probabilities of producing so-called outliers at very large scales.

The definition of heavy-tailed can be viewed directly from the formula as meaning that $f(x)$ does not dominate $e^{\lambda x}$, so the expression eventually diverges; put another way, it means the decay rate of $f$ is not exponential, so the tail is fat.

The definition of long-tailed is analogous: normally, as $x$ grows larger one would expect the tail to get progressively shorter to facilitate convergence, but the fact that it persists no matter how far out you go is what it means for the tail to be long. Long-tailed distributions are also heavy-tailed.