📂Probability Distribution

Cauchy Distribution: A Distribution Without a Population Mean

Definition

A continuous probability distribution $C$ with the following probability density function is called the Cauchy distribution. $$ f(x) = {1 \over \pi} {1 \over {x^2 + 1}} \qquad , x \in \mathbb{R} $$

Explanation

It may seem like every probability distribution has a mean and a variance, but in reality this is not the case. The representative example is the Cauchy distribution, which at first glance resembles the normal distribution but has heavier tails on both sides. Since the moment generating function does not exist regardless of the parameters, nothing involving moments—whether the population mean or the population variance—can exist.

Of course, whether or not the population mean exists, the sample mean can still be computed. In fact, for a Cauchy distribution translated by $\theta$ along the $x$ axis, the mle $\hat{\theta}$ of $\theta$ is given by the sample mean.

Meanwhile, the probability density function of the t-distribution is $$ g(y) = {{\Gamma ( (n+1)/2 ) } \over { \sqrt{\pi n} \Gamma (n/2) }} { {1} \over {(1 + y^{2} / n)^{(n+1)/2} } } $$ so the Cauchy distribution can also be viewed as a t-distribution with degrees of freedom $n=1$.

Theorem

The moment generating function of the Cauchy distribution does not exist.

Proof¹

The probability distribution function of the Cauchy distribution is given by $\displaystyle f(x) = {1 \over \pi} {1 \over {x^2 + 1}}, -\infty < x < \infty$. It suffices to show that the moment generating function $\displaystyle E(e^{tx}) = \int_{-\infty}^{\infty} e^{tx} {1 \over \pi} {1 \over {x^2 + 1}} dx$ diverges.

Considering $e^{tx}$ when $t>0$, by the mean value theorem there exists $0< \xi < tx$ satisfying $$ {{e^{tx} - e^0} \over {tx - 0}} = { { e^{tx} - 1 } \over {tx} } = e^{\xi} \ge e^0 = 1 $$ Rearranging the above expression slightly yields the following inequality. $$ e^{tx} \ge 1 + tx \ge tx $$ Returning to the integral, $$ \begin{align*} E(e^{tx}) \ge& \int_{-\infty}^{\infty} e^{tx} {1 \over \pi} {1 \over {x^2 + 1}} dx \\ \ge& \int_{0}^{\infty} e^{tx} {1 \over \pi} {1 \over {x^2 + 1}} dx \\ \ge& \int_{0}^{\infty} {1 \over \pi} {tx \over {x^2 + 1}} dx \\ =& { t \over {2 \pi} } \left[ \ln (x^2+1) \right]_{0}^{\infty} \\ =& \infty \end{align*} $$ Therefore, the moment generating function of the Cauchy distribution does not exist.

■

Code

The following is Julia code that shows the probability density functions of the Cauchy distribution, the t-distribution, and the Cauchy distribution.

@time using LaTeXStrings
@time using Distributions
@time using Plots

cd(@__DIR__)

x = -4:0.1:4
plot(x, pdf.(Cauchy(), x),
 color = :red,
 label = "Cauchy", size = (400,300))
plot!(x, pdf.(TDist(3), x),
 color = :orange,
 label = "t(3)", size = (400,300))
plot!(x, pdf.(TDist(30), x),
 color = :black, linestyle = :dash,
 label = "t(30)", size = (400,300))
plot!(x, pdf.(Normal(), x),
 color = :black,
 label = "Standard Normal", size = (400,300))

xlims!(-4,5); ylims!(0,0.5); title!(L"\mathrm{pdf\,of\, t}(\nu)")
png("pdf")