logo

Maximum Likelihood Estimator of the Laplace Distribution 📂Probability Distribution

Maximum Likelihood Estimator of the Laplace Distribution

Theorem

Suppose we are given a random sample $\mathbf{X} := \left( X_{1} , \cdots , X_{n} \right) \sim \operatorname{Laplace}(\mu, b)$ that follows a Laplace distribution.

The maximum likelihood estimator $(\hat{\mu}, \hat{b})$ for $(\mu, b)$ is as follows.

$$ \hat{\mu} = \text{median}(\mathbf{x}_{1}, \cdots, \mathbf{x}_{n}) $$

$$ \hat{b} = \dfrac{1}{n} \sum\limits_{k=1}^{n} |x_{k} - \mu| $$

Proof

Laplace Distribution:

The Laplace distribution with parameters $\mu \in \mathbb{R}$ and $b > 0$ is a continuous probability distribution $\operatorname{Laplace}(\mu, b)$ characterized by the following probability density function.

$$ f(x) = \dfrac{1}{2b} \exp \left( -\dfrac{|x - \mu|}{b} \right) $$

Calculating the log likelihood gives us:

$$ \begin{align*} \log L(\mu, b ; \mathbf{x}) &= \log f(\mathbf{x}; \mu, b) = \log \prod\limits_{k=1}^{n} f(x_{k}; \mu, b) \\ &= \sum\limits_{k=1}^{n} \log f(x_{k}; \mu, b) \\ &= \sum\limits_{k=1}^{n} \log \left( \dfrac{1}{2b} \exp \left( -\dfrac{|x_{k} - \mu|}{b} \right) \right) \\ &= -n \log 2b - \dfrac{1}{b} \sum\limits_{k=1}^{n} |x_{k} - \mu| \end{align*} $$

Therefore, solving for $\argmax\limits_{\mu} \log L(\mu, b; \mathbf{x})$, we have:

$$ \begin{align*} \argmax_{\mu} \left( -n \log 2b - \dfrac{1}{b} \sum\limits_{k=1}^{n} |x_{k} - \mu| \right) &= \argmax_{\mu} \left( -\sum\limits_{k=1}^{n} |x_{k} - \mu| \right) \\ &= \argmin_{\mu} \left( \sum\limits_{k=1}^{n} |x_{k} - \mu| \right) \\ \end{align*} $$

Since minimizing the absolute value is the median,

$$ \hat{\mu} = \text{median}(\mathbf{x}_{1}, \cdots, \mathbf{x}_{n}) $$

Moreover, $\argmax\limits_{b} \log L(\mu, b; \mathbf{x})$ satisfies the following condition $b$.

$$ \dfrac{\partial}{\partial b} \log L(\mu, b; \mathbf{x}) = 0 $$

$$ \implies -n\dfrac{1}{b} + \dfrac{1}{b^{2}} \sum\limits_{k=1}^{n} |x_{k} - \mu| = 0 $$

$$ \implies b = \dfrac{1}{n} \sum\limits_{k=1}^{n} |x_{k} - \mu| $$

$$ \implies \hat{b} = \dfrac{1}{n} \sum\limits_{k=1}^{n} |x_{k} - \mu| $$