📂Mathematical Statistics

Sufficient Statistics and Maximum Likelihood Estimates of the Location Family

Theorem

Given a random sample $X_{1} , \cdots , X_{n} \sim X$ obtained from a location family with the probability density function $f_{X} \left( x ; \theta \right) = f_{X} \left( x - \theta \right)$, the sufficient statistic and maximum likelihood estimator depend on

• if the support of $X$ is upper bounded, then $\max X_{k}$
• if the support of $X$ is lower bounded, then $\min X_{k}$.

---

• The support of a random variable refers to the set of points where the function value of the probability density function is greater than $0$. $$S_{X} := \left\{ x \in \mathbb{R} : f_{X} (x ; \theta) > 0 \right\}$$
• A bounded set refers to a set for which there exists an element that is greater than or equal to every element of the given set.

Explanation

For example, if data $$0.7 \\ 0.8 \\ 0.1 \\ 0.2 \\ 0.1 \\ 0.9$$ were obtained from a uniform distribution $U \left( 0, \theta \right)$, then the sufficient statistic and maximum likelihood estimator for $\theta$ are naturally the largest value, $0.9$. In cases where the support itself is dependent on the location parameter $\theta$, one can simply think of $\min X_{k}$ without much consideration. The same applies to exponential distributions.

Proof

Let us only show the case where the support is lower bounded. That the support of $X$ is lower bounded is equivalent to the probability density function being represented as follows with respect to the indicator function $I$. $$f_{X} ( x ; \theta ) = f_{X} ( x ; \theta ) I_{[\theta, \infty)} (x)$$

Product of indicator functions: $$\prod_{i=1}^{n} I_{[\theta,\infty)} \left( x_{i} \right) = I_{[\theta,\infty)} \left( \min_{i \in [n]} x_{i} \right)$$

It might be perplexing that $I$ is not a differentiable function, but if one thinks based on its definition, there is nothing difficult. The likelihood function $L$ is $$L \left( \theta ; \mathbf{x} \right) = \prod_{k=1}^{n} f \left( x_{k} ; \theta \right) I_{[\theta, \infty)} \left( x_{k} \right) = I_{[\theta,\infty)} \left( \min_{{k} \in [n]} x_{k} \right) \prod_{k=1}^{n} f \left( x_{k} ; \theta \right)$$ thus, if $\min x_{k}$ is less than the estimate of $\theta$, $\hat{\theta}$, then $0$ inevitably gets multiplied. Therefore, the maximum likelihood estimator must depend on $\min X_{k}$, and since $L$ has the same form as the joint probability density function of $X_{1} , \cdots , X_{n}$, according to the Neyman factorization theorem, the sufficient statistic also must appear as a function of $\min X_{k}$.

■