📂Mathematical Statistics

Proof of Bézout's Theorem

Theorem

If $T \left( \mathbf{X} \right)$ is a complete statistic as well as a minimal sufficient statistic, then $T \left( \mathbf{X} \right)$ is independent of all ancillary statistics.

Description

Basu’s theorem is one of the most important results, among those related to sufficient statistics, allowing for a very strong conclusion that certain statistics are independent. Intuitively, a sufficient statistic contains all the information about a parameter $\theta$, and since ancillary statistics are not dependent on $\theta$, they naturally seem independent. However, completeness, a ‘common sense’ property along with minimalism, is required for a statistic to ensure independence.

A classic result is that the sample mean $\overline{X}$ and the sample variance $S^{2}$ from a normal distribution are independent, a fact that can also be shown without Basu’s theorem, for example, through Student’s theorem, but Basu’s theorem offers a slightly more general method of proof.

Proof ¹

Strategy: Let’s prove for the discrete probability distribution case. Assuming $S \left( \mathbf{X} \right)$ is an ancillary statistic for parameter $\theta$, then $P \left( S \left( \mathbf{X} \right) = s \right)$ is not dependent on $\theta$, and according to the definition of sufficient statistics, the conditional probability $$P \left( S \left( \mathbf{X} \right) = s \mid T \left( \mathbf{X} \right) = t \right) = P \left( \mathbf{X} \in \left\{ \mathbf{x} : S \left( \mathbf{x} \right) = s \right\} \mid T \left( \mathbf{X} \right) = t \right)$$ is also not dependent on $\theta$. Thus, it suffices to show the following: $$P \left( S \left( \mathbf{X} \right) = s \mid T \left( \mathbf{X} \right) = t \right) = P \left( S \left( \mathbf{X} \right) = s \right) \qquad , \forall t$$

---

According to the law of addition of probabilities: $$P \left( S \left( \mathbf{X} \right) = s \right) = \sum_{t} P \left( S \left( \mathbf{X} \right) = s \mid T \left( \mathbf{X} \right) = t \right) P_{\theta} \left( T \left( \mathbf{X} \right) = t \right)$$ Meanwhile, multiplying both sides of $1 = \sum_{t} P_{\theta} \left( T \left( \mathbf{X} \right) = t \right)$ by $P \left( S \left( \mathbf{X} \right) = s \right)$ gives: $$\begin{align*} P \left( S \left( \mathbf{X} \right) = s \right) \cdot 1 =& P \left( S \left( \mathbf{X} \right) = s \right) \cdot \sum_{t} P_{\theta} \left( T \left( \mathbf{X} \right) = t \right) \\ =& \sum_{t} P \left( S \left( \mathbf{X} \right) = s \right) P_{\theta} \left( T \left( \mathbf{X} \right) = t \right) \end{align*}$$ Defining the following statistic: $$g(t) := P \left( S \left( \mathbf{X} \right) = s \mid T \left( \mathbf{X} \right) = t \right) - P \left( S \left( \mathbf{X} \right) = s \right)$$ for all $\theta$, yields: $$\begin{align*} E_{\theta} g (T) =& \sum_{t} g(t) P_{\theta} \left( T \left( \mathbf{X} \right) = t \right) \\ =& P \left( S \left( \mathbf{X} \right) = s \right) - P \left( S \left( \mathbf{X} \right) = s \right) \\ =& 0 \end{align*}$$

Definition of a complete statistic: $$\forall \theta, E_{\theta} g (T) = 0 \implies \forall \theta, P_{\theta} \left( g(T) = 0 \right) = 1$$

Since $T \left( \mathbf{X} \right)$ is assumed to be a complete statistic, for all possible $t$ and all $\theta$, it follows: $$P_{\theta} \left( g(T) = 0 \right) = 1$$ In other words, the following holds: $$P \left( S \left( \mathbf{X} \right) = s \mid T \left( \mathbf{X} \right) = t \right) = P \left( S \left( \mathbf{X} \right) = s \right) \qquad , \forall t$$

■