📂Mathematical Statistics

Rao-Blackwell Theorem Proof

Theorem ^{1 2}

Description

To put the Rao-Blackwell Theorem into simple terms, it could be summarized as a theorem that ’tells why sufficient statistics are useful.’ An unbiased estimator becomes more effective, as in having a reduced variance, when information about the sufficient statistic is provided. Especially if $T$ is the minimum sufficient statistic, then $\phi \left( T \right)$ becomes the best unbiased estimator, as proven by the theorem.

Proof

Since from the assumption $T$ is a sufficient statistic, by its definition, the distribution of $W | T$ is independent of $\theta$, and likewise, $\phi \left( T \right) = E \left( W | T \right)$ is independent of $\theta$.

Properties of Conditional Expectation: $$E \left[ E ( X | Y ) \right] = E(X)$$

By the properties of conditional expectation: $$\begin{align*} \tau (\theta) =& E_{\theta} W \\ =& E_{\theta} \left[ E ( W | T ) \right] \\ =& E_{\theta} \phi (T) \end{align*}$$

Thus, $\phi (T)$ is an unbiased estimator for $\tau (\theta)$.

Properties of Conditional Variance: $$\operatorname{Var}(X) = E \left( \operatorname{Var}(X | Y) \right) + \operatorname{Var}(E(X | Y))$$

By the properties of conditional variance: $$\begin{align*} \operatorname{Var}_{\theta} W =& \operatorname{Var}_{\theta} \left[ E ( W | T ) \right] + E_{\theta} \left[ \operatorname{Var} ( W | T ) \right] \\ =& \operatorname{Var}_{\theta} \phi (T) + E_{\theta} \left[ \operatorname{Var} ( W | T ) \right] \\ \ge& \operatorname{Var}_{\theta} \phi (T) & \because \operatorname{Var} ( W | T ) \ge 0 \end{align*}$$

Therefore, $\operatorname{Var}_{\theta} \phi (T)$ always has a smaller variance than $W$.

■