📂Mathematical Statistics

Power Function of Hypothesis Testing

Definition ¹

Hypothesis Testing: $$\begin{align*} H_{0} :& \theta \in \Theta_{0} \\ H_{1} :& \theta \in \Theta_{0}^{c} \end{align*}$$

Given the hypothesis testing above, let’s denote it as $\alpha \in [0,1]$.

1. For the parameter $\theta$, the function $\beta (\theta) := P_{\theta} \left( \mathbf{X} \in \mathbb{R} \right)$ with the rejection region $R$ is called the Power Function.
2. If $\sup_{\theta \in \Theta_{0}} \beta (\theta) = \alpha$, then the given hypothesis test is called a Size $\alpha$ hypothesis test.
3. If $\sup_{\theta \in \Theta_{0}} \beta (\theta) \le \alpha$, then the given hypothesis test is called a Level $\alpha$ hypothesis test.

Explanation

Power?

In mathematics, power is usually discussed in terms of exponentiation pow or using the character 冪 to talk about power functions $f(x) = x^{-\alpha}$. However, in the context of statistics, power simply refers to the strength of a Hypothesis Testing.

The Power of a Test?

Since $\beta$ is defined through the probability $P_{\theta}$, its range is naturally a subset of $[0,1]$. A high value of $\beta (\theta)$ – meaning high test power, or the strength of the test – indicates the strength to reject the null hypothesis. One might wonder, isn’t rejecting the alternative hypothesis also a test? But notionally, when discussing any hypothesis test, it’s usually in reference to the null hypothesis, and since $R$ is the rejection region of the null, the focus should solely be on whether the null hypothesis is rejected or not.

We evaluate the goodness of a hypothesis test through the power function. If there is a better method of testing, we say it is More Powerful. This terminology suggests, unlike the general mathematical usage of “power”, that it indeed refers to “strength”. It’s a natural motivation in mathematical statistics to question whether a hypothesis test is reasonable or efficient.

However, one should not blindly consider the power of a test as the sole measure of its quality. Consider a test $\beta (\theta) = 1 = 100 \%$ that rejects the null hypothesis for any sample. While it may seem very powerful, its excessive strength fails to capture any Type I errors (rejecting a true null hypothesis).

Size and Level

The terms size and level are often used interchangeably, and their usage may vary. However, once defined as mentioned, it’s natural that the set of level $\alpha$ tests includes the set of size $\alpha$ tests. This distinction should be rigorously maintained in research topics that require careful consideration.

What does $\alpha$ mean? Whether it’s size or level, a high $\alpha$ indicates the presence of parameters that have a high probability of rejecting the null hypothesis when it is true. A larger $\alpha$ tends to reject the null hypothesis more liberally, and conversely, a smaller $\alpha$ would result in a more conservative test. Such differences arise from the rejection region. Meanwhile, terms like Level and the discussions here naturally bring the concept of Significance Level to mind, but ultimately they are separate considerations and should not be forcibly linked. It’s better to understand them conceptually.

Example: Normal Distribution

$$\begin{align*} H_{0} :& \theta \le \theta_{0} \\ H_{1} :& \theta > \theta_{0} \end{align*}$$ Considering the hypothesis test for a random sample $X_{1} , \cdots , X_{n}$ from a normal distribution $N \left( \theta , \sigma^{2} \right)$ with known variance, if the z-score is greater than a certain constant $c$, then the null hypothesis can be rejected. The power function $\beta$ can be determined by converting the probability $P$ of having $\displaystyle {{ \bar{X} - \theta_{0} } \over { \sigma / \sqrt{n} }}$ into an expression in terms of $\theta$. $$\begin{align*} \beta \left( \theta \right) =& P_{\theta} \left( {{ \bar{X} - \theta_{0} } \over { \sigma / \sqrt{n} }} > c \right) \\ =& P_{\theta} \left( {{ \bar{X} - \theta } \over { \sigma / \sqrt{n} }} > c + {{ \theta_{0} - \theta } \over { \sigma / \sqrt{n} }} \right) \\ =& P_{\theta} \left( Z > c + {{ \theta_{0} - \theta } \over { \sigma / \sqrt{n} }} \right) \end{align*}$$ Here, $\displaystyle Z := {{ \bar{X} - \theta_{0} } \over { \sigma / \sqrt{n} }}$ is a random variable that follows the standard normal distribution.