📂Mathematical Statistics

Mathematical Definition of Statistical Significance

Definition ¹

Let’s assume that there is a given hypothesis test $H_{0} \text{ vs } H_{1}$. For all realizations $\mathbf{x} \in \Omega$, a test statistic $p \left( \mathbf{X} \right)$ that satisfies $0 \le p \left( \mathbf{x} \right) \le 1$ is called the significance probability or p-value. If $p \left( \mathbf{X} \right)$ satisfies the following for all $\theta \in \Theta_{0}$ and all $\alpha \in [0,1]$, it is said to be valid. $$ P_{\theta} \left( p \left( \mathbf{X} \right) \le \alpha \right) \le \alpha $$

Description

Thinking about the condition for a valid p-value expressed in the equations, it means that the probability of $p \left( \mathbf{X} \right) \le \alpha$ under the null hypothesis is small. In other words, if $p \left( \mathbf{X} \right)$ is small, it provides the basis for rejecting $H_{0}$. In this sense, the condition’s inequality can be confirmed as valid for ’effective significance probability'.

Although we will see the significance probability countless times as long as we study statistics, we will not add an obvious explanation. However, a noteworthy point in terms of mathematical statistics is that the significance probability is also a statistic and thus, a random variable. The validity does not consider the intuitive meaning of ‘reject the null hypothesis if it is small’, as long as the range of the significance probability belongs to $[0,1]$. Although there is no reason to use an invalid significance probability, the ability to define the p-value so clearly and concisely with equations is crucial.