📂Mathematical Statistics

Hypothesis Testing Through Bayesian Factors

Buildup

To be able to use classical hypothesis testing, one must have a mathematical understanding of concepts such as rejection region, p-value, and even a statistical sense intuitive enough to understand them. It is no surprise that many students, even at the freshman college level, spend hours being taught and still fail to properly understand hypothesis testing. It is similar to how many students learn statistics in high school, find problem-solving easy, yet do not grasp its true meaning.

Hypothesis Testing ¹

On the other hand, Bayesian statistics allows for very easy hypothesis testing through something called the Bayes Factor.

Let’s assume null hypothesis and alternative hypothesis are given as $H_{0}$ vs $H_{1}$.

1. $\pi_{0}, \pi_{1}$ is called prior information for each null hypothesis and alternative hypothesis respectively.
2. $\alpha_{0}, \alpha_{1}$ is called posterior information for each null hypothesis and alternative hypothesis respectively.
3. $\displaystyle B_{01} := {{ \alpha_{0 } / \alpha_{1} } \over { \pi_{0 } / \pi_{1} }} = {{ \alpha_{0 } / \pi_{0} } \over { \alpha_{1 } / \pi_{1} }}$ is called the Bayes factor supporting $H_{0}$.

Looking closely at the Bayes factor $$B_{01} = {{ \displaystyle {{ \alpha_{0} } \over { \cdot }} } \over { \displaystyle {{ \cdot } \over { \pi_{1} }} }}$$ in $\cdot$, $\alpha_{1}$ and $\pi_{0}$ can freely enter each position. Therefore, there is no need to memorize the formula in a complicated way, just remember that $\alpha_{0}$ is at the very top and $\pi_{1}$ is at the very bottom.

In Bayesian analysis, hypothesis testing is simple: if $B_{01}$ is greater than $1$, it supports the null hypothesis; if smaller, it supports the alternative hypothesis. In particular, $$B_{01} = {{ \alpha_{0 } / \pi_{0} } \over { \alpha_{1 } / \pi_{1} }} = {{ \text{귀무} } \over { \text{대립} }}$$ thinking of it this way makes understanding much simpler. In simple terms, if the probability of the null hypothesis is higher when actually calculated with the data, it supports the null hypothesis. There’s no need to think about rejection regions or p-values.

If it said $B_{01} = 3$, it means the posterior information supports $H_{0}$ to the extent that it is $3$ times more than it supports $H_{1}$.

Jeffrey’s Interpretation

Regarding the extent to which the null hypothesis is supported, Jeffrey proposed the following interpretation. From the perspective of supporting $H_{0}$, the Bayes factor is interpreted as follows:

• $1 \le B_{01} \le 3$: Weak evidence
• $3 < B_{01} \le 12$: Positive evidence
• $12 < B_{01} \le 150$: Strong evidence
• $150 \le B_{01}$: Very strong evidence

The advantage of this interpretation is much more flexible compared to the extreme dichotomy of whether ’the p-value exceeds the significance level or not’ of frequentist hypothesis testing. If you frequently use regression analysis, you might have wanted to set the significance level to $\alpha = 0.05$, but the p-value turned out to be $p = 0.069925$, leading you to discard the regression coefficient. Honestly, as analysts are human, experiencing this can only be frustrating. Hence, one looks for solutions in every possible way, but most end up futile.

In contrast, Bayesian hypothesis testing simply accepts the data as is, whether sufficient or not.

Example

When $Y \sim B (10, \theta )$, to conduct a Bayesian test against $\displaystyle H_{0} : \theta = {{1} \over {2}}$ vs $\displaystyle H_{1} : \theta \ne {{1} \over {2}}$. The prior probabilities of $H_{0}$ and $H_{1}$ are the same, under $H_{1}$ $\theta \sim \text{Beta} (1,1)$ and the observation is $Y=7$. Calculate the Bayes factor $B_{01}$.

Solution

$$\begin{align*} B_{01} =& {{ \alpha_{0 } / \pi_{0} } \over { \alpha_{1 } / \pi_{1} }} = {{ p ( y \mid \theta_{0} ) } \over { \int_{\Theta_{1}} p ( y \mid \theta ) g ( \theta ) d \theta }} = {{ p ( Y = 7 \mid \theta = {{1} \over {2}} ) } \over { \int_{\Theta_{1}} p ( y \mid \theta ) d \theta }} \\ =& {{ \binom{10}{7} \left( {{1} \over {2}} \right)^{7} \left( 1- {{1} \over {2}} \right)^{3} } \over { \int_{0}^{1} \binom{10}{7} \theta^{7} \left( 1 - \theta \right)^{3} d \theta }} = {{1} \over {2^{10}}} {{1} \over { \int_{0}^{1} \theta^{8-1} (1 - \theta)^{4-1} d \theta }} = {{1} \over {2^{10}}} {{ \Gamma ( 8 + 4 ) } \over { \Gamma ( 8 ) \Gamma ( 4 ) }} \\ =& {{1} \over {2^{10}}} {{ 11! } \over { 7! \cdot 3! }} = {{1} \over {2^{10}}} {{ 8 \cdot 9 \cdot 10 \cdot 11 } \over { 2 \cdot 3 }} = {{ 2^4 \cdot 3^2 \cdot 5 \cdot 11 } \over { 2^{11} \cdot 3 }} = {{ 165 } \over { 2^{7} }} = 1.2890625 \end{align*}$$ Therefore, $B_{01}$ is weak evidence supporting the null hypothesis.