📂Mathematical Statistics

Hypothesis Testing via Bayes Factor

Buildup

To be able to use classical hypothesis testing, one must acquire a mathematical understanding of concepts such as the rejection region and the significance probability, along with enough statistical sense to accept them intuitively. It is only natural that, even at the freshman general-education level, several hours are devoted to teaching this, and yet plenty of students still fail to properly grasp hypothesis testing. This is similar to how, in the statistics taught in high school, the problems may be easy to solve but not many students understand its true meaning.

Hypothesis Testing ¹

In contrast, Bayesian statistics allows hypothesis testing to be done very easily through something called the Bayes factor.

Suppose the null hypothesis and the alternative hypothesis are given as $H_{0}$ vs $H_{1}$.

1. Let $\pi_{0}, \pi_{1}$ be the prior information for the null hypothesis and the alternative hypothesis, respectively.
2. Let $\alpha_{0}, \alpha_{1}$ be the posterior information for the null hypothesis and the alternative hypothesis, respectively.
3. Call $\displaystyle B_{01} := {{ \alpha_{0 } / \alpha_{1} } \over { \pi_{0 } / \pi_{1} }} = {{ \alpha_{0 } / \pi_{0} } \over { \alpha_{1 } / \pi_{1} }}$ the Bayes factor supporting $H_{0}$.

Looking closely at the Bayes factor here, in $$ B_{01} = {{ \displaystyle {{ \alpha_{0} } \over { \cdot }} } \over { \displaystyle {{ \cdot } \over { \pi_{1} }} }} $$ it does not matter that $\alpha_{1}$ and $\pi_{0}$ can go freely into either of the $\cdot$ positions. Therefore, there is no need to memorize the formula in a complicated way; one only needs to keep in mind that $\alpha_{0}$ goes at the very top and $\pi_{1}$ goes at the very bottom.

In Bayesian analysis, hypothesis testing is entirely a matter of supporting the null hypothesis if $B_{01}$ is greater than $1$, and supporting the alternative hypothesis if it is less than $1$. In particular, thinking of it as $$ B_{01} = {{ \alpha_{0 } / \pi_{0} } \over { \alpha_{1 } / \pi_{1} }} = {{ \text{귀무} } \over { \text{대립} }} $$ makes it much simpler to understand. Put simply, if you actually compute with the data and the probability of the null hypothesis turns out high, then you support the null hypothesis. There is no need to think about rejection regions or significance probabilities.

If $B_{01} = 3$, it means that the posterior information supports $H_{0}$ to a degree $3$ times the degree to which it supports $H_{1}$.

Jeffreys’s Interpretation

For this degree of supporting the null hypothesis, Jeffreys proposed the following interpretation. From the standpoint of supporting $H_{0}$, the value of the Bayes factor is interpreted as follows.

• $1 \le B_{01} \le 3$: weak evidence
• $3 < B_{01} \le 12$: positive evidence
• $12 < B_{01} \le 150$: strong evidence
• $150 \le B_{01}$: very strong evidence

The advantage of this interpretation is that it is far more flexible than the extreme dichotomy of frequentist hypothesis testing, namely ‘whether the significance probability exceeds the significance level or not.’ Anyone who frequently uses regression analysis has probably had the experience of wanting to set the significance level at $\alpha = 0.05$, but having to discard a regression coefficient because the significance probability came out as something like $p = 0.069925$. Frankly, since the analyst is human too, one cannot help but feel frustrated when this happens. So one searches high and low for a solution, but most of the time it comes to nothing.

By comparison, Bayesian hypothesis testing simply accepts the data as it is—sufficient when it is sufficient, and insufficient when it is insufficient.

Example

When $Y \sim B (10, \theta )$, we wish to perform a Bayes test on $\displaystyle H_{0} : \theta = {{1} \over {2}}$ vs $\displaystyle H_{1} : \theta \ne {{1} \over {2}}$. The prior probabilities of $H_{0}$ and $H_{1}$ are equal, under $H_{1}$ we have $\theta \sim \text{Beta} (1,1)$, and the observed value is $Y=7$. Find the Bayes factor $B_{01}$.

Solution

$$ \begin{align*} B_{01} =& {{ \alpha_{0 } / \pi_{0} } \over { \alpha_{1 } / \pi_{1} }} = {{ p ( y \mid \theta_{0} ) } \over { \int_{\Theta_{1}} p ( y \mid \theta ) g ( \theta ) d \theta }} = {{ p ( Y = 7 \mid \theta = {{1} \over {2}} ) } \over { \int_{\Theta_{1}} p ( y \mid \theta ) d \theta }} \\ =& {{ \binom{10}{7} \left( {{1} \over {2}} \right)^{7} \left( 1- {{1} \over {2}} \right)^{3} } \over { \int_{0}^{1} \binom{10}{7} \theta^{7} \left( 1 - \theta \right)^{3} d \theta }} = {{1} \over {2^{10}}} {{1} \over { \int_{0}^{1} \theta^{8-1} (1 - \theta)^{4-1} d \theta }} = {{1} \over {2^{10}}} {{ \Gamma ( 8 + 4 ) } \over { \Gamma ( 8 ) \Gamma ( 4 ) }} \\ =& {{1} \over {2^{10}}} {{ 11! } \over { 7! \cdot 3! }} = {{1} \over {2^{10}}} {{ 8 \cdot 9 \cdot 10 \cdot 11 } \over { 2 \cdot 3 }} = {{ 2^4 \cdot 3^2 \cdot 5 \cdot 11 } \over { 2^{11} \cdot 3 }} = {{ 165 } \over { 2^{7} }} = 1.2890625 \end{align*} $$ Therefore, $B_{01}$ constitutes weak evidence supporting the null hypothesis.