📂Mathematical Statistics

Confidence Intervals

Definition ¹

When a subset $C \subset \Theta$ of the parameter space $\Theta$ satisfies $P ( \theta \in C | y ) \ge 1 - \alpha$ for a significance level $\alpha$, $C$ is called the Credible Interval for $\theta$ given data $y$.

Explanation

Interval estimation in Bayesian statistics is about finding intervals that are highly probable to contain the parameter $\theta$. The ‘Credible Interval’ found in this manner corresponds to the concept of confidence interval for frequentists.

Understanding the Equation

The equation might seem a bit complicated at first, but let’s break it down. If represented in integral form, $$ P ( \theta \in C | y ) = \int_{ \theta \in C} p ( \theta | y) d \theta $$ For ease of understanding, if we simply set the significance level to $\alpha = 0.05$, then $$ \int_{ \theta \in C} p ( \theta | y) d \theta \ge 0.95 $$ defines $C$ as the Credible Interval. Rewriting this in a more familiar term as $C = [a,b] $, we get $$ \int_{a}^{b} p ( \theta | y) d \theta \ge 0.95 $$ If the area under the curve in the shaded parts of the two figures below is greater than or equal to $0.95$, then regardless of what it is, this integral interval $C$ becomes a Credible Interval.

However, the shorter the length of the Credible Interval, the more accurate it is, so among those satisfying the condition, the smallest one is preferable. Therefore, if one had to choose, the right side would be chosen, and in actual estimation, more accurate methods are used.

Conservative Definition

The reason why the Credible Interval is not precisely defined as $P ( \theta \in C | y ) = 1 - \alpha$ but as $P ( \theta \in C | y ) \ge 1 - \alpha$ is for safety. There might be cases where one cannot exactly match during calculation, so it’s better to err on the wider side than narrowing down and being wrong.

The more you look into this, the more you might wonder what exactly differs from the frequentists’ confidence intervals and why there’s a need for a new definition. But this subtle difference is precisely what makes Bayesian so appealing and is one of its key elements.