Definition of a Mathematical-Statistical Confidence Set 📂Mathematical Statistics

Definition of a Mathematical-Statistical Confidence Set

Definition ¹

The following is referred to as the coverage probability for the interval estimator $\left[ L \left( \mathbf{X} \right), U \left( \mathbf{X} \right) \right]$ of parameter $\theta$. $$ P_{\theta} \left( \theta \in \left[ L \left( \mathbf{X} \right), U \left( \mathbf{X} \right) \right] \right) = P \left( \theta \in \left[ L \left( \mathbf{X} \right), U \left( \mathbf{X} \right) \right] | \theta \right) $$
The infimum of the coverage probability is called the confidence coefficient. $$ \inf_{\theta} P_{\theta} \left( \theta \in \left[ L \left( \mathbf{X} \right), U \left( \mathbf{X} \right) \right] \right) $$

Explanation

Confidence Interval

The confidence coefficient is the same as the confidence level, and when the interval estimator and confidence level appear together, we call that interval a confidence interval.

Cutting out all the unnecessary talk, speaking purely mathematically, from the definition of the interval estimator, the interval estimator is indeed a random interval made from a statistic. Seeing this clean statistical definition might now give you a sense of what a confidence interval is. When explaining the difference from Bayesian credible intervals, if you make $N$ confidence intervals, yada yada, there’s no distribution of parameters so yada yada gets condensed into the expressions below. $$ P \left( \theta \in \left[ L \left( \mathbf{X} \right), U \left( \mathbf{X} \right) \right] | \theta \right) $$ What is moving, changing, random, it was always the confidence interval itself, not $\theta$. As shown in the formula, $\theta$ is a constant, just given and stationary, and it’s the fluctuation around it. We don’t know nor need to know the distribution of $\theta$ since it’s essentially a constant anyway.

Why Was It Confusing

The inability to think this way comes from most of you seeing confidence intervals specifically written as numbers. Suppose the average is $3.14$ and at a confidence level of $95 \%$, the confidence interval is $[3.00, 3.28]$.

Only a madman seeing this for the first time would think the confidence interval moves. Normal human intuition goes, “So, that means there’s a chance that $3.14$ is within $[3.00, 3.28]$, right? But there’s also a chance it might go outside with $5\%$ probability? So even if we believe it, we shouldn’t fully trust it but rather to about 95%?”
It’s unthinkable that it moves to $3.14$ becoming $3.30$ and thus going outside, rather than imagining the confidence interval being drawn as $[2.00, 2.28]$ and not covering $3.14$. $3.00 = L \left( \mathbf{x} \right)$, and $3.28 = U \left( \mathbf{x} \right)$ it is.

Generalization to Confidence Sets

In defining the confidence interval through the coverage probability, there was no need at all for the characteristics of the interval estimator. For example, assumptions such as topological connectedness were not necessary, leading to the generalization to sets themselves, called confidence sets. Naturally, a confidence set is a subset of the parameter space $\Theta$, expressed as a random set $C \left( \mathbf{X} \right)$ dependent on the sample $\mathbf{X}$.