📂Mathematical Statistics

Probability and the Addition Law of Probability in Mathematical Statistics

Definition ¹

1. An experiment that can be repeated under the same conditions is referred to as a Random Experiment.
2. The set $\Omega$ of all possible outcomes that can be obtained from a random experiment is called the Sample Space.
3. The set of outcomes in the sample space that we are interested in, i.e., $B \subset \Omega$ is called an Event, and these sets are represented as $\mathcal{B}$.
4. A function $P : \mathcal{B} \to \mathbb{R}$ that satisfies the following three conditions is called Probability:
   • (i): For all $B \in \mathcal{B}$, $P(B) \ge 0$
   • (ii): For the entire space $\Omega \in \mathcal{B}$, $P(\Omega) = 1$
   • (iii) Additive Law of Probability: For a sequence of mutually exclusive events $\left\{ B_{i} \right\}_{i=1}^{\infty}$, i.e., $n \ne m \implies B_{n} \cap B_{m} = \emptyset$, for $\left\{ B_{i} \right\}$ $$P \left( \bigcup_{i=1}^{\infty} B_{i} \right) = \sum_{i=1}^{\infty} P \left( B_{i} \right)$$

Explanation

Even though it is called mathematical statistics, the basic concepts remain the same as those used in the curriculum’s probability and undergraduate-level probability theory. Regardless of the theoretical basis, the concepts cannot change even if expressions and reasoning may vary. Don’t be overwhelmed by sets and functions, and take your time to read through the explanations:

Events and Sample Space

One difference from high school-level probability and statistics is the more active use of sets to describe the concept of probability. Though the concepts of probability at the undergraduate level in mathematical statistics might still retain vague expressions such as ‘random experiment’ or ‘interested in,’ these might seem strict and difficult at first glance. It’s normal, so don’t worry.

Assuming human height follows a normal distribution, the sample space $\Omega$ becomes the set of real numbers $\mathbb{R}$ itself. Although height must always be positive, let’s put aside such unnecessary strictness for now. Then, an event $B$ would be represented by a set that includes the height $x$ of a man named Adam when measured. For example, $[172,190] \subset \Omega$ would be the event that the measured height is at least 172 and at most 190. This measurement is the random experiment described in the definition, and the measured value $x$ is an outcome, with all possible outcomes being compiled into the sample space. Even if you can’t grasp this abstraction, it might not significantly impede your study of mathematical statistics. However, be prepared that this may weaken your foundation.

The next step in abstraction is formalization. Event $B \subset \Omega$ belongs to the power set $\mathscr{P}(\Omega)$ of $\Omega$. Let’s check a few relationships for the collection of these $\mathcal{B}$. $$B \subset \Omega \\ \mathcal{B} \not\subset \Omega \\ B \in \mathscr{P}(\Omega) \\ B \in \mathcal{B} \\ B \notin \Omega \\ \mathcal{B} \subset \mathscr{P}(\Omega)$$

Probability

The reason for using such complex expressions is because the domain of the probability (function) $P$ has to be the events rather than the sample space $\Omega$ itself. From a high school level perspective, it might be considered irrelevant what the exact probability of Adam’s height being 181 ($x=181$) is, and it should instead calculate probabilities like being taller than 180 but shorter than 182 ($180<x<182$) as $\displaystyle \int_{180}^{182}f(x) dx > 0$. Probability is a function that quantifies the likelihood of an event from $0$ to $1$.

For the entire space, i.e., $\Omega$, $P(\Omega)=1$ means, intuitively, ’the probability of anything happening is 100%.’ Formally, it can be explained as “nothing can be more certain than what is bound to occur.”

Exclusive Events

An event $A \subset \Omega$ that satisfies the following is called an Exclusive Event of $B$. $$P \left( B \cap A \right) = 0$$ Obvious examples of exclusive events include $\emptyset$ or $B^{C}$, but it’s important to remember that the definition doesn’t exactly say $B \cap A = \emptyset$. Exclusive events are defined probabilistically, and it’s not concerned with how these sets actually look like as concrete sets.

Rigorous Definition

• Probability rigorously defined through measure theory