Binomial Distribution
Definition 1
The discrete probability distribution $\text{Bin}(n,p)$ with the following probability mass function for $n \in \mathbb{N}$ and $p \in [0,1]$ is called the Binomial Distribution. $$ p(x) = \binom{n}{x} p^{x} (1-p)^{n-x} \qquad , x = 0 , 1, \cdots n $$
Basic Properties
Moment Generating Function
- [1]: $$m(t) = \left[ (1-p) + pe^{t} \right]^{n} \qquad , t \in \mathbb{R}$$
Mean and Variance
- [2]: If $X \sim \text{Bin}(n,p)$ then $$ \begin{align*} E(X) =& np \\ \operatorname{Var}(X) =& np(1-p) \end{align*} $$
Theorems
Deriving the Poisson Distribution as a Limiting Distribution of the Binomial Distribution
- [a]: Let $X_{n} \sim B(n,p)$. If $\mu \approx np$ then $$ X_{n} \overset{D}{\to} \text{Poi} (\mu) $$
Deriving the Standard Normal Distribution as a Limiting Distribution of the Binomial Distribution
- [b]: If $X_i \sim B(1,p)$ and $Y_n = X_1 + X_2 + \cdots + X_n$ then $Y_n \sim B(n,p)$ and $$ { { Y_n - np } \over {\sqrt{ np(1-p) } } }\overset{D}{\to} N(0,1) $$
Explanation
Bernoulli Distribution
The Binomial Distribution originates from the Bernoulli Trial, which is the simplest form of probability experiment most humans can think of. A Bernoulli trial has only two possible outcomes, success with probability $0 \le p \le 1$ or failure, and generalizing this to $n$ times yields the Binomial Distribution. Conversely, the Bernoulli Distribution is a special case of the Binomial Distribution when $n=1$.
Multinomial Distribution
Furthermore, generalizing from a binary outcome, success or failure, to $k$ possible outcomes yields the Multivariate Distribution $M (n; p_{1} , \cdots , p_{k})$ known as the Multinomial Distribution. Its probability mass function is given as follows: $$ p(x_{1} , \cdots , x_{k}) = {{ n! } \over { x_{1} ! \cdots x_{k}! }} p_{1}^{x_{1}} \cdots p_{k}^{x_{k}} $$
Proof
[1]
$$ \begin{align*} M(t) =& \sum_{x=0}^{n} e^{tx} p(x) \\ =& \sum_{x=0}^{n} e^{tx} \binom{n}{x} p^{x} (1-p)^{n-x} \\ =& \sum_{x=0}^{n} \binom{n}{x} \left( pe^{t} \right)^{x} (1-p)^{n-x} \end{align*} $$ According to the Binomial Theorem $$ \sum_{x=0}^{n} \binom{n}{x} \left( pe^{t} \right)^{x} (1-p)^{n-x} = \left[ pe^{t} + (1-p) \right]^{n} $$
■
[2]
Strategy: Although it can be derived using algebraic tricks similar to those in the curriculum, let’s use the theory of mathematical statistics to derive it more easily since we’ve also derived the moment generating function.
The derivative of $M$ is $$ M ' (t) = n \left[ (1-p) + pe^{t} \right]^{n-1} \left( pe^{t} \right) $$ Since by the definition of the moment generating function $ E(X) = M ' (0):$ is given, $$ \mu := E(X) = M ' (0) = np $$ The second derivative of $M$ is $$ M '' (t) = n \left[ (1-p) + pe^{t} \right]^{n-1} \left( pe^{t} \right) + n(n-1) \left[ (1-p) + pe^{t} \right]^{n-2} \left( pe^{t} \right)^{2} $$ Since $M '' (0) = np + n(n-1)p^{2}$, $$ \begin{align*} \operatorname{Var}(X) =& E \left( X^{2} \right) - \mu^{2} \\ =& M '' (0) - (np)^{2} \\ =& np + n(n-1)p^{2} - n^{2}p^{2} \\ =& np(1-p) \end{align*} $$
■
[a]
Approximated by the moment generating function.
■
[b]
Approximated similarly to the Central Limit Theorem.
■
Code
Following is Julia code displaying the probability mass function of the binomial distribution as a GIF.
@time using LaTeXStrings
@time using Distributions
@time using Plots
cd(@__DIR__)
x = 0:20
P = collect(0.0:0.01:1.0); append!(P, reverse(P))
animation = @animate for p ∈ P
scatter(x, pdf.(Binomial(10, p), x),
color = :black, markerstrokecolor = :black,
label = "n = 10, p = $(rpad(p, 4, '0'))", size = (400,300))
xlims!(0,20); ylims!(0,0.5); title!(L"\mathrm{pmf\,of\,Bin}(10, p)")
end
gif(animation, "pmf10.gif")
animation = @animate for p ∈ P
scatter(x, pdf.(Binomial(20, p), x),
color = :black, markerstrokecolor = :black,
label = "n = 20, p = $(rpad(p, 4, '0'))", size = (400,300))
xlims!(0,20); ylims!(0,0.5); title!(L"\mathrm{pmf\,of\,Bin}(20, p)")
end
gif(animation, "pmf20.gif")
Hogg et al. (2013). Introduction to Mathematical Statistics(7th Edition): p142. ↩︎