📂Probability Distribution

Binomial Distribution

Definition ¹

The discrete probability distribution $\text{Bin}(n,p)$ with the following probability mass function for $n \in \mathbb{N}$ and $p \in [0,1]$ is called the Binomial Distribution. $$ p(x) = \binom{n}{x} p^{x} (1-p)^{n-x} \qquad , x = 0 , 1, \cdots n $$

Basic Properties

Moment Generating Function

• [1]: $$m(t) = \left[ (1-p) + pe^{t} \right]^{n} \qquad , t \in \mathbb{R}$$

Mean and Variance

• [2]: If $X \sim \text{Bin}(n,p)$ then $$ \begin{align*} E(X) =& np \\ \text{Var}(X) =& np(1-p) \end{align*} $$

Theorems

Deriving the Poisson Distribution as a Limiting Distribution of the Binomial Distribution

• [a]: Let $X_{n} \sim B(n,p)$. If $\mu \approx np$ then $$ X_{n} \overset{D}{\to} \text{Poi} (\mu) $$

Deriving the Standard Normal Distribution as a Limiting Distribution of the Binomial Distribution

• [b]: If $X_i \sim B(1,p)$ and $Y_n = X_1 + X_2 + \cdots + X_n$ then $Y_n \sim B(n,p)$ and $$ { { Y_n - np } \over {\sqrt{ np(1-p) } } }\overset{D}{\to} N(0,1) $$

Explanation

Bernoulli Distribution

The Binomial Distribution originates from the Bernoulli Trial, which is the simplest form of probability experiment most humans can think of. A Bernoulli trial has only two possible outcomes, success with probability $0 \le p \le 1$ or failure, and generalizing this to $n$ times yields the Binomial Distribution. Conversely, the Bernoulli Distribution is a special case of the Binomial Distribution when $n=1$.

Multinomial Distribution

Furthermore, generalizing from a binary outcome, success or failure, to $k$ possible outcomes yields the Multivariate Distribution $M (n; p_{1} , \cdots , p_{k})$ known as the Multinomial Distribution. Its probability mass function is given as follows: $$ p(x_{1} , \cdots , x_{k}) = {{ n! } \over { x_{1} ! \cdots x_{k}! }} p_{1}^{x_{1}} \cdots p_{k}^{x_{k}} $$

Proof

[1]

$$ \begin{align*} M(t) =& \sum_{x=0}^{n} e^{tx} p(x) \\ =& \sum_{x=0}^{n} e^{tx} \binom{n}{x} p^{x} (1-p)^{n-x} \\ =& \sum_{x=0}^{n} \binom{n}{x} \left( pe^{t} \right)^{x} (1-p)^{n-x} \end{align*} $$ According to the Binomial Theorem $$ \sum_{x=0}^{n} \binom{n}{x} \left( pe^{t} \right)^{x} (1-p)^{n-x} = \left[ pe^{t} + (1-p) \right]^{n} $$

■

[2]

Strategy: Although it can be derived using algebraic tricks similar to those in the curriculum, let’s use the theory of mathematical statistics to derive it more easily since we’ve also derived the moment generating function.

---

The derivative of $M$ is $$ M ' (t) = n \left[ (1-p) + pe^{t} \right]^{n-1} \left( pe^{t} \right) $$ Since by the definition of the moment generating function $ E(X) = M ' (0):$ is given, $$ \mu := E(X) = M ' (0) = np $$ The second derivative of $M$ is $$ M '' (t) = n \left[ (1-p) + pe^{t} \right]^{n-1} \left( pe^{t} \right) + n(n-1) \left[ (1-p) + pe^{t} \right]^{n-2} \left( pe^{t} \right)^{2} $$ Since $M '' (0) = np + n(n-1)p^{2}$, $$ \begin{align*} \text{Var}(X) =& E \left( X^{2} \right) - \mu^{2} \\ =& M '' (0) - (np)^{2} \\ =& np + n(n-1)p^{2} - n^{2}p^{2} \\ =& np(1-p) \end{align*} $$

■

[a]

Approximated by the moment generating function.

■

[b]

Approximated similarly to the Central Limit Theorem.

■

Code

Following is Julia code displaying the probability mass function of the binomial distribution as a GIF.

@time using LaTeXStrings
@time using Distributions
@time using Plots

cd(@__DIR__)

x = 0:20
P = collect(0.0:0.01:1.0); append!(P, reverse(P))

animation = @animate for p ∈ P
    scatter(x, pdf.(Binomial(10, p), x),
     color = :black, markerstrokecolor = :black,
     label = "n = 10, p = $(rpad(p, 4, '0'))", size = (400,300))
    xlims!(0,20); ylims!(0,0.5); title!(L"\mathrm{pmf\,of\,Bin}(10, p)")
end
gif(animation, "pmf10.gif")

animation = @animate for p ∈ P
    scatter(x, pdf.(Binomial(20, p), x),
     color = :black, markerstrokecolor = :black,
     label = "n = 20, p = $(rpad(p, 4, '0'))", size = (400,300))
    xlims!(0,20); ylims!(0,0.5); title!(L"\mathrm{pmf\,of\,Bin}(20, p)")
end
gif(animation, "pmf20.gif")