Mean and Variance of the Geometric Distribution
Formula
$X \sim \text{Geo} (p)$ Area $$ E(X) = {{ 1 } \over { p }} \\ \operatorname{Var}(X) = {{ 1-p } \over { p^{2} }} $$
Derivation
The mean and variance of the Geometric Distribution are not as easily calculated as one might think. This post introduces two interesting and useful proofs.
The discrete probability distribution that has the following probability mass function is called a Geometric Distribution, according to the definition of Geometric Distribution$p \in (0,1]$: $$ p(x) = p (1 - p)^{x-1} \qquad , x = 1 , 2, 3, \cdots $$
First Method
Strategy: Use the formula for a geometric series and differentiation.
Mean
$$ E(X)=\sum _{ x=1 }^{ \infty }{ xp { (1-p) }^{ x-1 } } $$ Let $\displaystyle f(p): =\sum _{ x=0 }^{ \infty }{ { (1-p) }^{ x } }$ be $$ f(p)=\frac { 1 }{ 1-(1-p) }=\frac { 1 }{ p } $$ Differentiating $p$, according to the formula for a geometric series, $$ f '(p)=-\frac { 1 }{ { p }^{ 2 } } $$ On the other hand, differentiating the geometric series directly gives $$ f ' (p)=\sum _{ x=1 }^{ \infty }{ {-x { (1-p) }^{ x-1 } } } $$ Therefore, $$ \begin{align*} & -\frac { 1 }{ { p }^{ 2 } }=-\sum _{ x=1 }^{ \infty }{ x { (1-p) }^{ x-1 } } \\ \implies& \frac { 1 }{ p }=p\sum _{ x=1 }^{ \infty }{ x { (1-p) }^{ x-1 } } \\ \implies& \frac { 1 }{ p }=\sum _{ x=1 }^{ \infty }{ xp { (1-p) }^{ x-1 } }=E(X) \end{align*} $$ Thus $\displaystyle E(X)=\frac { 1 }{ p }$
■
Variance
$$ V(X)=E({ X }^{ 2 })-{ {E(X)} }^{ 2 }=\sum _{ x=1 }^{ \infty }{ { x^ 2} { p { (1-p) }^{ x-1 } }-\frac { 1 }{ { p }^{ 2 } } } $$ Therefore, it suffices to find $\displaystyle E({ X }^{ 2 })=\sum _{ x=1 }^{ \infty }{ { x^2 }{ p { (1-p) }^{ x-1 } } }$.
Likewise, let $\displaystyle f(p) :=\sum _{ x=0 }^{ \infty }{ { (1-p) }^{ x } }$ be $$ f(p)=\frac { 1 }{ 1-(1-p) }=\frac { 1 }{ p } \\ f '(p) = - \frac { 1 }{ p^{2} } \\ f ''(p)=\frac { 2 }{ { p }^{ 3 } } $$ On the other hand, $\displaystyle f ''(p)=\sum _{ x=1 }^{ \infty }{ x(x-1) { (1-p) }^{ x-2 } }$ also serves, $$ \begin{align*} & \frac { 2 }{ { p }^{ 3 } }=\sum _{ x=1 }^{ \infty }{ x(x-1) { (1-p) }^{ x-2 } } \\ \implies& \frac { 2 }{ { p }^{ 3 } }=\sum _{ x=1 }^{ \infty }{ { x^2 } { { (1-p) }^{ x-2 } }-\sum _{ x=1 }^{ \infty }{ x { (1-p) }^{ x-2 } } } \\ \implies& p\frac { 2 }{ { p }^{ 3 } }=p\sum _{ x=1 }^{ \infty }{ { x^2 } { { (1-p) }^{ x-2 } }-p\sum _{ x=1 }^{ \infty }{ x { (1-p) }^{ x-2 } } } \\ \implies& \frac { 2 }{ { p }^{ 2 } }=\sum _{ x=1 }^{ \infty }{ { x^2 } { p { (1-p) }^{ x-2 } }-\sum _{ x=1 }^{ \infty }{ xp { (1-p) }^{ x-2 } } } \\ \implies& \frac { 2 }{ { p }^{ 2 } }=\frac { 1 }{ 1-p }\sum _{ x=1 }^{ \infty }{ { x^2 } { p { (1-p) }^{ x-1 } }-\frac { 1 }{ 1-p }\sum _{ x=1 }^{ \infty }{ xp { (1-p) }^{ x-1 } } } \\ \implies& \frac { 2(1-p) }{ { p }^{ 2 } }=E({ X }^{ 2 })-\frac { 1 }{ p } \\ \implies& E({ X }^{ 2 })=\frac { 2-p }{ { p }^{ 2 } } \end{align*} $$ Therefore, $\displaystyle V(X)=\frac { 1-p }{ { p }^{ 2 } }$
■
Second Method
Strategy: Use the forgetfulness of the geometric distribution. In a way, it feels like skipping complex formulas and explaining with words, but it might feel more difficult for some people.
Mean
$$ E(X)=1 \cdot P(\text{ 첫번째 시행에서 성공 })+E(Y+1)\cdot P( \text{첫번째 시행에서 실패}) $$ According to the definition of expectation, the expectation $E(X)$ is the sum of the product of the probability that the first trial is successful and the number of trials in that case, $1$, and the product of the probability that the first trial fails and the expectation in that scenario, $E(Y+1)$. Of course, the mentioned $Y$ follows the same rule as $X$, which in turn is based on $\text{Geo} (p)$. Whether the first trial is successful or not, the geometric distribution is forgetful, so it starts from the beginning, and $Y$ has an added correction of $1$ separately. If we state it cleanly again, it goes as follows. $$ E(X)=1\cdot p+E(Y+1)\cdot (1-p) $$ But $E(Y+1)$ can be represented by $E(Y+1)=E(Y)+E(1)=E(Y)+1$, and since $X \sim \text{Geo} (p)$ and $Y \sim \text{Geo} (p)$, $$ E(Y)=E(X) $$ Organizing $\displaystyle E(X)=p+{E(X)+1}(1-p)$ with respect to $E(X)$ gives: $$ E(X)=\frac { 1 }{ p } $$
■
Variance
$$ \begin{align*} E({ X }^{ 2 }) =& 1\cdot p+E({ (Y+1) }^{ 2 })\cdot (1-p) \\ &=p+{E({ X }^{ 2 })+2E(X)+1}(1-p) \\ &=p+E({ X }^{ 2 })+2E(X)+1-pE({ X }^{ 2 })-2pE(X)-p \end{align*} $$ To neatly arrange, $$ 0=2E(X)+1-pE({ X }^{ 2 })-2pE(X) $$ Binomially scaling the $2$th moment gives $$ \begin{align*} pE({ X }^{ 2 }) =& 2(1-p)E(X)+1 \\ &=2(1-p)\frac { 1 }{ p }+1 \\ =& \frac { 2-p }{ p } \end{align*} $$ Dividing both sides by $p$ gives $$ E({ X }^{ 2 })=\frac { 2-p }{ { p }^{ 2 } } $$ Therefore, $\displaystyle V(X)=\frac { 1-p }{ { p }^{ 2 } }$
■