Relationship Between the Exponential Distribution and the Poisson Distribution
Theorem
For the time $X_{k}$ taken for an event to occur, if $X_{k} \sim \exp (\lambda)$, then for the number of events $N$ occurring per unit time, $\displaystyle N \sim \text{Poi} (\lambda)$
Explanation
Let us consider the intuitive definitions of the exponential distribution and the Poisson distribution. The exponential distribution is concerned with the time it takes until some event occurs, and the Poisson distribution is concerned with how many times some event occurs within a unit of time. Regarding the time at which an event occurs and the number of times an event occurs, the two distributions fix one side and take interest in the other. For example, suppose the parameter of $\exp (\lambda)$ and $\text{Poi}(\lambda)$ is $\lambda = 1$. Looking at the exponential distribution, it takes a unit of time until an event occurs, and looking at the Poisson distribution, one event occurs per unit time.
Here, if the $\lambda$ of the Poisson distribution grows larger, the number of events occurring per unit time increases, and accordingly the time it takes for one event to occur becomes shorter. In this sense, it can be seen as reasonable that the mean of the exponential distribution $\displaystyle {{1} \over {\lambda}}$ and the mean of the Poisson distribution $\lambda$ share the symbol $\lambda$ denoting the parameter. Many textbooks denote the parameters of both by $\lambda$ of all symbols, and thinking of the reason this way makes it easy to accept.
Mathematically, the gamma distribution and the exponential distribution have a relationship, and the gamma distribution has a relationship with the Poisson distribution, so it is not difficult to surmise that the exponential distribution and the Poisson distribution also have some relationship.
Proof
By the relationship between the gamma distribution and the exponential distribution, $$ X_{i} \sim \exp (\lambda) \iff X_{i} \sim \Gamma (1, {{1} \over {\lambda}} ) $$ Summing all $k$ random variables following the gamma distribution, $$ Y_{k} = \sum_{i=1}^{k} X_{i} \sim \Gamma (n, {{1} \over {\lambda}} ) $$ Since the exponential distribution has memorylessness, $Y_{i}$ and $Y_{j}$ are independent, and it can be seen that $Y_{k}$ simply represents the time at which the $k$-th event occurs. Meanwhile, if the cumulative distribution function of $\displaystyle Y_{k}$ is $F_{k}$, then $$ F_{k}(1) = 1 - \int_{1}^{\infty} { {1} \over {\Gamma (k) {{1} \over {\lambda ^ k}} }} x^{k-1} e^{-\lambda x} dx $$ Rearranging, $$ F_{k}(1) = 1 - \int_{1}^{\infty} { { \lambda^{k} } \over {\Gamma (k) }} x^{k-1} e^{-\lambda x} dx $$ Substituting $\lambda x = z$, since $\lambda dx = dz$, $$ F_{k}(1) = 1 - \int_{\lambda}^{\infty} { { z^{k-1} e^{- z } } \over {\Gamma (k) }} dz $$ By the relationship between the gamma distribution and the Poisson distribution, $$ F_{k}(1) = 1 - \int_{\lambda}^{\infty} { { z^{k-1} e^{-z} } \over { \Gamma (k) } } dz = 1 - \sum_{y=0}^{k-1} { { {\lambda}^{y} e^{-\lambda} } \over {y!} } $$ Since $Y_{k}$ is the time at which the $k$-th event occurs, the probability that an event occurs exactly $n$ times during a unit time of $1$ equals the probability that $Y_{n}$ is less than or equal to $1$ and $Y_{n+1}$ is greater than $1$. $$ \begin{align*} P(N = n) =& P( Y_{n} \le 1 \land Y_{n+1}>1 ) \\ =& P( Y_{n} \le 1 ) P ( Y_{n+1}>1 ) \\ =& P( Y_{n} \le 1 ) \left( 1 - P ( Y_{n+1} \le 1 ) \right) \\ =& P( Y_{n} \le 1 ) - P( Y_{n} \le 1 ) P ( Y_{n+1} \le 1 ) \\ =& P( Y_{n} \le 1 ) - P( Y_{n} \le 1 \land Y_{n+1} \le 1 ) \\ =& P( Y_{n} \le 1 ) - P( Y_{n+1} \le 1 ) \\ =& F_{n}(1) - F_{n+1}(1) \\ =& \left( 1 - \sum_{y=0}^{n-1} { { {\lambda}^{y} e^{-\lambda} } \over {y!} } \right) - \left( 1 - \sum_{y=0}^{n} { { {\lambda}^{y} e^{-\lambda} } \over {y!} } \right) \\ =& { { {\lambda}^{n} e^{-\lambda} } \over {n!} } \end{align*} $$ This is the probability mass function of the Poisson distribution with parameter $\lambda$, so $N \sim \text{Poi} (\lambda)$
■