Probability Distribution

Elementary

In the realm of Mathematical Statistics, an unavoidable topic for statisticians is Probability Distributions, which is covered here.

Basic Distributions

• Uniform Distribution $U(a,b)$
  • Mean and Variance of Uniform Distribution
  • Sufficient Statistic and MLE for Uniform Distribution
• Binomial Distribution $\text{Bin} (n,p)$
  • Mean and Variance of Binomial Distribution
  • Derivation of Poisson Distribution as a Limit of Binomial Distribution
  • Derivation of Standard Normal Distribution as a Limit of Binomial Distribution
  • Variance Stabilization for Normal Approximation of Binomial Distribution
• Geometric Distribution $\text{Geo} (p)$
  • Two Definitions of Geometric Distribution and Their Differences
  • Mean and Variance of Geometric Distribution
  • Sufficient Statistic and MLE for Geometric Distribution
  • Memorylessness of Geometric Distribution
• Negative Binomial Distribution $\text{NB} (r,p)$
  • Mean and Variance of Negative Binomial Distribution
• Poisson Distribution $\text{Poi} ( \lambda )$
  • Mean and Variance of Poisson Distribution
  • Sufficient Statistic and MLE for Poisson Distribution
  • Derivation of Standard Normal Distribution as a Limit of Poisson Distribution
• Exponential Distribution $\exp ( \lambda)$
  • Mean and Variance of Exponential Distribution
  • Sufficient Statistic and MLE for Exponential Distribution
  • Memorylessness of Exponential Distribution
  • Relationship between Exponential and Poisson Distributions
• Normal Distribution $N \left( \mu,\sigma^{2} \right)$
  • Mean and Variance of Normal Distribution
  • Sufficient Statistic and MLE for Normal Distribution
  • Square of Standard Normal Distribution follows Chi-Square Distribution with 1 Degree of Freedom
  • Entropy of Normal Distribution
  • Expectation of the Power of Normally Distributed Random Variables with Mean Zero

Sampling Distributions

• Gamma Distribution $\Gamma (k,\theta)$
  • Mean and Variance of Gamma Distribution
  • Sufficient Statistic for Gamma Distribution
  • Relationship between Gamma and Poisson Distributions
  • Relationship between Gamma and Exponential Distributions
  • Relationship between Gamma and Chi-Square Distributions
• Beta Distribution $\text{Beta} (\alpha,\beta)$
  • Mean and Variance of Beta Distribution
  • Sufficient Statistic for Beta Distribution
  • Derivation of Beta Distribution from Two Independent Gamma Distributions
  • Derivation of Beta Distribution from F-Distribution
• Chi-Square Distribution $\chi^{2}(r)$
  • Mean and Variance of Chi-Square Distribution
  • Sufficient Statistic for Chi-Square Distribution
• F-Distribution $F \left( r_{1} , r_{2} \right)$
  • Mean and Variance of F-Distribution
  • Derivation of F-Distribution from Two Independent Chi-Square Distributions
  • Derivation of F-Distribution from t-Distribution
• t-Distribution $t \left( \nu \right)$
  • Mean and Variance of t-Distribution
  • Derivation of Standard Normal Distribution as a Limit of t-Distribution
  • Derivation of Student’s t-Distribution from Independent Normal and Chi-Square Distributions
• Cauchy Distribution: A Distribution without a Mean $C$

Advanced

In the actual world of formal sciences, Probability Distribution Theory extends beyond the scope of statistics and finds applications in numerous areas. However, well-organized references akin to textbooks are scarce. The reality is that one often needs to explore various papers and study each distribution separately, but in the future, this blog aims to comprehensively organize this information.

Special Distributions

• Pareto Distribution
• Log-Normal Distribution
• Skew-Normal Distribution
• Weibull Distribution
  • Rayleigh Distribution
• Non-central Chi-Square Distribution
• Non-central F-Distribution

Multivariate Distributions

• Multinomial Distribution $M_{k} \left( n, \mathbf{p} \right)$
  • Derivation of Covariance Matrix for Multinomial Distribution
  • Proof of Pearson’s Theorem
• Multivariate Normal Distribution $N_{p} \left( \mu , \Sigma \right)$
  • Linear Transformation of Multivariate Normal Distribution
  • Independence and Zero Correlation are Equivalent in Multivariate Normal Distribution
  • Conditional Mean and Variance in Multivariate Normal Distribution
  • Proof of Multivariate Normality of Regression Coefficient Vector
• Multivariate t-Distribution $t_{p} \left(\nu; \mu , \Sigma \right)$

Directional Distributions

• Reason Modified Bessel Function of the First Kind $I_{\nu}$ Appears in Directional Statistics
• Von Mises Distribution $\text{vM} \left( \mu , \kappa \right)$
• Bivariate Von Mises Distribution $\text{vM}^{2} \left( \mu , \nu , \kappa_{1} , \kappa_{2} \right)$
• Von Mises-Fisher Distribution $\text{vMF}_{p} \left( \mu , \kappa \right)$
• Bingham-Mardia Distribution
• Kent Distribution

References

• Casella. (2001). Statistical Inference(2nd Edition)
• Hogg et al. (2013). Introduction to Mathematical Statistcs(7th Edition)