📂Probability Theory

Characteristic Functions and Moment Generating Functions Defined by Measure Theory

Definition ¹

Suppose a probability space $( \Omega , \mathcal{F} , P)$ is given. A $\varphi_{X} (t)$ defined as follows for the random variables $X$ and $t \in \mathbb{R}$ is called the characteristic function of $X$. $$ \varphi_{X} (t) := E \left( e^{i t X} \right) = \int_{\mathbb{R}} e^{it x} f_{X} (x) dx $$

---

• If you haven’t encountered measure theory yet, you can ignore the term probability space.

Explanation

The random variable $Z : = X + i Y$ is defined to have the following properties with respect to the two random variables $X, Y : \Omega \to \mathbb{R}$. $$ \int Z dP = \int X dP + i \int Y dP $$ Then, the characteristic function, according to its expectation representation and the Euler’s formula, is $$ \begin{align*} \varphi_{X} (t) =& E \left( e^{i t X} \right) \\ =& \int \left[ \cos(tX) + i \sin (t X) \right] dP \\ =& \int e^{it X} dP \\ =& \int \cos ( tX ) dP + i \int \sin ( t X ) dP \end{align*} $$ and, it can be seen that $e^{itX}$ extends well into the complex numbers.

The characteristic function, from its very definition, is reminiscent of the moment-generating function $M(t) = E \left( e^{tX} \right)$, and indeed, it serves a similar purpose in probability theory. The introduction of complex numbers should not be too intimidating. Deriving the mgf from the characteristic function is straightforward. If we set $t = -i T$ for $T \in \mathbb{R}$, $$ \begin{align*} \varphi_{X} (t) =& E \left( e^{i t X} \right) \\ =& E \left( e^{i (- i T) X} \right) \\ =& E \left( e^{T X} \right) \\ =& M(T) \end{align*} $$ it becomes the moment-generating function for $T$. The characteristic function can be considered almost the same as the mgf.

Throughout mathematics, the term Characteristic is used extensively. From the perspective of studying probability theory, it’s reasonable to be proud that our characteristic function is genuine. At least when searched on Google, the most prominent results relate to the characteristic function in probability theory. In other fields, “characteristic” is often used when there’s a comparably difficult problem, to transform it into a $n$th degree equation and study only its “characteristics”, not really focusing on the equation itself. Of course, $\varphi_{X}$ is also usually used to study the distribution of $X$, but it is indeed treated more frequently and importantly compared to other fields.