📂Probability Theory

Definition ¹

Probability Space $( \Omega , \mathcal{F} , P)$ is given. For Random Variables $X$ and $t \in \mathbb{R}$, the function defined as follows $\varphi_{X} (t)$ is called the characteristic function of $X$. $$ \varphi_{X} (t) := E \left( e^{i t X} \right) = \int_{\mathbb{R}} e^{it x} f_{X} (x) dx $$

---

• If you haven’t encountered measure theory yet, you can ignore the term “probability space.”

Explanation

The random variable $Z : = X + i Y$ is defined to have the following properties for the two random variables $X, Y : \Omega \to \mathbb{R}$. $$ \int Z dP = \int X dP + i \int Y dP $$ Then, the characteristic function, in terms of its expected value representation and Euler’s formula, is $$ \begin{align*} \varphi_{X} (t) =& E \left( e^{i t X} \right) \\ =& \int \left[ \cos(tX) + i \sin (t X) \right] dP \\ =& \int e^{it X} dP \\ =& \int \cos ( tX ) dP + i \int \sin ( t X ) dP \end{align*} $$ , and you can confirm that $e^{itX}$ has been well extended to complex numbers.

The characteristic function, from its form, looks strikingly similar to the moment generating function $M(t) = E \left( e^{tX} \right)$, and it is indeed often used for similar purposes in probability theory. The introduction of complex numbers shouldn’t be too intimidating. Deriving the mgf from the characteristic function is straightforward. For $T \in \mathbb{R}$ given $t = -i T$, it can be denoted as $$ \begin{align*} \varphi_{X} (t) =& E \left( e^{i t X} \right) \\ =& E \left( e^{i (- i T) X} \right) \\ =& E \left( e^{T X} \right) \\ =& M(T) \end{align*} $$ , which becomes a moment generating function for $T$. The characteristic function can almost be viewed as equivalent to the mgf.

Meanwhile, the term characteristic is extensively used throughout mathematics. As a student of probability theory, one might feel pride in the thought that ‘our characteristic function is the real deal’. At least, when you Google it, the top results are usually about the characteristic function in probability theory. In other fields, ‘characteristic’ is mostly used when dealing with relatively complex problems that transform them into $n$-th order equations, focusing on their ‘characteristics’ rather than the equations themselves. Of course, $\varphi_{X}$ is also often used to study the distribution of $X$, but it is treated much more frequently and importantly in probability theory compared to other fields.

See Also

• Fourier Transform: Formally, the characteristic function is the Fourier inverse transform of the probability density function.