📂Lemmas

Proof of the Expectation Form of Jensen's Inequality

Theorem ¹

If a function $\phi$ is convex and twice differentiable on an open interval $I$, the expectation $\mu$ of a random variable $X$ exists, and $X \subset I $, then

$$ \phi [ E(X) ] \le E [ \phi (X)] $$

If $\phi$ is concave, the inequality holds in the opposite direction.

$$ \phi [ E(X) ] \ge E [ \phi (X)] $$

Other Forms

• Finite form of Jensen’s inequality
• Integral form of Jensen’s inequality
• Conditional Jensen’s inequality

It has a form quite similar to the integral form. If you think about it carefully, the finite form too, although its terms are not infinite, can be regarded as an expectation in the sense that it is an inequality about the weighted average.

Proof

Strategy: In the original general proof, $\phi$ need not have a second derivative and it suffices for it to be merely convex. For convenience, we will assume it has a second derivative.

---

By Taylor’s theorem, $$ \phi (x) = \phi (\mu) + \phi^{\prime} (\mu) (x - \mu) + \phi^{\prime \prime} (\xi) {{(x - \mu)^2} \over {2}} $$ there exists $\xi$ between $x$ and $\mu$ satisfying this. Since $\phi$ is convex, $\phi^{\prime \prime} (\xi) > 0$ and $$ \phi^{\prime \prime} (\xi) {{(x - \mu)^2} \over {2}} > 0 $$ Rearranging, $$ \phi (x) \ge \phi (\mu) + \phi^{\prime} (\mu) (x - \mu) $$ Taking the expectation $E$ on both sides, since $E(X-\mu) = 0$, $$ E( \phi ( X ) ) \ge \phi ( E (X) ) $$

■