Proof of the Finite Form of Jensen's Inequality
Theorem
For a convex function $f : I \to \mathbb{R}$ on $I \subset \mathbb{R}$ and $\displaystyle \sum_{k=1}^{n} \lambda_{k} = 1, \lambda_{k}>0$,
$$ \begin{align*} f( \lambda_{1} x_{1} + \lambda_{2} x_{2} + \cdots + \lambda_{n} x_{n} ) & \le \lambda_{1} f( x_{1}) + \lambda_{2} f( x_{2}) + \cdots + \lambda_{n} f( x_{n} ) \\ f\left( \sum\limits_{k=1}^{n}\lambda_{k}x_{k} \right) &\le \sum\limits_{k=1}^{n} \lambda_{k} f(x_{k}) \end{align*} $$
If $f$ is a concave function, the reverse inequality holds.
$$ \begin{align*} f( \lambda_{1} x_{1} + \lambda_{2} x_{2} + \cdots + \lambda_{n} x_{n} ) & \ge \lambda_{1} f( x_{1}) + \lambda_{2} f( x_{2}) + \cdots + \lambda_{n} f( x_{n} ) \\ f\left( \sum\limits_{k=1}^{n}\lambda_{k}x_{k} \right) &\ge \sum\limits_{k=1}^{n} \lambda_{k} f(x_{k}) \end{align*} $$
Explanation
Jensen’s inequality is a representative application of convex functions and is used widely across many fields. Its finite form has the shape obtained by generalizing the original definition of convexity with respect to the number of points and the weights.
Proof
Since the proof method is the same, we prove it only for convex functions.
Strategy: Using the definition of a convex function, we insert terms inside and outside the function and apply mathematical induction.
Definition of a convex function: For two elements $x_{1} , x_{2}$ of an interval $I \subset \mathbb{R}$, a function $f : I \to \mathbb{R}$, and $0 \le t \le 1$, if $f( t x_{1} + (1-t) x_{2}) \le t f(x_{1}) + (1-t) f(x_{2})$, then $f$ is called a convex function on $I$.
When $n=1$, since $f$ is convex,
$$ f( \lambda x ) \le \lambda f(x) $$
When $n=2$, since $f$ is convex,
$$ f( \lambda_{1} x_{1} + \lambda_{2} x_{2} ) \le \lambda_{1} f( x_{1} ) + \lambda_{2} f( x_{2}) $$
Assume that at $n = N \ge 3$
$$ f( \lambda_{1} x_{1} + \lambda_{2} x_{2} + \cdots + \lambda_{N} x_{N} ) \le \lambda_{1} f( x_{1}) + \lambda_{2} f( x_{2}) + \cdots + \lambda_{N} f( x_{N} ) $$
holds, and the proof is complete if we show that it also holds for $n = N + 1$.
From $\displaystyle \sum_{k=1}^{N+1} \lambda_{k} = 1$, defining $\displaystyle \mu_{i} := {{\lambda_{i}} \over {1 - \lambda_{N+1}}}$ gives $\displaystyle \sum_{k=1}^{N} \mu_{k} = 1$, so
$$ f( \mu_{1} x_{1} + \mu_{2} x_{2} + \cdots + \mu_{N} x_{N} ) \le \mu_{1} f( x_{1}) + \mu_{2} f( x_{2}) + \cdots + \mu_{N} f( x_{N} ) $$
By the definition of $\mu_{i}$,
$$ f( {{\lambda_{1}} \over {1 - \lambda_{N+1}}} x_{1} + {{\lambda_{2}} \over {1 - \lambda_{N+1}}} x_{2} + \cdots + {{\lambda_{N}} \over {1 - \lambda_{N+1}}} x_{N} ) \\ \le {{\lambda_{1}} \over {1 - \lambda_{N+1}}} f( x_{1}) + {{\lambda_{2}} \over {1 - \lambda_{N+1}}} f( x_{2}) + \cdots + {{\lambda_{N}} \over {1 - \lambda_{N+1}}} f( x_{N} ) $$
Factoring out $\displaystyle {{1} \over {1 - \lambda_{N+1}}}$,
$$ f \left( {{1} \over {1 - \lambda_{N+1}}} \left( \lambda_{1} x_{1} + \lambda_{2} x_{2} + \cdots + \lambda_{N} x_{N} \right) \right) \\ \le {{1} \over {1 - \lambda_{N+1}}} \left( \lambda_{1} f( x_{1}) + \lambda_{2} f( x_{2}) + \cdots + \lambda_{N} f( x_{N} ) \right) $$
Multiplying both sides by $1 - \lambda_{N+1}$,
$$ (1 - \lambda_{N+1}) f \left( {{1} \over {1 - \lambda_{N+1}}} \left( \lambda_{1} x_{1} + \lambda_{2} x_{2} + \cdots + \lambda_{N} x_{N} \right) \right) \\ \le \lambda_{1} f( x_{1}) + \lambda_{2} f( x_{2}) + \cdots + \lambda_{N} f( x_{N} ) $$
Adding $\lambda_{N+1} f( x_{N+1} )$ to both sides,
$$ (1 - \lambda_{N+1}) f \left( {{1} \over {1 - \lambda_{N+1}}} \left( \lambda_{1} x_{1} + \lambda_{2} x_{2} + \cdots + \lambda_{N} x_{N} \right) \right) + \lambda_{N+1} f (x_{N+1}) \\ \le \lambda_{1} f( x_{1}) + \lambda_{2} f( x_{2}) + \cdots + \lambda_{N} f( x_{N} ) + \lambda_{N+1} f( x_{N+1} ) $$
Since $( 1 - \lambda_{N+1} ) + \lambda_{N+1} = 1$ and $f$ is a convex function,
$$ f \left( {{1 - \lambda_{N+1}} \over {1 - \lambda_{N+1}}} \left( \lambda_{1} x_{1} + \lambda_{2} x_{2} + \cdots + \lambda_{N} x_{N} \right) + \lambda_{N+1} x_{N+1} \right) \\ \le (1 - \lambda_{N+1}) f \left( {{1} \over {1 - \lambda_{N+1}}} \left( \lambda_{1} x_{1} + \lambda_{2} x_{2} + \cdots + \lambda_{N} x_{N} \right) \right) + \lambda_{N+1} f (x_{N+1}) $$
Chaining the inequalities,
$$ f \left( \lambda_{1} x_{1} + \lambda_{2} x_{2} + \cdots + \lambda_{N} x_{N} + \lambda_{N+1} x_{N+1} \right) \\ \le (1 - \lambda_{N+1}) f \left( {{1} \over {1 - \lambda_{N+1}}} \left( \lambda_{1} x_{1} + \lambda_{2} x_{2} + \cdots + \lambda_{N} x_{N} \right) \right) + \lambda_{N+1} f (x_{N+1}) \\ \le \lambda_{1} f( x_{1}) + \lambda_{2} f( x_{2}) + \cdots + \lambda_{N} f( x_{N} ) + \lambda_{N+1} f( x_{N+1} ) $$
Rearranging,
$$ f( \lambda_{1} x_{1} + \lambda_{2} x_{2} + \cdots + \lambda_{N+1} x_{N+1} ) \le \lambda_{1} f( x_{1}) + \lambda_{2} f( x_{2}) + \cdots + \lambda_{N+1} f( x_{N+1} ) $$
■