First-Order Necessary Conditions for Extrema of Multivariable Functions
Theorem1
Let’s assume the function $f : \mathbb{R}^{n} \to \mathbb{R}$ is given. If $x^{\ast}$ is a local optimizer and $f \in C^{1}$ in the vicinity of $x^{\ast}$, then,
$$ \nabla f(x^{\ast}) = 0 $$
$\nabla f$ is the gradient of $f$. Note here that $0$ is not the numeric zero, but a zero vector.
Explanation
The first-order necessary condition tells us about the property of the gradient, which is the first-order derivative of $f$, when $x^{\ast}$ is a local minimizer of $f$. Named and extended to multivariable functions, the concept that differentiation at a maximum or minimum yields $0$ is something we learn even in high school.
Proof
We prove by contradiction. Assume $\nabla f (x^{\ast}) \ne 0$. And denote as follows:
$$ p = - \nabla f (x^{\ast}),\quad p^{t}\nabla f (x^{\ast}) = - \left\| \nabla f (x^{\ast}) \right\|^{2} \lt 0 $$
Here, $p^{t}$ denotes the transpose matrix. Then, because $\nabla f$ is continuous, there exists $s \gt 0$ such that the following equation holds:
$$ \begin{equation} p^{t}\nabla f (x^{\ast} + \xi p) \lt 0, \qquad \forall \xi \in [0, s] \end{equation} $$
Furthermore, by the Taylor expansion formula for multivariable functions,
$$ \begin{equation} f(x^{\ast} + \xi p) = f (x^{\ast}) + \xi p^{t} \nabla f(x^{\ast} + \bar{\xi} p),\quad \text{for some } \bar{\xi} \in (0, \xi) \end{equation} $$
Then, by $(1)$ and $(2)$, we obtain the following:
$$ f(x^{\ast} + \xi p) \lt f (x^{\ast}), \qquad \forall \xi \in [0, s] $$
This contradicts the fact that $x^{\ast}$ is a local minimizer. Therefore, the assumption is wrong, and $\nabla f (x^{\ast}) = 0$ is true.
■
See Also
J. Nocedal and Stephen J. Wright, Numerical Optimization (2nd), p14-15 ↩︎