Alternating Optimization
Definition
When optimizing a multivariate objective function, the practice of optimizing over one variable at a time, alternating between variables, is known as alternating optimization.
Description
Consider the following optimization problem where the objective function is $H(x,y)$.
$$ \argmin\limits_{x,y} H(x,y) $$
This can be divided into two subproblems by fixing one variable and optimizing over the other.
$$ \begin{cases} \argmin\limits_{x} H(x,y) \\ \argmin\limits_{y} H(x,y) \end{cases} $$
Alternating optimization is the process of iteratively updating the optimal solution for the two variables as follows.
$$ \begin{cases} x^{(k+1)} = \argmin\limits_{x} H(x,y^{(k)}) \\ y^{(k+1)} = \argmin\limits_{y} H(x^{(k+1)},y) \end{cases} $$