logo

Alternating Optimization 📂Optimization

Alternating Optimization

Definition

When optimizing a multivariate objective function, the practice of optimizing over one variable at a time, alternating between variables, is known as alternating optimization.

Description

Consider the following optimization problem where the objective function is $H(x,y)$.

$$ \argmin\limits_{x,y} H(x,y) $$

This can be divided into two subproblems by fixing one variable and optimizing over the other.

$$ \begin{cases} \argmin\limits_{x} H(x,y) \\ \argmin\limits_{y} H(x,y) \end{cases} $$

Alternating optimization is the process of iteratively updating the optimal solution for the two variables as follows.

$$ \begin{cases} x^{(k+1)} = \argmin\limits_{x} H(x,y^{(k)}) \\ y^{(k+1)} = \argmin\limits_{y} H(x^{(k+1)},y) \end{cases} $$