logo

Alternating Optimization 📂Optimization

Alternating Optimization

Definition

When optimizing a multivariate objective function, the practice of optimizing over one variable at a time, alternating between variables, is known as alternating optimization.

Description

Consider the following optimization problem where the objective function is H(x,y)H(x,y).

arg min⁡x,yH(x,y) \argmin\limits_{x,y} H(x,y)

This can be divided into two subproblems by fixing one variable and optimizing over the other.

{arg min⁡xH(x,y)arg min⁡yH(x,y) \begin{cases} \argmin\limits_{x} H(x,y) \\ \argmin\limits_{y} H(x,y) \end{cases}

Alternating optimization is the process of iteratively updating the optimal solution for the two variables as follows.

{x(k+1)=arg min⁡xH(x,y(k))y(k+1)=arg min⁡yH(x(k+1),y) \begin{cases} x^{(k+1)} = \argmin\limits_{x} H(x,y^{(k)}) \\ y^{(k+1)} = \argmin\limits_{y} H(x^{(k+1)},y) \end{cases}