📂Optimization

What is the Population Method in Optimization Theory?

Terminology ¹

In solving optimization problems, unlike methods that iteratively improve a single solution such as gradient descent, approaches that treat multiple solutions as individuals and improve solutions through changes among these individuals are broadly called population methods.

Description

In short, a population method is an approach that solves problems by mass sampling — relying on sheer numbers.

If you place an initial population as above — in other words, scatter many individuals throughout the search space — evaluate the function values, and simply select the best one, you can often find a reasonably good solution. A basic population method without a specific strategy is no different from grid search, and, being subject to the curse of dimensionality, cannot be considered a particularly efficient method.

However, methods founded on the population-method paradigm that incorporate various ideas — especially techniques inspired by natural phenomena — have become an established and developing strand of research. There is criticism that many are superficial imitations lacking theoretical analysis, and with the advent of AI and GPUs they have gradually lost some prominence, yet they still appear frequently in research papers. A notable advantage is that they are convenient to apply to nonlinear problems that are non-differentiable.