📂Optimization

Elitism in Genetic Algorithms

Definition ^{1 2}

In genetic algorithms, elitism is the strategy of unconditionally carrying a few of the highest-fitness individuals into the next generation each time natural selection occurs.

Explanation

Elitism can be seen as an extreme form of rank selection, and, without stochastic elements, it can also be considered another name for truncation selection, which passes a subset of individuals that meet a condition to the next generation. In a genetic algorithm that adopts elitism, the objective function’s profile over generations takes the form of a monotonic function.

Surprisingly, losing the “best-so-far” solution is common in optimization problems, and this also happens in gradient descent. For example, one might obtain the best solution at a relatively early stage of training, but if training continues through all epochs, overfitting can occur so that the final solution is worse than that earlier one. This is usually avoided by checkpointing and saving the best solution.

Meanwhile, in genetic algorithms elitism differs in that elites are not only preserved as checkpoints but continually influence the solution set during the evolutionary process. Offspring tend to resemble elite parents and therefore share similar genotypes; offspring are also likely to have high fitness, increasing the probability that those genes are transmitted to subsequent generations. From the perspective of population-based methods, this can be interpreted as an intent to more thoroughly explore the neighborhood of a local optimum.