When merging two large models or teaching a robot to walk, AI often can't directly compute the gradient of the objective function. Instead, it has to try different parameters, then inch toward the optimal solution based on results. This kind of trial-and-error approach, where you don't know the internal rules and can only judge by outcomes, is called "black-box optimization."
https://twitter.com/SakanaAILabs/status/2072930178003779928
Previously, algorithms like evolution strategies and consensus optimization developed independently, lacking a unified theoretical connection. In a paper accepted at ICML 2026, the Sakana AI team has, for the first time, unified these algorithms under a single mathematical framework. They found that the differences mostly come down to two design choices: one, whether you prioritize stability or peak performance; and two, whether all searches converge to a single answer or simultaneously explore multiple possible answers.
Based on this discovery, the team proposed two families of hybrid optimization algorithms. ES-OVI lets developers dial a parameter to freely decide whether the algorithm leans toward "playing it safe" or "going for the high score." The other family (AdaPol, SchedPol) allows the algorithm to dynamically switch between "all searches converge to one answer" and "explore multiple answers at once," blending single-point and multi-point search.
That said, the unified framework currently only works for optimizers that probe uniformly around the current answer (spherical Gaussian distributions). It doesn't yet cover more complex algorithms like CMA-ES, which automatically learn search directions. Also, in model-merging experiments, limited training samples led to some overfitting, and final accuracy hasn't yet matched the level of traditional large-scale evaluation setups.