Distributed evolution strategies for large-scale optimization

As their underlying models are becoming larger and data-driven, an increasing number of modern real-world applications can be mathematically formulated as large-scale continuous optimization. In this paper, we propose a distributed evolution strategy (DES) for large-scale black-box optimization (specifically with memory-costly function evaluations), running on the mainstream clustering computing platform. In order to amortize the memory cost, DES utilizes the distributed shared memory to support parallelism of function evaluations. For better fitting into the scalable computing architecture of interest, DES adopts the well-known island model to distribute one low-rank version of covariance matrix adaptation (CMA), because the quadratic complexity of the standard CMA is not well scalable. For a proper trade-off between exploration and exploitation, DES needs to, on-the-fly, adjust strategy parameters at two time-scale levels. Experiments show its efficiency on most of test functions chosen.