Automated Metaheuristic Algorithm Design with Autoregressive Learning

Automated design of metaheuristic algorithms offers an attractive avenue to reduce human effort and gain enhanced performance beyond human intuition. Current automated methods design algorithms within a fixed structure and operate from scratch. This poses a clear gap towards fully discovering potentials over the metaheuristic family and fertilizing from prior design experience. To bridge the gap, this paper proposes an autoregressive learning-based designer for automated design of metaheuristic algorithms. Our designer formulates metaheuristic algorithm design as a sequence generation task, and harnesses an autoregressive generative network to handle the task. This offers two advances. First, through autoregressive inference, the designer generates algorithms with diverse lengths and structures, enabling to fully discover potentials over the metaheuristic family. Second, prior design knowledge learned and accumulated in neurons of the designer can be retrieved for designing algorithms for future problems, paving the way to continual design of algorithms for open-ended problem-solving. Extensive experiments on numeral benchmarks and real-world problems reveal that the proposed designer generates algorithms that outperform all human-created baselines on 24 out of 25 test problems. The generated algorithms display various structures and behaviors, reasonably fitting for different problem-solving contexts. Code will be released after paper publication.

Heuristic Solution to Joint Deployment and Beamforming Design for STAR-RIS Aided Networks

This paper tackles the deployment challenges of Simultaneous Transmitting and Reflecting Reconfigurable Intelligent Surface (STAR-RIS) in communication systems. Unlike existing works that use fixed deployment setups or solely optimize the location, this paper emphasizes the joint optimization of the location and orientation of STAR-RIS. This enables searching across all user grouping possibilities and fully boosting the system's performance. We consider a sum rate maximization problem with joint optimization and hybrid beamforming design. An offline heuristic solution is proposed for the problem, developed based on differential evolution and semi-definite programming methods. In particular, a point-point representation is proposed for characterizing and exploiting the user-grouping. A balanced grouping method is designed to achieve a desired user grouping with low complexity. Numerical results demonstrate the substantial performance gains achievable through optimal deployment design.

AutoOptLib: A Library of Automatically Designing Metaheuristic Optimization Algorithms in MATLAB

Metaheuristic algorithms are widely-recognized solvers for challenging optimization problems with multi-modality, discretization, large-scale, multi-objectivity, etc. Automatically designing metaheuristic algorithms leverages today's increasing computing resources to conceive, build up, and verify the design choices of algorithms. It requires much less expertise, labor resources, and time cost than the traditional manual design. Furthermore, by fully exploring the design choices with computing power, automated design is potential to reach or even surpass human-level design, subsequently gaining enhanced performance compared with human problem-solving. These significant advantages have attracted increasing interest and development in the automated design techniques. Open source software is indispensable in response to the increasing interest and development of the techniques. To this end, we have developed a MATLAB library, AutoOptLib, to automatically design metaheuristic algorithms. AutoOptLib, for the first time, provides throughout support to the whole design process, including: 1) plenty of algorithmic components for continuous, discrete, and permutation problems, 2) flexible algorithm representation for evolving diverse algorithm structures, 3) various design objectives and design techniques for different experimentation and application scenarios, and 4) useful experimental tools and graphic user interface (GUI) for practicability and accessibility. In this paper, we first introduce the key features and architecture of the AutoOptLib library. We then illustrate how to use the library by either command or GUI. We further describe additional uses and experimental tools, including parameter importance analysis and benchmark comparison. Finally, we present academic and piratical applications of AutoOptLib, which verifies its efficiency and practicability.

A Survey on Automated Design of Metaheuristic Algorithms

Metaheuristic algorithms have attracted wide attention from academia and industry due to their capability of conducting search independent of problem structures and problem domains. Often, human experts are requested to manually tailor algorithms to fit for solving a targeted problem. The manual tailoring process may be laborious, error-prone, and require intensive specialized knowledge. This gives rise to increasing interests and demands for automated design of metaheuristic algorithms with less human intervention. The automated design could make high-performance algorithms accessible to a much broader range of researchers and practitioners; and by leveraging computing power to fully explore the potential design choices, automated design could reach or even surpass human-level design. This paper presents a broad picture of the formalization, methodologies, challenges, and research trends of automated design of metaheuristic algorithms, by conducting a survey on the common grounds and representative techniques in this field. In the survey, we first present the concept of automated design of metaheuristic algorithms and provide a taxonomy by abstracting the automated design process into four parts, i.e., design space, design strategies, performance evaluation strategies, and targeted problems. Then, we overview the techniques concerning the four parts of the taxonomy and discuss their strengths, weaknesses, challenges, and usability, respectively. Finally, we present research trends in this field.

AutoOpt: A Methodological Framework of Automatically Designing Metaheuristics for Optimization Problems

Metaheuristics are gradient-free and problem-independent search algorithms. They have gained huge success in solving various optimization problems in academia and industry. Automated metaheuristic design is a promising alternative to human-made design. This paper proposes a general and comprehensive methodological framework, AutoOpt, for automatically designing metaheuristics for various optimization problems. AutoOpt consists of: (1) a bi-level criterion to evaluate the designed algorithms' performance; (2) a general schema of the decision space from where the algorithms will be designed; (3) a mixed graph- and real number-based representation to represent the designed algorithms; and (4) a model-free method to conduct the design process. AutoOpt benefits academic researchers and practical users struggling to design metaheuristics for optimization problems. A real-world case study demonstrates AutoOpt's effectiveness and efficiency.

Hybrid Beamforming for RIS-Aided Communications: Fitness Landscape Analysis and Niching Genetic Algorithm

Reconfigurable Intelligent Surface (RIS) is a revolutionizing approach to provide cost-effective yet energy-efficient communications. The transmit beamforming of the base station (BS) and discrete phase shifts of the RIS are jointly optimized to provide high quality of service. However, existing works ignore the high dependence between the large number of phase shifts and estimate them separately, consequently, easily getting trapped into local optima. To investigate the number and distribution of local optima, we conduct a fitness landscape analysis on the sum rate maximization problems. Two landscape features, the fitness distribution correlation and autocorrelation, are employed to investigate the ruggedness of landscape. The investigation results indicate that the landscape exhibits a rugged, multi-modal structure, i.e., has many local peaks, particularly in the cases with large-scale RISs. To handle the multi-modal landscape structure, we propose a novel niching genetic algorithm to solve the sum rate maximization problem. Particularly, a niching technique, nearest-better clustering, is incorporated to partition the population into several neighborhood species, thereby locating multiple local optima and enhance the global search ability. We also present a minimum species size to further improve the convergence speed. Simulation results demonstrate that our method achieves significant capacity gains compared to existing algorithms, particularly in the cases with large-scale RISs.

Gridless Evolutionary Approach for Line Spectral Estimation with Unknown Model Order

Gridless methods show great superiority in line spectral estimation. These methods need to solve an atomic $l_0$ norm (i.e., the continuous analog of $l_0$ norm) minimization problem to estimate frequencies and model order. Since this problem is NP-hard to compute, relaxations of atomic $l_0$ norm, such as nuclear norm and reweighted atomic norm, have been employed for promoting sparsity. However, the relaxations give rise to a resolution limit, subsequently leading to biased model order and convergence error. To overcome the above shortcomings of relaxation, we propose a novel idea of simultaneously estimating the frequencies and model order by means of the atomic $l_0$ norm. To accomplish this idea, we build a multiobjective optimization model. The measurment error and the atomic $l_0$ norm are taken as the two optimization objectives. The proposed model directly exploits the model order via the atomic $l_0$ norm, thus breaking the resolution limit. We further design a variable-length evolutionary algorithm to solve the proposed model, which includes two innovations. One is a variable-length coding and search strategy. It flexibly codes and interactively searches diverse solutions with different model orders. These solutions act as steppingstones that help fully exploring the variable and open-ended frequency search space and provide extensive potentials towards the optima. Another innovation is a model order pruning mechanism, which heuristically prunes less contributive frequencies within the solutions, thus significantly enhancing convergence and diversity. Simulation results confirm the superiority of our approach in both frequency estimation and model order selection.

Multiobjective Bilevel Evolutionary Approach for Off-Grid Direction-of-Arrival Estimation

The source number identification is an essential step in direction-of-arrival (DOA) estimation. Existing methods may provide a wrong source number due to inferior statistical properties (in low SNR or limited snapshots) or modeling errors (caused by relaxing sparse penalties), especially in impulsive noise. To address this issue, we propose a novel idea of simultaneous source number identification and DOA estimation. We formulate a multiobjective off-grid DOA estimation model to realize this idea, by which the source number can be automatically identified together with DOA estimation. In particular, the source number is properly exploited by the $l_0$ norm of impinging signals without relaxations, guaranteeing accuracy. Furthermore, we design a multiobjective bilevel evolutionary algorithm to solve the proposed model. The source number identification and sparse recovery are simultaneously optimized at the on-grid (lower) level. A forward search strategy is developed to further refine the grid at the off-grid (upper) level. This strategy does not need linear approximations and can eliminate the off-grid gap with low computational complexity. Simulation results demonstrate the outperformance of our method in terms of source number and root mean square error.

Evolutionary Robust Clustering Over Time for Temporal Data

In many clustering scenes, data samples' attribute values change over time. For such data, we are often interested in obtaining a partition for each time step and tracking the dynamic change of partitions. Normally, a smooth change is assumed for data to have a temporal smooth nature. Existing algorithms consider the temporal smoothness as an a priori preference and bias the search towards the preferred direction. This a priori manner leads to a risk of converging to an unexpected region because it is not always the case that a reasonable preference can be elicited given the little prior knowledge about the data. To address this issue, this paper proposes a new clustering framework called evolutionary robust clustering over time. One significant innovation of the proposed framework is processing the temporal smoothness in an a posteriori manner, which avoids unexpected convergence that occurs in existing algorithms. Furthermore, the proposed framework automatically tunes the weight of smoothness without data's affinity matrix and predefined parameters, which holds better applicability and scalability. The effectiveness and efficiency of the proposed framework are confirmed by comparing with state-of-the-art algorithms on both synthetic and real datasets.