Catch Me If You Can: Deep Meta-RL for Search-and-Rescue using LoRa UAV Networks

Long range (LoRa) wireless networks have been widely proposed as a efficient wireless access networks for the battery-constrained Internet of Things (IoT) devices. In many practical search-and-rescue (SAR) operations, one challenging problem is finding the location of devices carried by a lost person. However, using a LoRa-based IoT network for SAR operations will have a limited coverage caused by high signal attenuation due to the terrestrial blockages especially in highly remote areas. To overcome this challenge, the use of unmanned aerial vehicles (UAVs) as a flying LoRa gateway to transfer messages from ground LoRa nodes to the ground rescue station can be a promising solution. In this paper, the problem of the flying LoRa (FL) gateway control in the search-and-rescue system using the UAV-assisted LoRa network is modeled as a partially observable Markov decision process. Then, a deep meta-RL-based policy is proposed to control the FL gateway trajectory during SAR operation. For initialization of proposed deep meta-RL-based policy, first, a deep RL-based policy is designed to determine the adaptive FL gateway trajectory in a fixed search environment including a fixed radio geometry. Then, as a general solution, a deep meta-RL framework is used for SAR in any new and unknown environments to integrate the prior FL gateway experience with information collected from the other search environments and rapidly adapt the SAR policy model for SAR operation in a new environment. The proposed UAV-assisted LoRa network is then experimentally designed and implemented. Practical evaluation results show that if the deep meta-RL based control policy is applied instead of the deep RL-based one, the number of SAR time slots decreases from 141 to 50.

Joint User Association and UAV Location Optimization for Two-Tired Visible Light Communication Networks

In this paper, an unmanned aerial vehicle (UAVs)-assisted visible light communication (VLC) has been considered which has two tiers: UAV-to-centroid and device-to-device (D2D). In the UAV-to-centroid tier, each UAV can simultaneously provide communications and illumination for the centroids of the ground users over VLC links. In the D2D tier, the centroids retransmit received data from UAV over D2D links to the cluster members. For network, the optimization problem of joint user association and deployment location of UAVs is formulated to maximize the received data, satisfy illumination constraint, and the user cluster size. An iterative algorithm is first proposed to transform the optimization problem into a series of two interdependent sub problems. Following the smallest enclosing disk theorem, a random incremental construction method is designed to find the optimal UAV locations. Then, inspired by unsupervised learning method, a clustering algorithm to find a suboptimal user association is proposed. Our simulation results show that the proposed scheme on average guarantees the users brightness 0.77 lux more than their threshold requirements. Moreover, the received bitrate plus number of D2D connected users under our proposed method is 50.69% more than the scenario in which we have RF Link instead of VLC link and do not optimize UAV location.

Energy-Efficient Resource Allocation for Multi-IRS-Aided Indoor 6G Networks

In this paper, we propose a distributed intelligent reflecting surface (IRS) assisted single-user and multi-user millimeter wave (mmWave) system. Then, we formulate the resource allocation problem as an optimization to maximize energy efficiency under individual quality of service (QoS) constraints. We first propose a centralized algorithm, and further, a low-complexity distributed one where the access point (AP) and IRSs independently adjust the transmit beamforming of AP, the phase shifts, and the on-off status of IRSs in an alternating manner until the convergence is reached. In a multi-user scenario, in the first stage, the successive convex approximation (SCA) and fractional programming (FP) approaches are applied to achieve a solution for optimization subproblems of the phase-shift coefficients and element on-off status of IRSs. Then, for the beamforming subproblem, a modified nested FP approach is proposed that finds an optimal solution for the beamforming vectors of AP. Our performance analysis on a practical scenario shows that the proposed centralized and distributed approach respectively enhances the energy efficiency by up to 55%, 42% for single-user, and up to 984% for multi-user scenarios, in comparison to the case where the on-off status and phase-shift coefficients of IRS elements are not selected optimally.

Ultra-Reliable Indoor Millimeter Wave Communications using Multiple Artificial Intelligence-Powered Intelligent Surfaces

In this paper, a novel framework for guaranteeing ultra-reliable millimeter wave (mmW) communications using multiple artificial intelligence (AI)-enabled reconfigurable intelligent surfaces (RISs) is proposed. The use of multiple AI-powered RISs allows changing the propagation direction of the signals transmitted from a mmW access point (AP) thereby improving coverage particularly for non-line-of-sight (NLoS) areas. However, due to the possibility of highly stochastic blockage over mmW links, designing an intelligent controller to jointly optimize the mmW AP beam and RIS phase shifts is a daunting task. In this regard, first, a parametric risk-sensitive episodic return is proposed to maximize the expected bit rate and mitigate the risk of mmW link blockage. Then, a closed-form approximation of the policy gradient of the risk-sensitive episodic return is analytically derived. Next, the problem of joint beamforming for mmW AP and phase shift control for mmW RISs is modeled as an identical payoff stochastic game within a cooperative multi-agent environment, in which the agents are the mmW AP and the RISs. Two centralized and distributed controllers are proposed to control the policies of the mmW AP and RISs. To directly find an optimal solution, the parametric functional-form policies for these controllers are modeled using deep recurrent neural networks (RNNs). Simulation results show that the error between policies of the optimal and the RNN-based controllers is less than 1.5%. Moreover, the variance of the achievable rates resulting from the deep RNN-based controllers is 60% less than the variance of the risk-averse baseline.