Reinforcement Learning for Enhancing Sensing Estimation in Bistatic ISAC Systems with UAV Swarms

This paper introduces a novel Multi-Agent Reinforcement Learning (MARL) framework to enhance integrated sensing and communication (ISAC) networks using unmanned aerial vehicle (UAV) swarms as sensing radars. By framing the positioning and trajectory optimization of UAVs as a Partially Observable Markov Decision Process, we develop a MARL approach that leverages centralized training with decentralized execution to maximize the overall sensing performance. Specifically, we implement a decentralized cooperative MARL strategy to enable UAVs to develop effective communication protocols, therefore enhancing their environmental awareness and operational efficiency. Additionally, we augment the MARL solution with a transmission power adaptation technique to mitigate interference between the communicating drones and optimize the communication protocol efficiency. Moreover, a transmission power adaptation technique is incorporated to mitigate interference and optimize the learned communication protocol efficiency. Despite the increased complexity, our solution demonstrates robust performance and adaptability across various scenarios, providing a scalable and cost-effective enhancement for future ISAC networks.

Complex Neural Network based Joint AoA and AoD Estimation for Bistatic ISAC

Integrated sensing and communication (ISAC) in wireless systems has emerged as a promising paradigm, offering the potential for improved performance, efficient resource utilization, and mutually beneficial interactions between radar sensing and wireless communications, thereby shaping the future of wireless technologies. In this work, we present two novel methods to address the joint angle of arrival and angle of departure estimation problem for bistatic ISAC systems. Our proposed methods consist of a deep learning (DL) solution leveraging complex neural networks, in addition to a parameterized algorithm. By exploiting the estimated channel matrix and incorporating a preprocessing step consisting of a coarse timing estimation, we are able to notably reduce the input size and improve the computational efficiency. In our findings, we emphasize the remarkable potential of our DL-based approach, which demonstrates comparable performance to the parameterized method that explicitly exploits the multiple-input multiple-output (MIMO) model, while exhibiting significantly lower computational complexity.

Emergent Communication in Multi-Agent Reinforcement Learning for Future Wireless Networks

In different wireless network scenarios, multiple network entities need to cooperate in order to achieve a common task with minimum delay and energy consumption. Future wireless networks mandate exchanging high dimensional data in dynamic and uncertain environments, therefore implementing communication control tasks becomes challenging and highly complex. Multi-agent reinforcement learning with emergent communication (EC-MARL) is a promising solution to address high dimensional continuous control problems with partially observable states in a cooperative fashion where agents build an emergent communication protocol to solve complex tasks. This paper articulates the importance of EC-MARL within the context of future 6G wireless networks, which imbues autonomous decision-making capabilities into network entities to solve complex tasks such as autonomous driving, robot navigation, flying base stations network planning, and smart city applications. An overview of EC-MARL algorithms and their design criteria are provided while presenting use cases and research opportunities on this emerging topic.