Towards Fast Rates for Federated and Multi-Task Reinforcement Learning

We consider a setting involving $N$ agents, where each agent interacts with an environment modeled as a Markov Decision Process (MDP). The agents' MDPs differ in their reward functions, capturing heterogeneous objectives/tasks. The collective goal of the agents is to communicate intermittently via a central server to find a policy that maximizes the average of long-term cumulative rewards across environments. The limited existing work on this topic either only provide asymptotic rates, or generate biased policies, or fail to establish any benefits of collaboration. In response, we propose Fast-FedPG - a novel federated policy gradient algorithm with a carefully designed bias-correction mechanism. Under a gradient-domination condition, we prove that our algorithm guarantees (i) fast linear convergence with exact gradients, and (ii) sub-linear rates that enjoy a linear speedup w.r.t. the number of agents with noisy, truncated policy gradients. Notably, in each case, the convergence is to a globally optimal policy with no heterogeneity-induced bias. In the absence of gradient-domination, we establish convergence to a first-order stationary point at a rate that continues to benefit from collaboration.

Towards 6G MIMO: Massive Spatial Multiplexing, Dense Arrays, and Interplay Between Electromagnetics and Processing

The increasing demand for wireless data transfer has been the driving force behind the widespread adoption of Massive MIMO (multiple-input multiple-output) technology in 5G. The next-generation MIMO technology is now being developed to cater to the new data traffic and performance expectations generated by new user devices and services in the next decade. The evolution towards "ultra-massive MIMO (UM-MIMO)" is not only about adding more antennas but will also uncover new propagation and hardware phenomena that can only be treated by jointly utilizing insights from the communication, electromagnetic (EM), and circuit theory areas. This article offers a comprehensive overview of the key benefits of the UM-MIMO technology and the associated challenges. It explores massive multiplexing facilitated by radiative near-field effects, characterizes the spatial degrees-of-freedom, and practical channel estimation schemes tailored for massive arrays. Moreover, we provide a tutorial on EM theory and circuit theory, and how it is used to obtain physically consistent antenna and channel models. Subsequently, the article describes different ways to implement massive and dense antenna arrays, and how to co-design antennas with signal processing. The main open research challenges are identified at the end.

Green Joint Communications and Sensing Employing Analogue Multi-Beam Antenna Arrays

Joint communications and sensing (JCAS) is potentially a hallmark technology for the sixth generation mobile network (6G). Most existing JCAS designs are based on digital arrays, analog arrays with tunable phase shifters, or hybrid arrays, which are effective but are generally complicated to design and power inefficient. This article introduces the energy-efficient and easy-to-design multi-beam antenna arrays (MBAAs) for JCAS. Using pre-designed and fixed analog devices, such as lens or Butler matrix, MBAA can simultaneously steer multiple beams yet with negligible power consumption compared with other techniques. Moreover, MBAAs enable flexible beam synthesis, accurate angle-of-arrival estimation, and easy handling/utilization of the beam squint effect. All these features have not been well captured by the JACS community yet. To promote the awareness of them, we intuitively illustrate them and also exploit them for constructing a multi-beam JCAS framework. Finally, the challenges and opportunities are discussed to foster the development of green JCAS systems.

An Overview of Signal Processing Techniques for Joint Communication and Radar Sensing

Joint communication and radar sensing (JCR) represents an emerging research field aiming to integrate the above two functionalities into a single system, sharing a majority of hardware and signal processing modules and, in a typical case, sharing a single transmitted signal. It is recognised as a key approach in significantly improving spectrum efficiency, reducing device size, cost and power consumption, and improving performance thanks to potential close cooperation of the two functions. Advanced signal processing techniques are critical for making the integration efficient, from transmission signal design to receiver processing. This paper provides a comprehensive overview of JCR systems from the signal processing perspective, with a focus on state-of-the-art. A balanced coverage on both transmitter and receiver is provided for three types of JCR systems, communication-centric, radar-centric, and joint design and optimization.

Grassmannian Learning: Embedding Geometry Awareness in Shallow and Deep Learning

Modern machine learning algorithms have been adopted in a range of signal-processing applications spanning computer vision, natural language processing, and artificial intelligence. Many relevant problems involve subspace-structured features, orthogonality constrained or low-rank constrained objective functions, or subspace distances. These mathematical characteristics are expressed naturally using the Grassmann manifold. Unfortunately, this fact is not yet explored in many traditional learning algorithms. In the last few years, there have been growing interests in studying Grassmann manifold to tackle new learning problems. Such attempts have been reassured by substantial performance improvements in both classic learning and learning using deep neural networks. We term the former as shallow and the latter deep Grassmannian learning. The aim of this paper is to introduce the emerging area of Grassmannian learning by surveying common mathematical problems and primary solution approaches, and overviewing various applications. We hope to inspire practitioners in different fields to adopt the powerful tool of Grassmannian learning in their research.