Dynamic Resource Allocation for Metaverse Applications with Deep Reinforcement Learning

This work proposes a novel framework to dynamically and effectively manage and allocate different types of resources for Metaverse applications, which are forecasted to demand massive resources of various types that have never been seen before. Specifically, by studying functions of Metaverse applications, we first propose an effective solution to divide applications into groups, namely MetaInstances, where common functions can be shared among applications to enhance resource usage efficiency. Then, to capture the real-time, dynamic, and uncertain characteristics of request arrival and application departure processes, we develop a semi-Markov decision process-based framework and propose an intelligent algorithm that can gradually learn the optimal admission policy to maximize the revenue and resource usage efficiency for the Metaverse service provider and at the same time enhance the Quality-of-Service for Metaverse users. Extensive simulation results show that our proposed approach can achieve up to 120% greater revenue for the Metaverse service providers and up to 178.9% higher acceptance probability for Metaverse application requests than those of other baselines.

Optimal Privacy Preserving in Wireless Federated Learning System over Mobile Edge Computing

Federated Learning (FL) with quantization and deliberately added noise over wireless networks is a promising approach to preserve the user differential privacy while reducing the wireless resources. Specifically, an FL learning process can be fused with quantized Binomial mechanism-based updates contributed by multiple users to reduce the communication overhead/cost as well as to protect the privacy of {participating} users. However, the optimization of wireless transmission and quantization parameters (e.g., transmit power, bandwidth, and quantization bits) as well as the added noise while guaranteeing the privacy requirement and the performance of the learned FL model remains an open and challenging problem. In this paper, we aim to jointly optimize the level of quantization, parameters of the Binomial mechanism, and devices' transmit powers to minimize the training time under the constraints of the wireless networks. The resulting optimization turns out to be a Mixed Integer Non-linear Programming (MINLP) problem, which is known to be NP-hard. To tackle it, we transform this MINLP problem into a new problem whose solutions are proved to be the optimal solutions of the original one. We then propose an approximate algorithm that can solve the transformed problem with an arbitrary relative error guarantee. Intensive simulations show that for the same wireless resources the proposed approach achieves the highest accuracy, close to that of the standard FL with no quantization and no noise added. This suggests the faster convergence/training time of the proposed wireless FL framework while optimally preserving users' privacy.

AI-empowered Joint Communication and Radar Systems with Adaptive Waveform for Autonomous Vehicles

In Joint Communication and Radar (JCR)-based Autonomous Vehicle (AV) systems, optimizing waveform structure is one of the most challenging tasks due to strong influences between radar and data communication functions. Specifically, the preamble of a data communication frame is typically leveraged for the radar function. As such, the higher number of preambles in a Coherent Processing Interval (CPI) is, the greater radar's performance is. In contrast, communication efficiency decreases as the number of preambles increases. Moreover, AVs' surrounding radio environments are usually dynamic with high uncertainties due to their high mobility, making the JCR's waveform optimization problem even more challenging. To that end, this paper develops a novel JCR framework based on the Markov decision process framework and recent advanced techniques in deep reinforcement learning. By doing so, without requiring complete knowledge of the surrounding environment in advance, the JCR-AV can adaptively optimize its waveform structure (i.e., number of frames in the CPI) to maximize radar and data communication performance under the surrounding environment's dynamic and uncertainty. Extensive simulations show that our proposed approach can improve the joint communication and radar performance up to 46.26% compared with those of conventional methods (e.g., greedy policy- and fixed waveform-based approaches).

A Novel Covert Communication Method using Ambient Backscatter Communications

This paper introduces a novel solution to enable covert communication in wireless systems by using ambient backscatter communication technology. In the considered system, the original message at the transmitter is first divided into two parts: (i) active transmit message and (ii) backscatter message. Then, the active transmit message is transmitted by using the conventional wireless transmission method while the backscatter message is transmitted by backscattering the active transmit signals via an ambient backscatter tag. As the backscatter tag does not generate any active signals, it is intractable for the adversary to detect the backscatter message. Therefore, secret information, e.g., secret key for decryption, can be carried by the backscattered message, making the adversary unable to decode the original message. Simulation results demonstrate that our proposed solution can help to significantly enhance security protection for communication systems.

Joint Speed Control and Energy Replenishment Optimization for UAV-assisted IoT Data Collection with Deep Reinforcement Transfer Learning

Unmanned aerial vehicle (UAV)-assisted data collection has been emerging as a prominent application due to its flexibility, mobility, and low operational cost. However, under the dynamic and uncertainty of IoT data collection and energy replenishment processes, optimizing the performance for UAV collectors is a very challenging task. Thus, this paper introduces a novel framework that jointly optimizes the flying speed and energy replenishment for each UAV to significantly improve the data collection performance. Specifically, we first develop a Markov decision process to help the UAV automatically and dynamically make optimal decisions under the dynamics and uncertainties of the environment. We then propose a highly-effective reinforcement learning algorithm leveraging deep Q-learning, double deep Q-learning, and a deep dueling neural network architecture to quickly obtain the UAV's optimal policy. The core ideas of this algorithm are to estimate the state values and action advantages separately and simultaneously and to employ double estimators for estimating the action values. Thus, these proposed techniques can stabilize the learning process and effectively address the overestimation problem of conventional Q-learning algorithms. To further reduce the learning time as well as significantly improve learning quality, we develop advanced transfer learning techniques to allow UAVs to ``share'' and ``transfer'' learning knowledge. Extensive simulations demonstrate that our proposed solution can improve the average data collection performance of the system up to 200% compared with those of current methods.

Transfer Learning for Future Wireless Networks: A Comprehensive Survey

With outstanding features, Machine Learning (ML) has been the backbone of numerous applications in wireless networks. However, the conventional ML approaches have been facing many challenges in practical implementation, such as the lack of labeled data, the constantly changing wireless environments, the long training process, and the limited capacity of wireless devices. These challenges, if not addressed, will impede the effectiveness and applicability of ML in future wireless networks. To address these problems, Transfer Learning (TL) has recently emerged to be a very promising solution. The core idea of TL is to leverage and synthesize distilled knowledge from similar tasks as well as from valuable experiences accumulated from the past to facilitate the learning of new problems. Doing so, TL techniques can reduce the dependence on labeled data, improve the learning speed, and enhance the ML methods' robustness to different wireless environments. This article aims to provide a comprehensive survey on applications of TL in wireless networks. Particularly, we first provide an overview of TL including formal definitions, classification, and various types of TL techniques. We then discuss diverse TL approaches proposed to address emerging issues in wireless networks. The issues include spectrum management, localization, signal recognition, security, human activity recognition and caching, which are all important to next-generation networks such as 5G and beyond. Finally, we highlight important challenges, open issues, and future research directions of TL in future wireless networks.