Network-Aided Intelligent Traffic Steering in 6G ORAN: A Multi-Layer Optimization Framework

To enable an intelligent, programmable and multi-vendor radio access network (RAN) for 6G networks, considerable efforts have been made in standardization and development of open RAN (ORAN). So far, however, the applicability of ORAN in controlling and optimizing RAN functions has not been widely investigated. In this paper, we jointly optimize the flow-split distribution, congestion control and scheduling (JFCS) to enable an intelligent traffic steering application in ORAN. Combining tools from network utility maximization and stochastic optimization, we introduce a multi-layer optimization framework that provides fast convergence, long-term utility-optimality and significant delay reduction compared to the state-of-the-art and baseline RAN approaches. Our main contributions are three-fold: i) we propose the novel JFCS framework to efficiently and adaptively direct traffic to appropriate radio units; ii) we develop low-complexity algorithms based on the reinforcement learning, inner approximation and bisection search methods to effectively solve the JFCS problem in different time scales; and iii) the rigorous theoretical performance results are analyzed to show that there exists a scaling factor to improve the tradeoff between delay and utility-optimization. Collectively, the insights in this work will open the door towards fully automated networks with enhanced control and flexibility. Numerical results are provided to demonstrate the effectiveness of the proposed algorithms in terms of the convergence rate, long-term utility-optimality and delay reduction.

Edge Computing for Semantic Communication Enabled Metaverse: An Incentive Mechanism Design

Semantic communication (SemCom) and edge computing are two disruptive solutions to address emerging requirements of huge data communication, bandwidth efficiency and low latency data processing in Metaverse. However, edge computing resources are often provided by computing service providers and thus it is essential to design appealingly incentive mechanisms for the provision of limited resources. Deep learning (DL)- based auction has recently proposed as an incentive mechanism that maximizes the revenue while holding important economic properties, i.e., individual rationality and incentive compatibility. Therefore, in this work, we introduce the design of the DLbased auction for the computing resource allocation in SemComenabled Metaverse. First, we briefly introduce the fundamentals and challenges of Metaverse. Second, we present the preliminaries of SemCom and edge computing. Third, we review various incentive mechanisms for edge computing resource trading. Fourth, we present the design of the DL-based auction for edge resource allocation in SemCom-enabled Metaverse. Simulation results demonstrate that the DL-based auction improves the revenue while nearly satisfying the individual rationality and incentive compatibility constraints.

Deep Learning-Based Signal Detection for Dual-Mode Index Modulation 3D-OFDM

In this paper, we propose a deep learning-based signal detector called DuaIM-3DNet for dual-mode index modulation-based three-dimensional (3D) orthogonal frequency division multiplexing (DM-IM-3D-OFDM). Herein, DM-IM-3D- OFDM is a subcarrier index modulation scheme which conveys data bits via both dual-mode 3D constellation symbols and indices of active subcarriers. Thus, this scheme obtains better error performance than the existing IM schemes when using the conventional maximum likelihood (ML) detector, which, however, suffers from high computational complexity, especially when the system parameters increase. In order to address this fundamental issue, we propose the usage of a deep neural network (DNN) at the receiver to jointly and reliably detect both symbols and index bits of DM-IM-3D-OFDM under Rayleigh fading channels in a data-driven manner. Simulation results demonstrate that our proposed DNN detector achieves near-optimal performance at significantly lower runtime complexity compared to the ML detector.

Generalized BER of MCIK-OFDM with Imperfect CSI: Selection combining GD versus ML receivers

This paper analyzes the bit error rate (BER) of multicarrier index keying - orthogonal frequency division multiplexing (MCIK-OFDM) with selection combining (SC) diversity reception. Particularly, we propose a generalized framework to derive the BER for both the low-complexity greedy detector (GD) and maximum likelihood (ML) detector. Based on this, closedform expressions for the BERs of MCIK-OFDM with the SC using either the ML or the GD are derived in presence of the channel state information (CSI) imperfection. The asymptotic analysis is presented to gain helpful insights into effects of different CSI conditions on the BERs of these two detectors. More importantly, we theoretically provide opportunities for using the GD instead of the ML under each specific CSI uncertainty, which depend on the number of receiver antennas and the M-ary modulation size. Finally, extensive simulation results are provided in order to validate our theoretical expressions and analysis.

Enhancing Diversity of OFDM with Joint Spread Spectrum and Subcarrier Index Modulations

This paper proposes a novel spread spectrum and sub-carrier index modulation (SS-SIM) scheme, which is integrated to orthogonal frequency division multiplexing (OFDM) framework to enhance the diversity over the conventional IM schemes. Particularly, the resulting scheme, called SS-SIMOFDM, jointly employs both spread spectrum and sub-carrier index modulations to form a precoding vector which is then used to spread an M-ary complex symbol across all active sub-carriers. As a result, the proposed scheme enables a novel transmission of three signal domains: SS and sub-carrier indices, and a single M-ary symbol. For practical implementations, two reduced-complexity near-optimal detectors are proposed, which have complexities less depending on the M-ary modulation size. Then, the bit error probability and its upper bound are analyzed to gain an insight into the diversity gain, which is shown to be strongly affected by the order of sub-carrier indices. Based on this observation, we propose two novel sub-carrier index mapping methods, which significantly increase the diversity gain of SSSIM-OFDM. Finally, simulation results show that our scheme achieves better error performance than the benchmarks at the cost of lower spectral efficiency compared to classical OFDM and OFDM-IM, which can carry multiple M-ary symbols.

Incentive Mechanisms for Federated Learning: From Economic and Game Theoretic Perspective

Federated learning (FL) becomes popular and has shown great potentials in training large-scale machine learning (ML) models without exposing the owners' raw data. In FL, the data owners can train ML models based on their local data and only send the model updates rather than raw data to the model owner for aggregation. To improve learning performance in terms of model accuracy and training completion time, it is essential to recruit sufficient participants. Meanwhile, the data owners are rational and may be unwilling to participate in the collaborative learning process due to the resource consumption. To address the issues, there have been various works recently proposed to motivate the data owners to contribute their resources. In this paper, we provide a comprehensive review for the economic and game theoretic approaches proposed in the literature to design various schemes for stimulating data owners to participate in FL training process. In particular, we first present the fundamentals and background of FL, economic theories commonly used in incentive mechanism design. Then, we review applications of game theory and economic approaches applied for incentive mechanisms design of FL. Finally, we highlight some open issues and future research directions concerning incentive mechanism design of FL.

Applications of Multi-Agent Reinforcement Learning in Future Internet: A Comprehensive Survey

Future Internet involves several emerging technologies such as 5G and beyond 5G networks, vehicular networks, unmanned aerial vehicle (UAV) networks, and Internet of Things (IoTs). Moreover, future Internet becomes heterogeneous and decentralized with a large number of involved network entities. Each entity may need to make its local decision to improve the network performance under dynamic and uncertain network environments. Standard learning algorithms such as single-agent Reinforcement Learning (RL) or Deep Reinforcement Learning (DRL) have been recently used to enable each network entity as an agent to learn an optimal decision-making policy adaptively through interacting with the unknown environments. However, such an algorithm fails to model the cooperations or competitions among network entities, and simply treats other entities as a part of the environment that may result in the non-stationarity issue. Multi-agent Reinforcement Learning (MARL) allows each network entity to learn its optimal policy by observing not only the environments, but also other entities' policies. As a result, MARL can significantly improve the learning efficiency of the network entities, and it has been recently used to solve various issues in the emerging networks. In this paper, we thus review the applications of MARL in the emerging networks. In particular, we provide a tutorial of MARL and a comprehensive survey of applications of MARL in next generation Internet. In particular, we first introduce single-agent RL and MARL. Then, we review a number of applications of MARL to solve emerging issues in future Internet. The issues consist of network access, transmit power control, computation offloading, content caching, packet routing, trajectory design for UAV-aided networks, and network security issues.

Toward Smart Security Enhancement of Federated Learning Networks

As traditional centralized learning networks (CLNs) are facing increasing challenges in terms of privacy preservation, communication overheads, and scalability, federated learning networks (FLNs) have been proposed as a promising alternative paradigm to support the training of machine learning (ML) models. In contrast to the centralized data storage and processing in CLNs, FLNs exploit a number of edge devices (EDs) to store data and perform training distributively. In this way, the EDs in FLNs can keep training data locally, which preserves privacy and reduces communication overheads. However, since the model training within FLNs relies on the contribution of all EDs, the training process can be disrupted if some of the EDs upload incorrect or falsified training results, i.e., poisoning attacks. In this paper, we review the vulnerabilities of FLNs, and particularly give an overview of poisoning attacks and mainstream countermeasures. Nevertheless, the existing countermeasures can only provide passive protection and fail to consider the training fees paid for the contributions of the EDs, resulting in a unnecessarily high training cost. Hence, we present a smart security enhancement framework for FLNs. In particular, a verify-before-aggregate (VBA) procedure is developed to identify and remove the non-benign training results from the EDs. Afterward, deep reinforcement learning (DRL) is applied to learn the behaving patterns of the EDs and to actively select the EDs that can provide benign training results and charge low training fees. Simulation results reveal that the proposed framework can protect FLNs effectively and efficiently.

Resource Management for Blockchain-enabled Federated Learning: A Deep Reinforcement Learning Approach

Blockchain-enabled Federated Learning (BFL) enables mobile devices to collaboratively train neural network models required by a Machine Learning Model Owner (MLMO) while keeping data on the mobile devices. Then, the model updates are stored in the blockchain in a decentralized and reliable manner. However, the issue of BFL is that the mobile devices have energy and CPU constraints that may reduce the system lifetime and training efficiency. The other issue is that the training latency may increase due to the blockchain mining process. To address these issues, the MLMO needs to (i) decide how much data and energy that the mobile devices use for the training and (ii) determine the block generation rate to minimize the system latency, energy consumption, and incentive cost while achieving the target accuracy for the model. Under the uncertainty of the BFL environment, it is challenging for the MLMO to determine the optimal decisions. We propose to use the Deep Reinforcement Learning (DRL) to derive the optimal decisions for the MLMO.

Resource Allocation in Mobility-Aware Federated Learning Networks: A Deep Reinforcement Learning Approach

Federated learning allows mobile devices, i.e., workers, to use their local data to collaboratively train a global model required by the model owner. Federated learning thus addresses the privacy issues of traditional machine learning. However, federated learning faces the energy constraints of the workers and the high network resource cost due to the fact that a number of global model transmissions may be required to achieve the target accuracy. To address the energy constraint, a power beacon can be used that recharges energy to the workers. However, the model owner may need to pay an energy cost to the power beacon for the energy recharge. To address the high network resource cost, the model owner can use a WiFi channel, called default channel, for the global model transmissions. However, communication interruptions may occur due to the instability of the default channel quality. For this, special channels such as LTE channels can be used, but this incurs channel cost. As such, the problem of the model owner is to decide amounts of energy recharged to the workers and to choose channels used to transmit its global model to the workers to maximize the number of global model transmissions while minimizing the energy and channel costs. This is challenging for the model owner under the uncertainty of the channel, energy and mobility states of the workers. In this paper, we thus propose to employ the Deep Q-Network (DQN) that enables the model owner to find the optimal decisions on the energy and the channels without any a priori network knowledge. Simulation results show that the proposed DQN always achieves better performance compared to the conventional algorithms.