Two Tales of Platoon Intelligence for Autonomous Mobility Control: Enabling Deep Learning Recipes

This paper presents the deep learning-based recent achievements to resolve the problem of autonomous mobility control and efficient resource management of autonomous vehicles and UAVs, i.e., (i) multi-agent reinforcement learning (MARL), and (ii) neural Myerson auction. Representatively, communication network (CommNet), which is one of the most popular MARL algorithms, is introduced to enable multiple agents to take actions in a distributed manner for their shared goals by training all agents' states and actions in a single neural network. Moreover, the neural Myerson auction guarantees trustfulness among multiple agents as well as achieves the optimal revenue of highly dynamic systems. Therefore, we survey the recent studies on autonomous mobility control based on MARL and neural Myerson auction. Furthermore, we emphasize that integration of MARL and neural Myerson auction is expected to be critical for efficient and trustful autonomous mobility services.

Cooperative Multi-Agent Deep Reinforcement Learning for Reliable and Energy-Efficient Mobile Access via Multi-UAV Control

This paper addresses a novel multi-agent deep reinforcement learning (MADRL)-based multiple unmanned aerial vehicles (UAV) positioning algorithm for reliable mobile access services (i.e., UAVs work as mobile base stations), where the MADRL is designed by the concept of centralized training and distributed execution (CTDE) for multi-agent cooperation and coordination. The reliable mobile access services can be achieved in following two ways, i.e., (i) energy-efficient UAV operation and (ii) reliable wireless communication services. For energy-efficient UAV operation, the reward of our proposed MADRL algorithm contains the features for UAV energy consumption models in order to realize efficient operations. Furthermore, for reliable wireless communication services, the quality of service (QoS) requirements of individual users are considered as a part of rewards and 60GHz mmWave radio is used for mobile access. This paper considers the 60GHz mmWave access for utilizing the benefits of (i) ultra-wide-bandwidth for multi-Gbps high-speed communications and (ii) high-directional communications for spatial reuse that is obviously good for densely deployed users. Lastly, the performance of our proposed MADRL-based multi-UAV positioning algorithm is evaluated; and it can be confirmed that the proposed algorithm outperforms the other existing algorithms.

Spatio-Temporal Attack Course-of-Action (COA) Search Learning for Scalable and Time-Varying Networks

One of the key topics in network security research is the autonomous COA (Couse-of-Action) attack search method. Traditional COA attack search methods that passively search for attacks can be difficult, especially as the network gets bigger. To address these issues, new autonomous COA techniques are being developed, and among them, an intelligent spatial algorithm is designed in this paper for efficient operations in scalable networks. On top of the spatial search, a Monte-Carlo (MC)- based temporal approach is additionally considered for taking care of time-varying network behaviors. Therefore, we propose a spatio-temporal attack COA search algorithm for scalable and time-varying networks.

Tutorial on Course-of-Action Attack Search Methods in Computer Networks

In the literature of modern network security research, deriving effective and efficient course-of-action (COA) attach search methods are of interests in industry and academia. As the network size grows, the traditional COA attack search methods can suffer from the limitations to computing and communication resources. Therefore, various methods have been developed to solve these problems, and reinforcement learning (RL)-based intelligent algorithms are one of the most effective solutions. Therefore, we review the RL-based COA attack search methods for network attack scenarios in terms of the trends and their contrib

Neural Myerson Auction for Truthful and Energy-Efficient Autonomous Aerial Data Delivery

A successful deployment of drones provides an ideal solution for surveillance systems. Using drones for surveillance can provide access to areas that may be difficult or impossible to reach by humans or in-land vehicles gathering images or video recordings of a specific target in their coverage. Therefore, we introduces a data delivery drone to transfer collected surveillance data in harsh communication conditions. This paper proposes a Myerson auction-based asynchronous data delivery in an aerial distributed data platform in surveillance systems taking battery limitation and long flight constraints into account. In this paper, multiple delivery drones compete to offer data transfer to a single fixed-location surveillance drone. Our proposed Myerson auction-based algorithm, which uses the truthful second-price auction (SPA) as a baseline, is to maximize the seller's revenue while meeting several desirable properties, i.e., individual rationality and incentive compatibility while pursuing truthful operations. On top of these SPA-based operations, a deep learning-based framework is additionally designed for delivery performance improvements.

Distributed and Autonomous Aerial Data Collection in Smart City Surveillance Applications

The massive growth of Smart City and Internet of Things applications enables safety and security. The data those are produced from surveillance cameras in aerial devices such as unmanned aerial networks (UAVs) are needed to be transferred to ground stations for secure data analysis. When the scale of network is relatively large compare to the wireless communication coverage of device, it is not always available to transmit the data to the ground stations, thus distributed and autonomous algorithms are essentially desired. Based on the needs, we propose a novel algorithm that is for collecting surveillance data under the consideration of mobility and flexibility of UAV networks. Due to the battery limitation in UAVs, we selectively collect data from the UAVs by setting rules under the consideration of distance and similarity. As a sequence, the UAV devices have to compete for a chance to get data processing. For this purpose, this paper designs a Myerson auction-based deep learning algorithm to leverage the UAV's revenue compare to traditional second-price auction while preserving truthfulness. Based on simulation results, we verify that our proposed algorithm achieves desired performance improvements.