Multi-Task Lane-Free Driving Strategy for Connected and Automated Vehicles: A Multi-Agent Deep Reinforcement Learning Approach

Deep reinforcement learning has shown promise in various engineering applications, including vehicular traffic control. The non-stationary nature of traffic, especially in the lane-free environment with more degrees of freedom in vehicle behaviors, poses challenges for decision-making since a wrong action might lead to a catastrophic failure. In this paper, we propose a novel driving strategy for Connected and Automated Vehicles (CAVs) based on a competitive Multi-Agent Deep Deterministic Policy Gradient approach. The developed multi-agent deep reinforcement learning algorithm creates a dynamic and non-stationary scenario, mirroring real-world traffic complexities and making trained agents more robust. The algorithm's reward function is strategically and uniquely formulated to cover multiple vehicle control tasks, including maintaining desired speeds, overtaking, collision avoidance, and merging and diverging maneuvers. Moreover, additional considerations for both lateral and longitudinal passenger comfort and safety criteria are taken into account. We employed inter-vehicle forces, known as nudging and repulsive forces, to manage the maneuvers of CAVs in a lane-free traffic environment. The proposed driving algorithm is trained and evaluated on lane-free roads using the Simulation of Urban Mobility platform. Experimental results demonstrate the algorithm's efficacy in handling different objectives, highlighting its potential to enhance safety and efficiency in autonomous driving within lane-free traffic environments.

Temporal Enhanced Floating Car Observers

Floating Car Observers (FCOs) are an innovative method to collect traffic data by deploying sensor-equipped vehicles to detect and locate other vehicles. We demonstrate that even a small penetration rate of FCOs can identify a significant amount of vehicles at a given intersection. This is achieved through the emulation of detection within a microscopic traffic simulation. Additionally, leveraging data from previous moments can enhance the detection of vehicles in the current frame. Our findings indicate that, with a 20-second observation window, it is possible to recover up to 20\% of vehicles that are not visible by FCOs in the current timestep. To exploit this, we developed a data-driven strategy, utilizing sequences of Bird's Eye View (BEV) representations of detected vehicles and deep learning models. This approach aims to bring currently undetected vehicles into view in the present moment, enhancing the currently detected vehicles. Results of different spatiotemporal architectures show that up to 41\% of the vehicles can be recovered into the current timestep at their current position. This enhancement enriches the information initially available by the FCO, allowing an improved estimation of traffic states and metrics (e.g. density and queue length) for improved implementation of traffic management strategies.

Unlocking Past Information: Temporal Embeddings in Cooperative Bird's Eye View Prediction

Accurate and comprehensive semantic segmentation of Bird's Eye View (BEV) is essential for ensuring safe and proactive navigation in autonomous driving. Although cooperative perception has exceeded the detection capabilities of single-agent systems, prevalent camera-based algorithms in cooperative perception neglect valuable information derived from historical observations. This limitation becomes critical during sensor failures or communication issues as cooperative perception reverts to single-agent perception, leading to degraded performance and incomplete BEV segmentation maps. This paper introduces TempCoBEV, a temporal module designed to incorporate historical cues into current observations, thereby improving the quality and reliability of BEV map segmentations. We propose an importance-guided attention architecture to effectively integrate temporal information that prioritizes relevant properties for BEV map segmentation. TempCoBEV is an independent temporal module that seamlessly integrates into state-of-the-art camera-based cooperative perception models. We demonstrate through extensive experiments on the OPV2V dataset that TempCoBEV performs better than non-temporal models in predicting current and future BEV map segmentations, particularly in scenarios involving communication failures. We show the efficacy of TempCoBEV and its capability to integrate historical cues into the current BEV map, improving predictions under optimal communication conditions by up to 2% and under communication failures by up to 19%. The code will be published on GitHub.

Autonomous Vehicles as a Sensor: Simulating Data Collection Process

Urban traffic state estimation is pivotal in furnishing precise and reliable insights into traffic flow characteristics, thereby enabling efficient traffic management. Traditional traffic estimation methodologies have predominantly hinged on labor-intensive and costly techniques such as loop detectors and floating car data. Nevertheless, the relentless progression in autonomous driving technology has catalyzed an increasing interest in capitalizing on the extensive potential of on-board sensor data, giving rise to a novel concept known as "Autonomous Vehicles as a Sensor" (AVaaS). This paper innovatively refines the AVaaS concept by simulating the data collection process. We take real-world sensor attributes into account and employ more accurate estimation techniques based on the on-board sensor data. Such data can facilitate the estimation of high-resolution, link-level traffic states and, more extensively, online cluster- and network-level traffic states. We substantiate the viability of the AVaaS concept through a case study conducted using a real-world traffic simulation in Ingolstadt, Germany. The results attest to the ability of AVaaS in estimating both microscopic (link-level) and macroscopic (cluster- and network-level) traffic states, thereby highlighting the immense potential of the AVaaS concept in effecting precise and reliable traffic state estimation and also further applications.

Multi-Sensor Data Fusion for Accurate Traffic Speed and Travel Time Reconstruction

This paper studies the joint reconstruction of traffic speeds and travel times by fusing sparse sensor data. Raw speed data from inductive loop detectors and floating cars as well as travel time measurements are combined using different fusion techniques. A novel fusion approach is developed which extends existing speed reconstruction methods to integrate low-resolution travel time data. Several state-of-the-art methods and the novel approach are evaluated on their performance in reconstructing traffic speeds and travel times using various combinations of sensor data. Algorithms and sensor setups are evaluated with real loop detector, floating car and Bluetooth data collected during severe congestion on German freeway A9. Two main aspects are examined: (i) which algorithm provides the most accurate result depending on the used data and (ii) which type of sensor and which combination of sensors yields higher estimation accuracies. Results show that, overall, the novel approach applied to a combination of floating-car data and loop data provides the best speed and travel time accuracy. Furthermore, a fusion of sources improves the reconstruction quality in many, but not all cases. In particular, Bluetooth data only provide a benefit for reconstruction purposes if integrated distinctively.

Estimating Traffic Speeds using Probe Data: A Deep Neural Network Approach

This paper presents a dedicated Deep Neural Network (DNN) architecture that reconstructs space-time traffic speeds on freeways given sparse data. The DNN is constructed in such a way, that it learns heterogeneous congestion patterns using a large dataset of sparse speed data, in particular from probe vehicles. Input to the DNN are two equally sized input matrices: one containing raw measurement data, and the other indicates the cells occupied with data. The DNN, comprising multiple stacked convolutional layers with an encoding-decoding structure and feed-forward paths, transforms the input into a full matrix of traffic speeds. The proposed DNN architecture is evaluated with respect to its ability to accurately reconstruct heterogeneous congestion patterns under varying input data sparsity. Therefore, a large set of empirical Floating-Car Data (FCD) collected on German freeway A9 during two months is utilized. In total, 43 congestion distinct scenarios are observed which comprise moving and stationary congestion patterns. A data augmentation technique is applied to generate input-output samples of the data, which makes the DNN shift-invariant as well as capable of managing varying data sparsities. The DNN is trained and subsequently applied to sparse data of an unseen congestion scenario. The results show that the DNN is able to apply learned patterns, and reconstructs moving as well as stationary congested traffic with high accuracy; even given highly sparse input data. Reconstructed speeds are compared qualitatively and quantitatively with the results of several state-of-the-art methods such as the Adaptive Smoothing Method (ASM), the Phase-Based Smoothing Method (PSM) and a standard Convolutional Neural Network (CNN) architecture. As a result, the DNN outperforms the other methods significantly.

Feature Engineering for Data-driven Traffic State Forecast in Urban Road Networks

Most traffic state forecast algorithms when applied to urban road networks consider only the links in close proximity to the target location. However, for longer-term forecasts also the traffic state of more distant links or regions of the network are expected to provide valuable information for a data-driven algorithm. This paper studies these expectations of using a network clustering algorithm and one year of Floating Car (FCD) collected by a large fleet of vehicles. First, a clustering algorithm is applied to the data in order to extract congestion-prone regions in the Munich city network. The level of congestion inside these clusters is analyzed with the help of statistical tools. Clear spatio-temporal congestion patterns and correlations between the clustered regions are identified. These correlations are integrated into a K- Nearest Neighbors (KNN) travel time prediction algorithm. In a comparison with other approaches, this method achieves the best results. The statistical results and the performance of the KNN predictor indicate that the consideration of the network-wide traffic is a valuable feature for predictors and a promising way to develop more accurate algorithms in the future.