Joint Pedestrian and Vehicle Traffic Optimization in Urban Environments using Reinforcement Learning

Reinforcement learning (RL) holds significant promise for adaptive traffic signal control. While existing RL-based methods demonstrate effectiveness in reducing vehicular congestion, their predominant focus on vehicle-centric optimization leaves pedestrian mobility needs and safety challenges unaddressed. In this paper, we present a deep RL framework for adaptive control of eight traffic signals along a real-world urban corridor, jointly optimizing both pedestrian and vehicular efficiency. Our single-agent policy is trained using real-world pedestrian and vehicle demand data derived from Wi-Fi logs and video analysis. The results demonstrate significant performance improvements over traditional fixed-time signals, reducing average wait times per pedestrian and per vehicle by up to 67% and 52%, respectively, while simultaneously decreasing total accumulated wait times for both groups by up to 67% and 53%. Additionally, our results demonstrate generalization capabilities across varying traffic demands, including conditions entirely unseen during training, validating RL's potential for developing transportation systems that serve all road users.

Large-Scale Mixed-Traffic and Intersection Control using Multi-agent Reinforcement Learning

Traffic congestion remains a significant challenge in modern urban networks. Autonomous driving technologies have emerged as a potential solution. Among traffic control methods, reinforcement learning has shown superior performance over traffic signals in various scenarios. However, prior research has largely focused on small-scale networks or isolated intersections, leaving large-scale mixed traffic control largely unexplored. This study presents the first attempt to use decentralized multi-agent reinforcement learning for large-scale mixed traffic control in which some intersections are managed by traffic signals and others by robot vehicles. Evaluating a real-world network in Colorado Springs, CO, USA with 14 intersections, we measure traffic efficiency via average waiting time of vehicles at intersections and the number of vehicles reaching their destinations within a time window (i.e., throughput). At 80% RV penetration rate, our method reduces waiting time from 6.17 s to 5.09 s and increases throughput from 454 vehicles per 500 seconds to 493 vehicles per 500 seconds, outperforming the baseline of fully signalized intersections. These findings suggest that integrating reinforcement learning-based control large-scale traffic can improve overall efficiency and may inform future urban planning strategies.

MIAT: Maneuver-Intention-Aware Transformer for Spatio-Temporal Trajectory Prediction

Accurate vehicle trajectory prediction is critical for safe and efficient autonomous driving, especially in mixed traffic environments with both human-driven and autonomous vehicles. However, uncertainties introduced by inherent driving behaviors -- such as acceleration, deceleration, and left and right maneuvers -- pose significant challenges for reliable trajectory prediction. We introduce a Maneuver-Intention-Aware Transformer (MIAT) architecture, which integrates a maneuver intention awareness mechanism with spatiotemporal interaction modeling to enhance long-horizon trajectory predictions. We systematically investigate the impact of varying awareness of maneuver intention on both short- and long-horizon trajectory predictions. Evaluated on the real-world NGSIM dataset and benchmarked against various transformer- and LSTM-based methods, our approach achieves an improvement of up to 4.7% in short-horizon predictions and a 1.6% in long-horizon predictions compared to other intention-aware benchmark methods. Moreover, by leveraging an intention awareness control mechanism, MIAT realizes an 11.1% performance boost in long-horizon predictions, with a modest drop in short-horizon performance.

Mutual Adaptation in Human-Robot Co-Transportation with Human Preference Uncertainty

Mutual adaptation can significantly enhance overall task performance in human-robot co-transportation by integrating both the robot's and human's understanding of the environment. While human modeling helps capture humans' subjective preferences, two challenges persist: (i) the uncertainty of human preference parameters and (ii) the need to balance adaptation strategies that benefit both humans and robots. In this paper, we propose a unified framework to address these challenges and improve task performance through mutual adaptation. First, instead of relying on fixed parameters, we model a probability distribution of human choices by incorporating a range of uncertain human parameters. Next, we introduce a time-varying stubbornness measure and a coordination mode transition model, which allows either the robot to lead the team's trajectory or, if a human's preferred path conflicts with the robot's plan and their stubbornness exceeds a threshold, the robot to transition to following the human. Finally, we introduce a pose optimization strategy to mitigate the uncertain human behaviors when they are leading. To validate the framework, we design and perform experiments with real human feedback. We then demonstrate, through simulations, the effectiveness of our models in enhancing task performance with mutual adaptation and pose optimization.

Analyzing Fundamental Diagrams of Mixed Traffic Control at Unsignalized Intersections

This report examines the effect of mixed traffic, specifically the variation in robot vehicle (RV) penetration rates, on the fundamental diagrams at unsignalized intersections. Through a series of simulations across four distinct intersections, the relationship between traffic flow characteristics were analyzed. The RV penetration rates were varied from 0% to 100% in increments of 25%. The study reveals that while the presence of RVs influences traffic dynamics, the impact on flow and speed is not uniform across different levels of RV penetration. The fundamental diagrams indicate that intersections may experience an increase in capacity with varying levels of RVs, but this trend does not consistently hold as RV penetration approaches 100%. The variability observed across intersections suggests that local factors possibly influence the traffic flow characteristics. These findings highlight the complexity of integrating RVs into the existing traffic system and underscore the need for intersection-specific traffic management strategies to accommodate the transition towards increased RV presence.

Exploring Transformer-Augmented LSTM for Temporal and Spatial Feature Learning in Trajectory Prediction

Accurate vehicle trajectory prediction is crucial for ensuring safe and efficient autonomous driving. This work explores the integration of Transformer based model with Long Short-Term Memory (LSTM) based technique to enhance spatial and temporal feature learning in vehicle trajectory prediction. Here, a hybrid model that combines LSTMs for temporal encoding with a Transformer encoder for capturing complex interactions between vehicles is proposed. Spatial trajectory features of the neighboring vehicles are processed and goes through a masked scatter mechanism in a grid based environment, which is then combined with temporal trajectory of the vehicles. This combined trajectory data are learned by sequential LSTM encoding and Transformer based attention layers. The proposed model is benchmarked against predecessor LSTM based methods, including STA-LSTM, SA-LSTM, CS-LSTM, and NaiveLSTM. Our results, while not outperforming it's predecessor, demonstrate the potential of integrating Transformers with LSTM based technique to build interpretable trajectory prediction model. Future work will explore alternative architectures using Transformer applications to further enhance performance. This study provides a promising direction for improving trajectory prediction models by leveraging transformer based architectures, paving the way for more robust and interpretable vehicle trajectory prediction system.

Neighbor-Aware Reinforcement Learning for Mixed Traffic Optimization in Large-scale Networks

Managing mixed traffic comprising human-driven and robot vehicles (RVs) across large-scale networks presents unique challenges beyond single-intersection control. This paper proposes a reinforcement learning framework for coordinating mixed traffic across multiple interconnected intersections. Our key contribution is a neighbor-aware reward mechanism that enables RVs to maintain balanced distribution across the network while optimizing local intersection efficiency. We evaluate our approach using a real-world network, demonstrating its effectiveness in managing realistic traffic patterns. Results show that our method reduces average waiting times by 39.2% compared to the state-of-the-art single-intersection control policy and 79.8% compared to traditional traffic signals. The framework's ability to coordinate traffic across multiple intersections while maintaining balanced RV distribution provides a foundation for deploying learning-based solutions in urban traffic systems.

A Comprehensive Review on Traffic Datasets and Simulators for Autonomous Vehicles

Autonomous driving has rapidly developed and shown promising performance due to recent advances in hardware and deep learning techniques. High-quality datasets are fundamental for developing reliable autonomous driving algorithms. Previous dataset surveys either focused on a limited number or lacked detailed investigation of dataset characteristics. Besides, we analyze the annotation processes, existing labeling tools, and the annotation quality of datasets, showing the importance of establishing a standard annotation pipeline. On the other hand, we thoroughly analyze the impact of geographical and adversarial environmental conditions on the performance of autonomous driving systems. Moreover, we exhibit the data distribution of several vital datasets and discuss their pros and cons accordingly. Additionally, this paper provides a comprehensive analysis of publicly available traffic simulators. In addition to informing about traffic datasets, it is also the goal of this paper to provide context and information on the current capabilities of traffic simulators for their specific contributions to autonomous vehicle simulation and development. Furthermore, this paper discusses future directions and the increasing importance of synthetic data generation in simulators to enhance the training and creation of effective simulations. Finally, we discuss the current challenges and the development trend of future autonomous driving datasets.

Beacon: A Naturalistic Driving Dataset During Blackouts for Benchmarking Traffic Reconstruction and Control

Extreme weather events and other vulnerabilities are causing blackouts with increasing frequency, disrupting traffic control systems and posing significant challenges to urban mobility. To address this growing concern, we introduce \model{}, a naturalistic driving dataset collected during blackouts at complex intersections. Beacon provides detailed traffic data from two unsignalized intersections in Memphis, TN, including timesteps, origin, and destination lanes for each vehicle over four hours. We analyze traffic demand, vehicle trajectories, and density across different scenarios. We also use the dataset to reconstruct unsignalized, signalized and mixed traffic conditions, demonstrating its utility for benchmarking traffic reconstruction techniques and control methods. To the best of our knowledge, Beacon could be the first public available traffic dataset that captures naturalistic driving behaviors at complex intersections.

LASIL: Learner-Aware Supervised Imitation Learning For Long-term Microscopic Traffic Simulation

Microscopic traffic simulation plays a crucial role in transportation engineering by providing insights into individual vehicle behavior and overall traffic flow. However, creating a realistic simulator that accurately replicates human driving behaviors in various traffic conditions presents significant challenges. Traditional simulators relying on heuristic models often fail to deliver accurate simulations due to the complexity of real-world traffic environments. Due to the covariate shift issue, existing imitation learning-based simulators often fail to generate stable long-term simulations. In this paper, we propose a novel approach called learner-aware supervised imitation learning to address the covariate shift problem in multi-agent imitation learning. By leveraging a variational autoencoder simultaneously modeling the expert and learner state distribution, our approach augments expert states such that the augmented state is aware of learner state distribution. Our method, applied to urban traffic simulation, demonstrates significant improvements over existing state-of-the-art baselines in both short-term microscopic and long-term macroscopic realism when evaluated on the real-world dataset pNEUMA.