Quantifying and Bridging the Fidelity Gap: A Decisive-Feature Approach to Comparing Synthetic and Real Imagery

Virtual testing using synthetic data has become a cornerstone of autonomous vehicle (AV) safety assurance. Despite progress in improving visual realism through advanced simulators and generative AI, recent studies reveal that pixel-level fidelity alone does not ensure reliable transfer from simulation to the real world. What truly matters is whether the system-under-test (SUT) bases its decisions on the same causal evidence in both real and simulated environments - not just whether images "look real" to humans. This paper addresses the lack of such a behavior-grounded fidelity measure by introducing Decisive Feature Fidelity (DFF), a new SUT-specific metric that extends the existing fidelity spectrum to capture mechanism parity - the agreement in causal evidence underlying the SUT's decisions across domains. DFF leverages explainable-AI (XAI) methods to identify and compare the decisive features driving the SUT's outputs for matched real-synthetic pairs. We further propose practical estimators based on counterfactual explanations, along with a DFF-guided calibration scheme to enhance simulator fidelity. Experiments on 2126 matched KITTI-VirtualKITTI2 pairs demonstrate that DFF reveals discrepancies overlooked by conventional output-value fidelity. Furthermore, results show that DFF-guided calibration improves decisive-feature and input-level fidelity without sacrificing output value fidelity across diverse SUTs.

A Vehicle-in-the-Loop Simulator with AI-Powered Digital Twins for Testing Automated Driving Controllers

Simulators are useful tools for testing automated driving controllers. Vehicle-in-the-loop (ViL) tests and digital twins (DTs) are widely used simulation technologies to facilitate the smooth deployment of controllers to physical vehicles. However, conventional ViL tests rely on full-size vehicles, requiring large space and high expenses. Also, physical-model-based DT suffers from the reality gap caused by modeling imprecision. This paper develops a comprehensive and practical simulator for testing automated driving controllers enhanced by scaled physical cars and AI-powered DT models. The scaled cars allow for saving space and expenses of simulation tests. The AI-powered DT models ensure superior simulation fidelity. Moreover, the simulator integrates well with off-the-shelf software and control algorithms, making it easy to extend. We use a filtered control benchmark with formal safety guarantees to showcase the capability of the simulator in validating automated driving controllers. Experimental studies are performed to showcase the efficacy of the simulator, implying its great potential in validating control solutions for autonomous vehicles and intelligent traffic.

A Learning-Based Model Predictive Contouring Control for Vehicle Evasive Manoeuvres

This paper presents a novel Learning-based Model Predictive Contouring Control (L-MPCC) algorithm for evasive manoeuvres at the limit of handling. The algorithm uses the Student-t Process (STP) to minimise model mismatches and uncertainties online. The proposed STP captures the mismatches between the prediction model and the measured lateral tyre forces and yaw rate. The mismatches correspond to the posterior means provided to the prediction model to improve its accuracy. Simultaneously, the posterior covariances are propagated to the vehicle lateral velocity and yaw rate along the prediction horizon. The STP posterior covariance directly depends on the variance of observed data, so its variance is more significant when the online measurements differ from the recorded ones in the training set and smaller in the opposite case. Thus, these covariances can be utilised in the L-MPCC's cost function to minimise the vehicle state uncertainties. In a high-fidelity simulation environment, we demonstrate that the proposed L-MPCC can successfully avoid obstacles, keeping the vehicle stable while driving a double lane change manoeuvre at a higher velocity than an MPCC without STP. Furthermore, the proposed controller yields a significantly lower peak sideslip angle, improving the vehicle's manoeuvrability compared to an L-MPCC with a Gaussian Process.

Awareness in robotics: An early perspective from the viewpoint of the EIC Pathfinder Challenge "Awareness Inside''

Consciousness has been historically a heavily debated topic in engineering, science, and philosophy. On the contrary, awareness had less success in raising the interest of scholars in the past. However, things are changing as more and more researchers are getting interested in answering questions concerning what awareness is and how it can be artificially generated. The landscape is rapidly evolving, with multiple voices and interpretations of the concept being conceived and techniques being developed. The goal of this paper is to summarize and discuss the ones among these voices connected with projects funded by the EIC Pathfinder Challenge called ``Awareness Inside'', a nonrecurring call for proposals within Horizon Europe designed specifically for fostering research on natural and synthetic awareness. In this perspective, we dedicate special attention to challenges and promises of applying synthetic awareness in robotics, as the development of mature techniques in this new field is expected to have a special impact on generating more capable and trustworthy embodied systems.

Model Validation of a Low-Speed and Reverse Driving Articulated Vehicle

For the autonomous operation of articulated vehicles at distribution centers, accurate positioning of the vehicle is of the utmost importance. Automation of these vehicle poses several challenges, e.g. large swept path, asymmetric steering response, large slide slip angles of non-steered trailer axles and trailer instability while reversing. Therefore, a validated vehicle model is required that accurately and efficiently predicts the states of the vehicle. Unlike forward driving, open-loop validation methods can not be used for reverse driving of articulated vehicles due to their unstable dynamics. This paper proposes an approach to stabilize the unstable pole of the system and compares three vehicle models (kinematic, non-linear single track and multibody dynamics model) against real-world test data obtained from low-speed experiments at a distribution center. It is concluded that single track non-linear model has a better performance in comparison to other models for large articulation angles and reverse driving maneuvers.

Model Predictive Contouring Control for Vehicle Obstacle Avoidance at the Limit of Handling

This paper proposes a non-linear Model Predictive Contouring Control (MPCC) for obstacle avoidance in automated vehicles driven at the limit of handling. The proposed controller integrates motion planning, path tracking and vehicle stability objectives, prioritising obstacle avoidance in emergencies. The controller's prediction model is a non-linear single-track vehicle model with the Fiala tyre to capture the vehicle's non-linear behaviour. The MPCC computes the optimal steering angle and brake torques to minimise tracking error in safe situations and maximise the vehicle-to-obstacle distance in emergencies. Furthermore, the MPCC is extended with the tyre friction circle to fully exploit the vehicle's manoeuvrability and stability. The MPCC controller is tested using real-time rapid prototyping hardware to prove its real-time capability. The performance is compared with a state-of-the-art Model Predictive Control (MPC) in a high-fidelity simulation environment. The double lane change scenario results demonstrate a significant improvement in successfully avoiding obstacles and maintaining vehicle stability.