Guidelines for evaluation of complex multi agent test scenarios

To support the testing of AVs, CETRAN has created a guideline for the evaluation of complex multi agent test scenarios presented in this report. This allows for a clear structured manner in evaluating complexity elements based on the corresponding difficulties an AV might encounter in Singapore traffic. This study aims to understand the source of complexity for AVs from traffic hazard, by breaking down the difficulties on AV capabilities as perception, situation awareness and decision-making. Guidelines created through this study are composed by a list of elements to be considered in the future as selection criteria to evaluate complexity of scenarios to support AV behaviour assessment. This study is intended to be a guide to understand the sources of complexity for Avs and can be used to challenge the risk management ability of autonomous vehicles in a scenario-based test approach or traffic situations faced on road trials. The report includes the usage of the guidelines created as application to evaluate the complexity of a set of 5 real events that occur on Singapore roads from Resembler webtool which is a database of real human accidents/incidents. Four scenarios were also designed for creation in simulation by the CETRAN team, applying the guidelines for complexity elements created in this work, to illustrate the difficulties an ADS could experience with such scenarios.

Behaviour Description Database for AVs in Singapore

A Technical Reference for Autonomous Vehicles (AVs), with part 1 focusing on basic behaviour guidelines (TR68-1) is published with the intent to be a reference for evaluation of appropriated behaviour on Autonomous Vehicles for Singapore. This is based on applicability from Basic Theory of Driving (BTD) and Final Theory of Driving (FTD) which are the traffic code/rules for human driving. This report contains a consolidation of current guidelines from TR68-1, BTD and FTD. It will allow an initial identification of missing guidelines for AV behaviour on roads; however, it is difficult to identify conflicting rules or gaps in guidance without going into identified traffic situations. Identified situations for analysis were chosen from Centre of Excellence for Testing & Research of Autonomous Vehicle (CETRAN) assessment experience for further investigation. The outcome of the report proposes additional behaviour characteristics and guidelines to situations identified to close the gap between assessors and developers on expected AV behaviour. These recommendations could improve current guidelines for AV behavioural in assessment and generally for the local AV ecosystem for urban tropical roads in Singapore. These recommendations could also serve as inputs for future TR 68-1 revisions where a sample set of reference situations can help to define clearer expectations or requirements for AV behaviour in those situations. This will help Singapore push forward in better definition of the expected AV behaviour for AV systems.