A framework for training and benchmarking algorithms that schedule robot tasks

Service robots work in a changing environment habited by exogenous agents like humans. In the service robotics domain, lots of uncertainties result from exogenous actions and inaccurate localisation of objects and the robot itself. This makes the robot task scheduling problem incredibly challenging. In this article, we propose a benchmarking system for systematically assessing the performance of algorithms scheduling robot tasks. The robot environment incorporates a room map, furniture, transportable objects, and moving humans; the system defines interfaces for the algorithms, tasks to be executed, and evaluation methods. The system consists of several tools, easing testing scenario generation for training AI-based scheduling algorithms and statistical testing. For benchmarking purposes, a set of scenarios is chosen, and the performance of several scheduling algorithms is assessed. The system source is published to serve the community for tuning and comparable assessment of robot task scheduling algorithms for service robots.