Concepts for End-to-end Augmented Reality based Human-Robot Interaction Systems

The field of Augmented Reality (AR) based Human Robot Interaction (HRI) has progressed significantly since its inception more than two decades ago. With more advanced devices, particularly head-mounted displays (HMD), freely available programming environments and better connectivity, the possible application space expanded significantly. Here we present concepts and systems currently being developed at our lab to enable a truly end-to-end application of AR in HRI, from setting up the working environment of the robot, through programming and finally interaction with the programmed robot. Relevant papers by other authors will also be overviewed. We demonstrate the use of such technologies with systems not inherently designed to be collaborative, namely industrial manipulators. By trying to make such industrial systems easily-installable, collaborative and interactive, the vision of universal robot co-workers can be pushed one step closer to reality. The main goal of the paper is to provide a short overview of the capabilities of HMD-based HRI to researchers unfamiliar with the concepts. For researchers already using such techniques, the hope is to perhaps introduce some new ideas and to broaden the field of research.