Virtual Reality via Object Poses and Active Learning: Realizing Telepresence Robots with Aerial Manipulation Capabilities

This article presents a novel telepresence system for advancing aerial manipulation in dynamic and unstructured environments. The proposed system not only features a haptic device, but also a virtual reality (VR) interface that provides real-time 3D displays of the robot's workspace as well as a haptic guidance to its remotely located operator. To realize this, multiple sensors namely a LiDAR, cameras and IMUs are utilized. For processing of the acquired sensory data, pose estimation pipelines are devised for industrial objects of both known and unknown geometries. We further propose an active learning pipeline in order to increase the sample efficiency of a pipeline component that relies on Deep Neural Networks (DNNs) based object detection. All these algorithms jointly address various challenges encountered during the execution of perception tasks in industrial scenarios. In the experiments, exhaustive ablation studies are provided to validate the proposed pipelines. Methodologically, these results commonly suggest how an awareness of the algorithms' own failures and uncertainty ("introspection") can be used tackle the encountered problems. Moreover, outdoor experiments are conducted to evaluate the effectiveness of the overall system in enhancing aerial manipulation capabilities. In particular, with flight campaigns over days and nights, from spring to winter, and with different users and locations, we demonstrate over 70 robust executions of pick-and-place, force application and peg-in-hole tasks with the DLR cable-Suspended Aerial Manipulator (SAM). As a result, we show the viability of the proposed system in future industrial applications.

Visual-Inertial Telepresence for Aerial Manipulation

This paper presents a novel telepresence system for enhancing aerial manipulation capabilities. It involves not only a haptic device, but also a virtual reality that provides a 3D visual feedback to a remotely-located teleoperator in real-time. We achieve this by utilizing onboard visual and inertial sensors, an object tracking algorithm and a pre-generated object database. As the virtual reality has to closely match the real remote scene, we propose an extension of a marker tracking algorithm with visual-inertial odometry. Both indoor and outdoor experiments show benefits of our proposed system in achieving advanced aerial manipulation tasks, namely grasping, placing, force exertion and peg-in-hole insertion.

A Nonlinear Observer for Free-Floating Target Motion using only Pose Measurements

In this paper, we design a nonlinear observer to estimate the inertial pose and the velocity of a free-floating non-cooperative satellite (Target) using only relative pose measurements. In the context of control design for orbital robotic capture of such a non-cooperative Target, due to lack of navigational aids, only a relative pose estimate may be obtained from slow-sampled and noisy exteroceptive sensors. The velocity, however, cannot be measured directly. To address this problem, we develop a model-based observer which acts as an internal model for Target kinematics/dynamics and therefore, may act as a predictor during periods of no measurement. To this end, firstly, we formalize the estimation problem on the SE(3) Lie group with different state and measurement spaces. Secondly, we develop the kinematics and dynamics observer such that the overall observer error dynamics possesses a stability property. Finally, the proposed observer is validated through robust Monte-Carlo simulations and experiments on a robotic facility.

Passive Compliance Control of Aerial Manipulators

This paper presents a passive compliance control for aerial manipulators to achieve stable environmental interactions. The main challenge is the absence of actuation along body-planar directions of the aerial vehicle which might be required during the interaction to preserve passivity. The controller proposed in this paper guarantees passivity of the manipulator through a proper choice of end-effector coordinates, and that of vehicle fuselage is guaranteed by exploiting time domain passivity technique. Simulation studies validate the proposed approach.