Semi-autonomous Teleoperation using Differential Flatness of a Crane Robot for Aircraft In-Wing Inspection

Visual inspection of confined spaces such as aircraft wings is ergonomically challenging for human mechanics. This work presents a novel crane robot that can travel the entire span of the aircraft wing, enabling mechanics to perform inspection from outside of the confined space. However, teleoperation of the crane robot can still be a challenge due to the need to avoid obstacles in the workspace and potential oscillations of the camera payload. The main contribution of this work is to exploit the differential flatness of the crane-robot dynamics for designing reduced-oscillation, collision-free time trajectories of the camera payload for use in teleoperation. Autonomous experiments verify the efficacy of removing undesired oscillations by 89%. Furthermore, teleoperation experiments demonstrate that the controller eliminated collisions (from 33% to 0%) when 12 participants performed an inspection task with the use of proposed trajectory selection when compared to the case without it. Moreover, even discounting the failures due to collisions, the proposed approach improved task efficiency by 18.7% when compared to the case without it.

Active Anomaly Detection in Confined Spaces Using Ergodic Traversal of Directed Region Graphs

We provide the first step toward developing a hierarchical control-estimation framework to actively plan robot trajectories for anomaly detection in confined spaces. The space is represented globally using a directed region graph, where a region is a landmark that needs to be visited (inspected). We devise a fast mixing Markov chain to find an ergodic route that traverses this graph so that the region visitation frequency is proportional to its anomaly detection uncertainty, while satisfying the edge directionality (region transition) constraint(s). Preliminary simulation results show fast convergence to the ergodic solution and confident estimation of the presence of anomalies in the inspected regions.

Communication-free Cohesive Flexible-Object Transport using Decentralized Robot Networks

Decentralized network theories focus on achieving consensus and in speeding up the rate of convergence to consensus. However, network cohesion (i.e., maintaining consensus) during transitions between consensus values is also important when transporting flexible structures. Deviations in the robot positions due to loss of cohesion when moving flexible structures from one position to another, such as uncuredcomposite aircraft wings, can cause large deformations, which in turn, can result in potential damage. The major contribution of this work is to develop a decentralized approach to transport flexible objects in a cohesive manner using local force measurements, without the need for additional communication between the robots. Additionally, stability conditions are developed for discrete-time implementation of the proposed cohesive transition approach, and experimental results are presented, which show that the proposed cohesive transportation approach can reduce the relative deformations by 85% when compared to the case without it.

MIMO ILC for Precision SEA robots using Input-weighted Complex-Kernel Regression

This work improves the positioning precision of lightweight robots with series elastic actuators (SEAs). Lightweight SEA robots, along with low-impedance control, can maneuver without causing damage in uncertain, confined spaces such as inside an aircraft wing during aircraft assembly. Nevertheless, substantial modeling uncertainties in SEA robots reduce the precision achieved by model-based approaches such as inversion-based feedforward. Therefore, this article improves the precision of SEA robots around specified operating points, through a multi-input multi-output (MIMO), iterative learning control (ILC) approach. The main contributions of this article are to (i) introduce an input-weighted complex kernel to estimate local MIMO models using complex Gaussian process regression (c-GPR) (ii) develop Ger\v{s}gorin-theorem-based conditions on the iteration gains for ensuring ILC convergence to precision within noise-related limits, even with errors in the estimated model; and (iii) demonstrate precision positioning with an experimental SEA robot. Comparative experimental results, with and without ILC, show around 90% improvement in the positioning precision (close to the repeatability limit of the robot) and a 10-times increase in the SEA robot's operating speed with the use of the MIMO ILC.

An Efficient Scheduling Algorithm for Multi-Robot Task Allocation in Assembling Aircraft Structures

Efficient utilization of cooperating robots in the assembly of aircraft structures relies on balancing the workload of the robots and ensuring collision-free scheduling. We cast this problem as that of allocating a large number of repetitive assembly tasks, such as drilling holes and installing fasteners, among multiple robots. Such task allocation is often formulated as a Traveling Salesman Problem (TSP), which is NP-hard, implying that computing an exactly optimal solution is computationally prohibitive for real-world applications. The problem complexity is further exacerbated by intermittent robot failures necessitating real-time task reallocation. In this letter, we present an efficient method that exploits workpart geometry and problem structure to initially generate balanced and conflict-free robot schedules under nominal conditions. Subsequently, we deal with the failures by allowing the robots to first complete their nominal schedules and then employing a market-based optimizer to allocate the leftover tasks. Results show an improvement of 13% in schedule efficiencies as compared to a greedy multi-agent scheduler on a four robot system, which is especially promising for aircraft assembly processes that take many hours to complete. Moreover, the computation times are similar and small, typically less than one second.

Predicting Ergonomic Risks During Indoor Object Manipulation Using Spatiotemporal Convolutional Networks

Automated real-time prediction of the ergonomic risks of manipulating objects is a key unsolved challenge in developing effective human-robot collaboration systems for logistics and manufacturing applications. We present a foundational paradigm to address this challenge by formulating the problem as one of action segmentation from RGB-D camera videos. Spatial features are first learned using a deep convolutional model from the video frames, which are then fed sequentially to temporal convolutional networks to semantically segment the frames into a hierarchy of actions, which are either ergonomically safe, require monitoring, or need immediate attention. For performance evaluation, in addition to an open-source kitchen dataset, we collected a new dataset comprising twenty individuals picking up and placing objects of varying weights to and from cabinet and table locations at various heights. Results show very high (87-94)% F1 overlap scores among the ground truth and predicted frame labels for videos lasting over two minutes and comprising a large number of actions.

Iterative Machine Learning for Precision Trajectory Tracking with Series Elastic Actuators

When robots operate in unknown environments small errors in postions can lead to large variations in the contact forces, especially with typical high-impedance designs. This can potentially damage the surroundings and/or the robot. Series elastic actuators (SEAs) are a popular way to reduce the output impedance of a robotic arm to improve control authority over the force exerted on the environment. However this increased control over forces with lower impedance comes at the cost of lower positioning precision and bandwidth. This article examines the use of an iteratively-learned feedforward command to improve position tracking when using SEAs. Over each iteration, the output responses of the system to the quantized inputs are used to estimate a linearized local system models. These estimated models are obtained using a complex-valued Gaussian Process Regression (cGPR) technique and then, used to generate a new feedforward input command based on the previous iteration's error. This article illustrates this iterative machine learning (IML) technique for a two degree of freedom (2-DOF) robotic arm, and demonstrates successful convergence of the IML approach to reduce the tracking error.

Uncertainty-based Arbitration of Human-Machine Shared Control

Manufacturing requires consistent production rate and task success for sustainable operation. Some manufacturing tasks require a semi-autonomous approach, exploiting the combination of human adaptability and machine precision and speed, to be cost effective. The main contribution of this paper is a new approach to determine the level of autonomy for human-machine shared control based on the automation uncertainty. Moreover, the haptic feedback is scaled by the level of autonomy to indicate machine confidence to the operator. Experimentation results, with a human-robot peg-in-a-hole testbed, show more than 5 times improvement in the error tolerance for task completion with the shared control approach when compared to a purely autonomous method.