Obstacle- and Occlusion-Responsive Visual Tracking Control for Redundant Manipulators using Reachability Measure

A vision system attached to a manipulator excels at tracing a moving target object while effectively handling obstacles, overcoming limitations arising from the camera's confined field of view and occluded line of sight. Meanwhile, the manipulator may encounter certain challenges, including restricted motion due to kinematic constraints and the risk of colliding with external obstacles. These challenges are typically addressed by assigning multiple task objectives to the manipulator. However, doing so can cause an increased risk of driving the manipulator to its kinematic limits, leading to failures in object tracking or obstacle avoidance. To address this issue, we propose a novel visual tracking control method for a redundant manipulator that takes the kinematic constraints into account via a reachability measure. Our method employs an optimization-based controller that considers object tracking, occlusion avoidance, collision avoidance, and the kinematic constraints represented by the reachability measure. Subsequently, it determines a suitable joint configuration through real-time inverse kinematics, accounting for dynamic obstacle avoidance and the continuity of joint configurations. To validate our approach, we conducted simulations and hardware experiments involving a moving target and dynamic obstacles. The results of our evaluations highlight the significance of incorporating the reachability measure.

TORM: Collision-Free Trajectory Optimization of Redundant Manipulator given an End-Effector Path

A redundant manipulator has multiple inverse kinematics solutions per an end-effector pose. Accordingly, there can be many trajectories for joints that follow a given end-effector path in a Cartesian space. In this paper, we present a trajectory optimization of a redundant manipulator (TORM) to synthesize a trajectory that follows a given end-effector path accurately, while achieving the smoothness and collision-free manipulation. Given these desirable properties, our method optimizes a trajectory using two-stage gradient descent to reduce potential competition between different properties during the update. To further improve the performance and avoid falling into local minima, we apply the quantum annealing that iteratively randomizes various configurations of the trajectory, followed by updating the trajectory. We first show benefits of our method with environments containing external obstacles. We then compare ours with the state-of-the-art methods in their favorable setting, environments without having obstacles. Our method robustly minimizes the pose error in a progressive manner while satisfying various desirable properties.

An Objectness Score for Accurate and Fast Detection during Navigation

We propose a novel method utilizing an objectness score for maintaining the locations and classes of objects detected from Mask R-CNN during mobile robot navigation. The objectness score is defined to measure how well the detector identifies the locations and classes of objects during navigation. Specifically, it is designed to increase when there is sufficient distance between a detected object and the camera. During the navigation process, we transform the locations of objects in 3D world coordinates into 2D image coordinates through an affine projection and decide whether to retain the classes of detected objects using the objectness score. We conducted experiments to determine how well the locations and classes of detected objects are maintained at various angles and positions. Experimental results showed that our approach is efficient and robust, regardless of changing angles and distances.

Harmonious Sampling for Mobile Manipulation Planning

Mobile manipulation planning commonly adopts a decoupled approach that performs planning separately on the base and the manipulator. While this approach is fast, it can generate sub-optimal paths. Another direction is a coupled approach jointly adjusting the base and manipulator in a high-dimensional configuration space. This coupled approach addresses sub-optimality and incompleteness of the decoupled approach, but has not been widely used due to its excessive computational overhead. Given this trade-off space, we present a simple, yet effective mobile manipulation sampling method, harmonious sampling, to perform the coupled approach mainly in difficult regions, where we need to simultaneously maneuver the base and the manipulator. Our method identifies such difficult regions through a low-dimensional base space by utilizing a reachability map given the target end-effector pose and narrow passage detected by generalized Voronoi diagram. For the rest of simple regions, we perform sampling mainly on the base configurations with a predefined joint configuration, accelerating the planning process. We compare our method with the decoupled and coupled approaches in six different problems with varying difficulty. Our method shows meaningful improvements experimentally in terms of time to find an initial solution (up to 5.6 times faster) and final solution cost (up to 17% lower) over the decoupled approach, especially in difficult scenes with narrow space. We also demonstrate these benefits with a real, mobile Hubo robot.