Increasing the Task Flexibility of Heavy-Duty Manipulators Using Visual 6D Pose Estimation of Objects

Recent advances in visual 6D pose estimation of objects using deep neural networks have enabled novel ways of vision-based control for heavy-duty robotic applications. In this study, we present a pipeline for the precise tool positioning of heavy-duty, long-reach (HDLR) manipulators using advanced machine vision. A camera is utilized in the so-called eye-in-hand configuration to estimate directly the poses of a tool and a target object of interest (OOI). Based on the pose error between the tool and the target, along with motion-based calibration between the camera and the robot, precise tool positioning can be reliably achieved using conventional robotic modeling and control methods prevalent in the industry. The proposed methodology comprises orientation and position alignment based on the visually estimated OOI poses, whereas camera-to-robot calibration is conducted based on motion utilizing visual SLAM. The methods seek to avert the inaccuracies resulting from rigid-body--based kinematics of structurally flexible HDLR manipulators via image-based algorithms. To train deep neural networks for OOI pose estimation, only synthetic data are utilized. The methods are validated in a real-world setting using an HDLR manipulator with a 5 m reach. The experimental results demonstrate that an image-based average tool positioning error of less than 2 mm along the non-depth axes is achieved, which facilitates a new way to increase the task flexibility and automation level of non-rigid HDLR manipulators.

Mechanic Modeling and Nonlinear Optimal Control of Actively Articulated Suspension of Mobile Heavy-Duty Manipulators

This paper presents the analytic modeling of mobile heavy-duty manipulators with actively articulated suspension and its optimal control to maximize its static and dynamic stabilization. By adopting the screw theory formalism, we consider the suspension mechanism as a rigid multibody composed of two closed kinematic chains. This mechanical modeling allows us to compute the spatial inertial parameters of the whole platform as a function of the suspension's linear actuators through the articulated-body inertia method. Our solution enhances the computation accuracy of the wheels' reaction normal forces by providing an exact solution for the center of mass and inertia tensor of the mobile manipulator. Moreover, these inertial parameters and the normal forces are used to define metrics of both static and dynamic stability of the mobile manipulator and formulate a nonlinear programming problem that optimizes such metrics to generate an optimal stability motion that prevents the platform's overturning, such optimal position of the actuator is tracked with a state-feedback hydraulic valve control. We demonstrate our method's efficiency in terms of C++ computational speed, accuracy and performance improvement by simulating a 7 degrees-of-freedom heavy-duty parallel-serial mobile manipulator with four wheels and actively articulated suspension.

Energy-Cautious Designation of Kinematic Parameters for a Sustainable Parallel-Serial Heavy-Duty Manipulator Driven by Electromechanical Linear Actuator

Electrification, a key strategy in combating climate change, is transforming industries, and off-highway machines (OHM) will be next to transition from combustion engines and hydraulic actuation to sustainable fully electrified machines. Electromechanical linear actuators (EMLAs) offer superior efficiency, safety, and reduced maintenance, and they unlock vast potential for high-performance autonomous operations. However, a key challenge lies in optimizing the kinematic parameters of OHMs' on-board manipulators for EMLA integration to exploit the full capabilities of actuation systems and maximize their performance. This work addresses this challenge by delving into the structural optimization of a prevalent closed kinematic chain configuration commonly employed in OHM manipulators. Our approach aims to retain the manipulator's existing capabilities while reducing its energy expenditure, paving the way for a greener future in industrial automation, one in which sustainable and high-performing robotized OHMs can evolve. The feasibility of our methodology is validated through simulation results obtained on a commercially available parallel-serial heavy-duty manipulator mounted on a battery electric vehicle. The results demonstrate the efficacy of our approach in modifying kinematic parameters to facilitate the replacement of conventional hydraulic actuators with EMLAs, all while minimizing the overall energy consumption of the system.

Impact-Resilient Orchestrated Robust Controller for Heavy-duty Hydraulic Manipulators

Heavy-duty operations, typically performed using heavy-duty hydraulic manipulators (HHMs), are susceptible to environmental contact due to tracking errors or sudden environmental changes. Therefore, beyond precise control design, it is crucial that the manipulator be resilient to potential impacts without relying on contact-force sensors, which mostly cannot be utilized. This paper proposes a novel force-sensorless robust impact-resilient controller for a generic 6-degree-of-freedom (DoF) HHM constituting from anthropomorphic arm and spherical wrist mechanisms. The scheme consists of a neuroadaptive subsystem-based impedance controller, which is designed to ensure both accurate tracking of position and orientation with stabilization of HHMs upon contact, along with a novel generalized momentum observer, which is for the first time introduced in Pl\"ucker coordinate, to estimate the impact force. Finally, by leveraging the concepts of virtual stability and virtual power flow, the semi-global uniformly ultimately boundedness of the entire system is assured. To demonstrate the efficacy and versatility of the proposed method, extensive experiments were conducted using a generic 6-DoF industrial HHM. The experimental results confirm the exceptional performance of the designed method by achieving a subcentimeter tracking accuracy and by 80% reduction of impact of the contact.

Analytical Forward Dynamics Modeling of Linearly Actuated Heavy-Duty Parallel-Serial Manipulators

This paper presents a new geometric and recursive algorithm for analytically computing the forward dynamics of heavy-duty parallel-serial mechanisms. Our solution relies on expressing the dynamics of a class of linearly-actuated parallel mechanism to a lower dimensional dual Lie algebra to find an analytical solution for the inverse dynamics problem. Thus, by applying the articulated-body inertias method, we successfully provide analytic expressions for the total wrench in the linear-actuator reference frame, the linear acceleration of the actuator, and the total wrench exerted in the base reference frame of the closed loop. This new formulation allows to backwardly project and assemble inertia matrices and wrench bias of multiple closed-loops mechanisms. The final algorithm holds an O(n) algorithmic complexity, where $n$ is the number of degrees of freedom (DoF). We provide accuracy results to demonstrate its efficiency with 1-DoF closed-loop mechanism and 4-DoF manipulator composed by serial and parallel mechanisms. Additionally, we release a URDF multi-DoF code for this recursive algorithm.

Obstacle Avoidance Deep Reinforcement Learning-Based Trajectory Planner with Robust Low-Level Control for Robotic Manipulators

In robotics, contemporary strategies are learning-based, characterized by a complex black-box nature and a lack of interpretability, which may pose challenges in ensuring stability and safety. To address these issues, we propose integrating an obstacle-free deep reinforcement learning (DRL) trajectory planner with a novel auto-tuning low- and joint-level control strategy, all while actively engaging in the learning phase through interactions with the environment. This approach circumvents the complexities associated with computations while also addressing nonrepetitive and random obstacle avoidance tasks. First, a model-free DRL agent to plan velocity-bounded and obstacle-free motion is employed for a manipulator with 'n' degrees of freedom (DoF) in task space through joint-level reasoning. This plan is then input into a robust subsystem-based adaptive controller, which produces the necessary torques, while the Cuckoo Search Optimization (CSO) algorithm enhances control gains to minimize the time required to reach, time taken to stabilize, the maximum deviation from the desired value, and persistent tracking error in the steady state. This approach guarantees that position and velocity errors exponentially converge to zero in an unfamiliar environment, despite unknown robotic manipulator modeling. Theoretical assertions are validated through the presentation of simulation outcomes.

Orchestrated Robust Controller for the Precision Control of Heavy-duty Hydraulic Manipulators

Vast industrial investment along with increased academic research on hydraulic heavy-duty manipulators has unavoidably paved the way for their automatization, necessitating the design of robust and high-precision controllers. In this study, an orchestrated robust controller is designed to address the mentioned issue. To do so, the entire robotic system is decomposed into subsystems, and a robust controller is designed at each local subsystem by considering unknown model uncertainties, unknown disturbances, and compound input constraints, thanks to virtual decomposition control (VDC). As such, radial basic function neural networks (RBFNNs) are incorporated into VDC to tackle unknown disturbances and uncertainties, resulting in novel decentralized RBFNNs. All these robust local controllers designed at each local subsystem are, then, orchestrated to accomplish high-precision control. In the end, for the first time in the context of VDC, a semi-globally uniformly ultimate boundedness is achieved under the designed controller. The validity of the theoretical results is verified by performing extensive simulations and experiments on a 6-degrees-of-freedom industrial manipulator with a nominal lifting capacity of $600\, kg$ at $5$ meters reach. Comparing the simulation result to state-of-the-art controller along with provided experimental results, demonstrates that the proposed method established all the promises and performed excellently.

Exponential Auto-Tuning Fault-Tolerant Control of N Degrees-of-Freedom Manipulators Subject to Torque Constraints

This paper presents a novel auto-tuning subsystem-based fault-tolerant control (SBFC) system designed for robot manipulator systems with n degrees of freedom. It first employs an actuator fault model to account for various faults that may occur, and second, a mathematical saturation function is incorporated to address torque constraints. Subsequently, a novel robust subsystem-based adaptive control method is proposed to direct system states to follow desired trajectories closely in the presence of input constraints, unknown modeling errors, and actuator faults, which are primary considerations of the proposed system. This ensures uniform exponential stability and sustained performance. In addition, optimal values are identified by tuning the SBFC gains and customizing the JAYA algorithm (JA), a high-performance swarm intelligence technique. Theoretical assertions are validated through the presentation of simulation outcomes.

Nonlinear Subsystem-based Adaptive Impedance Control of Physical Human-Robot-Environment Interaction in Contact-rich Tasks

Haptic upper limb exoskeletons are robots that assist human operators during task execution while having the ability to render virtual or remote environments. Therefore, the stability of such robots in physical human-robot-environment interaction must be guaranteed, in addition to performing well during task execution. Having a wide range of Z-width, which shows the region of passively renderable impedance by a haptic display, is also important to render a wide range of virtual environments. To address these issues, in this study, subsystem-based adaptive impedance control is designed for having a stable human-robot-environment interaction of 7 degrees of freedom haptic exoskeleton. The presented control decomposes the entire system into subsystems and designs the controller at the subsystem level. The stability of the controller in the presence of contact with the virtual environment and human arm force is proved by employing the virtual stability concept. Additionally, the Z-width of the 7-DoF haptic exoskeleton is drawn using experimental data and improved using varying virtual mass element for the virtual environment. Finally, experimental results are provided to demonstrate the perfect performance of the proposed controller in accomplishing the predefined task.

Task Space Control of Robot Manipulators based on Visual SLAM

This paper aims to address the open problem of designing a globally stable vision-based controller for robot manipulators. Accordingly, based on a hybrid mechanism, this paper proposes a novel task-space control law attained by taking the gradient of a potential function in SE(3). The key idea is to employ the Visual Simultaneous Localization and Mapping (VSLAM) algorithm to estimate a robot pose. The estimated robot pose is then used in the proposed hybrid controller as feedback information. Invoking Barbalats lemma and Lyapunov's stability theorem, it is guaranteed that the resulting closed-loop system is globally asymptotically stable, which is the main accomplishment of the proposed structure. Simulation studies are conducted on a six degrees of freedom (6-DOF) robot manipulator to demonstrate the effectiveness and validate the performance of the proposed VSLAM-based control scheme.