Large Scale Robotic Material Handling: Learning, Planning, and Control

Bulk material handling involves the efficient and precise moving of large quantities of materials, a core operation in many industries, including cargo ship unloading, waste sorting, construction, and demolition. These repetitive, labor-intensive, and safety-critical operations are typically performed using large hydraulic material handlers equipped with underactuated grippers. In this work, we present a comprehensive framework for the autonomous execution of large-scale material handling tasks. The system integrates specialized modules for environment perception, pile attack point selection, path planning, and motion control. The main contributions of this work are two reinforcement learning-based modules: an attack point planner that selects optimal grasping locations on the material pile to maximize removal efficiency and minimize the number of scoops, and a robust trajectory following controller that addresses the precision and safety challenges associated with underactuated grippers in movement, while utilizing their free-swinging nature to release material through dynamic throwing. We validate our framework through real-world experiments on a 40 t material handler in a representative worksite, focusing on two key tasks: high-throughput bulk pile management and high-precision truck loading. Comparative evaluations against human operators demonstrate the system's effectiveness in terms of precision, repeatability, and operational safety. To the best of our knowledge, this is the first complete automation of material handling tasks on a full scale.

Reinforcement Learning Control for Autonomous Hydraulic Material Handling Machines with Underactuated Tools

The precise and safe control of heavy material handling machines presents numerous challenges due to the hard-to-model hydraulically actuated joints and the need for collision-free trajectory planning with a free-swinging end-effector tool. In this work, we propose an RL-based controller that commands the cabin joint and the arm simultaneously. It is trained in a simulation combining data-driven modeling techniques with first-principles modeling. On the one hand, we employ a neural network model to capture the highly nonlinear dynamics of the upper carriage turn hydraulic motor, incorporating explicit pressure prediction to handle delays better. On the other hand, we model the arm as velocity-controllable and the free-swinging end-effector tool as a damped pendulum using first principles. This combined model enhances our simulation environment, enabling the training of RL controllers that can be directly transferred to the real machine. Designed to reach steady-state Cartesian targets, the RL controller learns to leverage the hydraulic dynamics to improve accuracy, maintain high speeds, and minimize end-effector tool oscillations. Our controller, tested on a mid-size prototype material handler, is more accurate than an inexperienced operator and causes fewer tool oscillations. It demonstrates competitive performance even compared to an experienced professional driver.

Dynamic Throwing with Robotic Material Handling Machines

Automation of hydraulic material handling machinery is currently limited to semi-static pick-and-place cycles. Dynamic throwing motions which utilize the passive joints, can greatly improve time efficiency as well as increase the dumping workspace. In this work, we use Reinforcement Learning (RL) to design dynamic controllers for material handlers with underactuated arms as commonly used in logistics. The controllers are tested both in simulation and in real-world experiments on a 12-ton test platform. The method is able to exploit the passive joints of the gripper to perform dynamic throwing motions. With the proposed controllers, the machine is able to throw individual objects to targets outside the static reachability zone with good accuracy for its practical applications. The work demonstrates the possibility of using RL to perform highly dynamic tasks with heavy machinery, suggesting a potential for improving the efficiency and precision of autonomous material handling tasks.

A Unified and Modular Model Predictive Control Framework for Soft Continuum Manipulators under Internal and External Constraints

The emerging field of fluidically actuated soft robotic control has promising capabilities such as inherent compliance and user safety. However, these are counterbalanced by issues not common to rigid robots, like nonlinear actuation dynamics, motion constraints, workspace limitations, and variable shape stiffness. In this work, we have adapted Model Predictive Control (MPC), that has recently seen an exponential rise in popularity and fields of applications, to a soft robotic arm called SoPrA. We have addressed the problems that current control methods are facing, trying to propose a unique environment to handle them in a modular way. This work shows, both with simulation and experimental results, that Task-Space MPC can be successfully implemented for dynamic soft robotic control, while past research has usually focused on Joint-Space references. We have provided a way to couple the Piece-wise Constant Curvature and the Augmented Rigid Body Model assumptions with internal and external constraints and actuation dynamics, delivering an algorithm that can manage all these information and optimize over them. We believe that an MPC implementation based on our approach could be the way to address most of model-based soft robotics control issues within a unified and modular framework, while allowing to include improvements that usually belong to other control domains such as learning techniques.