Learning Quadrupedal Locomotion with Impaired Joints Using Random Joint Masking

Quadrupedal robots have played a crucial role in various environments, from structured environments to complex harsh terrains, thanks to their agile locomotion ability. However, these robots can easily lose their locomotion functionality if damaged by external accidents or internal malfunctions. In this paper, we propose a novel deep reinforcement learning framework to enable a quadrupedal robot to walk with impaired joints. The proposed framework consists of three components: 1) a random joint masking strategy for simulating impaired joint scenarios, 2) a joint state estimator to predict an implicit status of current joint condition based on past observation history, and 3) progressive curriculum learning to allow a single network to conduct both normal gait and various joint-impaired gaits. We verify that our framework enables the Unitree's Go1 robot to walk under various impaired joint conditions in real-world indoor and outdoor environments.

Differential Spiral Joint Mechanism for Coupled Variable Stiffness Actuation

In this study, we present the Differential Spiral Joint (DSJ) mechanism for variable stiffness actuation in tendon-driven robots. The DSJ mechanism semi-decouples the modulation of position and mechanical stiffness, allowing independent trajectory tracking in different parameter space. Past studies show that increasing the mechanical stiffness achieves the wider range of renderable stiffness, whereas decreasing the mechanical stiffness improves the quality of actuator decoupling and shock absorbance. Therefore, it is often useful to modulate the mechanical stiffness to balance the required level of stiffness and safety. In addition, the DSJ mechanism offers a compact form factor, which is suitable for applications where the size and weight are important. The performance of the DSJ mechanism in various areas is validated through a set of experiments.

SCAPE: Learning Stiffness Control from Augmented Position Control Experiences

We introduce a sample-efficient method for learning state-dependent stiffness control policies for dexterous manipulation. The ability to control stiffness facilitates safe and reliable manipulation by providing compliance and robustness to uncertainties. So far, most current reinforcement learning approaches to achieve robotic manipulation have exclusively focused on position control, often due to the difficulty of learning high-dimensional stiffness control policies. This difficulty can be partially mitigated via policy guidance such as in imitation learning. However, expert stiffness control demonstrations are often expensive or infeasible to record. Therefore, we present an approach to learn Stiffness Control from Augmented Position control Experiences (SCAPE) that bypasses this difficulty by transforming position control demonstrations into approximate, suboptimal stiffness control demonstrations. Then, the suboptimality of the augmented demonstrations is addressed by using complementary techniques that help the agent safely learn from both the demonstrations and reinforcement learning. By using simulation tools and experiments on a robotic testbed, we show that the proposed approach efficiently learns safe manipulation policies and outperforms learned position control policies and several other baseline learning algorithms.