Closing the Sim2Real Gap in Dynamic Cloth Manipulation

Cloth manipulation is a challenging task due to the many degrees of freedom and properties of the material affecting the dynamics of the cloth. The nonlinear dynamics of the cloth have particularly strong significance in dynamic cloth manipulation, where some parts of the cloth are not directly controllable. In this paper, we present a novel approach for solving dynamic cloth manipulation by training policies using reinforcement learning (RL) in simulation and transferring the learned policies to the real world in a zero-shot manner. The proposed method uses visual feedback and material property randomization in a physics simulator to achieve generalization in the real world. Experimental results show that using only visual feedback is enough for the policies to learn the dynamic manipulation task in a way that transfers from simulation to the real world. In addition, the randomization of the dynamics in simulation enables capturing the behavior of a variety of cloths in the real world.