Trajectory Optimization Using Neural Network Gradients of Learned Dynamics

Trajectory optimization methods have achieved an exceptional level of performance on real-world robots in recent years. These methods heavily rely on accurate physics simulators, yet some aspects of the physical world, such as friction, can only be captured to a limited extent by most simulators. The goal of this paper is to leverage trajectory optimization for performing highly dynamic and complex tasks with robotic systems in absence of an accurate physics simulator. This is achieved by applying machine learning techniques to learn a differentiable dynamics model of the system from data. On the example of a RC car, we show that from data collected in only 15 minutes of human-operated interactions with the car, a neural network is able to model highly nonlinear behaviors such as loss of traction and drifting. Furthermore, we use the analytical gradients of the neural network to perform gradient-based trajectory optimization, both in an offline and online setting. We find that our learned model is able to represent complex physical behavior, like drifting and gives unprecedented performance in combination with trajectory optimization methods.