This work proposes an end-to-end differentiable physics engine for tensegrity robots, which introduces a data-efficient linear contact model for accurately predicting collision responses that arise due to contacting surfaces, and a linear actuator model that can drive these robots by expanding and contracting their flexible cables. To the best of the authors' knowledge, this is the \emph{first} differentiable physics engine for tensegrity robots that supports cable modeling, contact, and actuation. This engine can be used inside an off-the-shelf, RL-based locomotion controller in order to provide training examples. This paper proposes a progressive training pipeline for the differentiable physics engine that helps avoid local optima during the training phase and reduces data requirements. It demonstrates the data-efficiency benefits of using the differentiable engine for learning locomotion policies for NASA's icosahedron SUPERballBot. In particular, after the engine has been trained with few trajectories to match a ground truth simulated model, then a policy learned on the differentiable engine is shown to be transferable back to the ground-truth model. Training the controller requires orders of magnitude more data than training the differential engine.