Collaborative robots and space manipulators contain significant joint flexibility. It complicates the control design, compromises the control bandwidth, and limits the tracking accuracy. The imprecise knowledge of the flexible joint dynamics compounds the challenge. In this paper, we present a new control architecture for controlling flexible-joint robots. Our approach uses a multi-layer neural network to approximate unknown dynamics needed for the feedforward control. The network may be viewed as a linear-in-parameter representation of the robot dynamics, with the nonlinear basis of the robot dynamics connected to the linear output layer. The output layer weights are updated based on the tracking error and the nonlinear basis. The internal weights of the nonlinear basis are updated by online backpropagation to further reduce the tracking error. To use time scale separation to reduce the coupling of the two steps - the update of the internal weights is at a lower rate compared to the update of the output layer weights. With the update of the output layer weights, our controller adapts quickly to the unknown dynamics change and disturbances (such as attaching a load). The update of the internal weights would continue to improve the converge of the nonlinear basis functions. We show the stability of the proposed scheme under the "outer loop" control, where the commanded joint position is considered as the control input. Simulation and physical experiments are conducted to demonstrate the performance of the proposed controller on a Baxter robot, which exhibits significant joint flexibility due to the series-elastic joint actuators.