The nonlinearity and hysteresis of soft robot motions have posed challenges in accurate soft robot control. Neural networks, especially recurrent neural networks (RNNs), have been widely leveraged for this issue due to their nonlinear activation functions and recurrent structures. Although they have shown satisfying accuracy in most tasks, these black-box approaches are not explainable, and hence, they are unsuitable for areas with high safety requirements, like robot-assisted surgery. Based on the RNN controllers, we propose a data-driven explainable controller (DDEC) whose parameters can be updated online. We discuss the Jacobian controller and kinematics controller in theory and demonstrate that they are only special cases of DDEC. Moreover, we utilize RNN, the Jacobian controller, the kinematics controller, and DDECs for trajectory following tasks. Experimental results have shown that our approach outperforms the other controllers considering trajectory following errors while being explainable. We also conduct a study to explore and explain the functions of each DDEC component. This is the first interpretable soft robot controller that overcomes the shortcomings of both NN controllers and interpretable controllers. Future work may involve proposing different DDECs based on different RNN controllers and exploiting them for high-safety-required applications.