Neural Networks have been identified as powerful tools for the study of complex systems. A noteworthy example is the Neural Network Differential Equation (NN DE) solver, which can provide functional approximations to the solutions of a wide variety of differential equations. However, there is a lack of work on the role precise error quantification can play in their predictions: most variants focus on ambiguous and/or global measures of performance like the loss function. We address this in the context of dynamical system NN DE solvers, leveraging their \textit{learnt} information to develop more accurate and efficient solvers, while still pursuing an unsupervised approach that does not rely on external tools or data. We achieve this via methods that precisely quantify NN DE solver errors at local scales, thus allowing the user the capacity for efficient and targeted error correction. We exemplify the utility of our methods by testing them on a nonlinear and a chaotic system each.