Neural Networks are well known to have universal approximation properties for wide classes of Lebesgue integrable functions. We describe a collection of strategies and applications sourced from various fields of mathematics and physics to detail a rough blueprint for building efficient Neural Network differential equation solvers.