Neural quantum states are variational wave functions parameterised by artificial neural networks, a mathematical model studied for decades in the machine learning community. In the context of many-body physics, methods such as variational Monte Carlo with neural quantum states as variational wave functions are successful in approximating, with great accuracy, the ground-state of a quantum Hamiltonian. However, all the difficulties of proposing neural network architectures, along with exploring their expressivity and trainability, permeate their application as neural quantum states. In this paper, we consider the Feynman-Kitaev Hamiltonian for the transverse field Ising model, whose ground state encodes the time evolution of a spin chain at discrete time steps. We show how this ground state problem specifically challenges the neural quantum state trainability as the time steps increase because the true ground state becomes more entangled, and the probability distribution starts to spread across the Hilbert space. Our results indicate that the considered neural quantum states are capable of accurately approximating the true ground state of the system, i.e., they are expressive enough. However, extensive hyper-parameter tuning experiments point towards the empirical fact that it is poor trainability--in the variational Monte Carlo setup--that prevents a faithful approximation of the true ground state.