Neural information field filter

We introduce neural information field filter, a Bayesian state and parameter estimation method for high-dimensional nonlinear dynamical systems given large measurement datasets. Solving such a problem using traditional methods, such as Kalman and particle filters, is computationally expensive. Information field theory is a Bayesian approach that can efficiently reconstruct dynamical model state paths and calibrate model parameters from noisy measurement data. To apply the method, we parameterize the time evolution state path using the span of a finite linear basis. The existing method has to reparameterize the state path by initial states to satisfy the initial condition. Designing an expressive yet simple linear basis before knowing the true state path is crucial for inference accuracy but challenging. Moreover, reparameterizing the state path using the initial state is easy to perform for a linear basis, but is nontrivial for more complex and expressive function parameterizations, such as neural networks. The objective of this paper is to simplify and enrich the class of state path parameterizations using neural networks for the information field theory approach. To this end, we propose a generalized physics-informed conditional prior using an auxiliary initial state. We show the existing reparameterization is a special case. We parameterize the state path using a residual neural network that consists of a linear basis function and a Fourier encoding fully connected neural network residual function. The residual function aims to correct the error of the linear basis function. To sample from the intractable posterior distribution, we develop an optimization algorithm, nested stochastic variational inference, and a sampling algorithm, nested preconditioned stochastic gradient Langevin dynamics. A series of numerical and experimental examples verify and validate the proposed method.

An information field theory approach to Bayesian state and parameter estimation in dynamical systems

Dynamical system state estimation and parameter calibration problems are ubiquitous across science and engineering. Bayesian approaches to the problem are the gold standard as they allow for the quantification of uncertainties and enable the seamless fusion of different experimental modalities. When the dynamics are discrete and stochastic, one may employ powerful techniques such as Kalman, particle, or variational filters. Practitioners commonly apply these methods to continuous-time, deterministic dynamical systems after discretizing the dynamics and introducing fictitious transition probabilities. However, approaches based on time-discretization suffer from the curse of dimensionality since the number of random variables grows linearly with the number of time-steps. Furthermore, the introduction of fictitious transition probabilities is an unsatisfactory solution because it increases the number of model parameters and may lead to inference bias. To address these drawbacks, the objective of this paper is to develop a scalable Bayesian approach to state and parameter estimation suitable for continuous-time, deterministic dynamical systems. Our methodology builds upon information field theory. Specifically, we construct a physics-informed prior probability measure on the function space of system responses so that functions that satisfy the physics are more likely. This prior allows us to quantify model form errors. We connect the system's response to observations through a probabilistic model of the measurement process. The joint posterior over the system responses and all parameters is given by Bayes' rule. To approximate the intractable posterior, we develop a stochastic variational inference algorithm. In summary, the developed methodology offers a powerful framework for Bayesian estimation in dynamical systems.