Neuro-symbolic partial differential equation solver

We present a highly scalable strategy for developing mesh-free neuro-symbolic partial differential equation solvers from existing numerical discretizations found in scientific computing. This strategy is unique in that it can be used to efficiently train neural network surrogate models for the solution functions and the differential operators, while retaining the accuracy and convergence properties of state-of-the-art numerical solvers. This neural bootstrapping method is based on minimizing residuals of discretized differential systems on a set of random collocation points with respect to the trainable parameters of the neural network, achieving unprecedented resolution and optimal scaling for solving physical and biological systems.

JAX-DIPS: Neural bootstrapping of finite discretization methods and application to elliptic problems with discontinuities

We present a scalable strategy for development of mesh-free hybrid neuro-symbolic partial differential equation solvers based on existing mesh-based numerical discretization methods. Particularly, this strategy can be used to efficiently train neural network surrogate models for the solution functions and operators of partial differential equations while retaining the accuracy and convergence properties of the state-of-the-art numerical solvers. The presented neural bootstrapping method (hereby dubbed NBM) is based on evaluation of the finite discretization residuals of the PDE system obtained on implicit Cartesian cells centered on a set of random collocation points with respect to trainable parameters of the neural network. We apply NBM to the important class of elliptic problems with jump conditions across irregular interfaces in three spatial dimensions. We show the method is convergent such that model accuracy improves by increasing number of collocation points in the domain. The algorithms presented here are implemented and released in a software package named JAX-DIPS (https://github.com/JAX-DIPS/JAX-DIPS), standing for differentiable interfacial PDE solver. JAX-DIPS is purely developed in JAX, offering end-to-end differentiability from mesh generation to the higher level discretization abstractions, geometric integrations, and interpolations, thus facilitating research into use of differentiable algorithms for developing hybrid PDE solvers.

Level-Set Curvature Neural Networks: A Hybrid Approach

We present a hybrid strategy based on deep learning to compute mean curvature in the level-set method. The proposed inference system combines a dictionary of improved regression models with standard numerical schemes to estimate curvature more accurately. The core of our framework is a switching mechanism that relies on well-established numerical techniques to gauge curvature. If the curvature magnitude is larger than a resolution-dependent threshold, it uses a neural network to yield a better approximation. Our networks are multi-layer perceptrons fitted to synthetic data sets composed of circular- and sinusoidal-interface samples at various configurations. To reduce data set size and training complexity, we leverage the problem's characteristic symmetry and build our models on just half of the curvature spectrum. These savings result in compact networks able to outperform any of the system's numerical or neural component alone. Experiments with static interfaces show that our hybrid approach is suitable and notoriously superior to conventional numerical methods in under-resolved and steep, concave regions. Compared to prior research, we have observed outstanding gains in precision after including training data pairs from more than a single interface type and other means of input preprocessing. In particular, our findings confirm that machine learning is a promising venue for devising viable solutions to the level-set method's numerical shortcomings.

A Deep Learning Approach for the Computation of Curvature in the Level-Set Method

We propose a deep learning strategy to compute the mean curvature of an implicit level-set representation of an interface. Our approach is based on fitting neural networks to synthetic datasets of pairs of nodal $\phi$ values and curvatures obtained from circular interfaces immersed in different uniform resolutions. These neural networks are multilayer perceptrons that ingest sample level-set values of grid points along a free boundary and output the dimensionless curvature at the center vertices of each sampled neighborhood. Evaluations with irregular (smooth and sharp) interfaces, in both uniform and adaptive meshes, show that our deep learning approach is systematically superior to conventional numerical approximation in the $L^2$ and $L^\infty$ norms. Our methodology is also less sensitive to steep curvatures and approximates them well with samples collected with fewer iterations of the reinitialization equation, often needed to regularize the underlying implicit function. Additionally, we show that an application-dependent map of local resolutions to neural networks can be constructed and employed to estimate interface curvatures more efficiently than using typically expensive numerical schemes while still attaining comparable or higher precision.

Solving inverse-PDE problems with physics-aware neural networks

We propose a novel composite framework that enables finding unknown fields in the context of inverse problems for partial differential equations (PDEs). We blend the high expressibility of deep neural networks as universal function estimators with the accuracy and reliability of existing numerical algorithms for partial differential equations. Our design brings together techniques of computational mathematics, machine learning and pattern recognition under one umbrella to seamlessly incorporate any domain-specific knowledge and insights through modeling. The network is explicitly aware of the governing physics through a hard-coded PDE solver stage; this subsequently focuses the computational load to only the discovery of the hidden fields. In addition, techniques of pattern recognition and surface reconstruction are used to further represent the unknown fields in a straightforward fashion. Most importantly, our inverse-PDE solver allows effortless integration of domain-specific knowledge about the physics of underlying fields, such as symmetries and proper basis functions. We call this approach Blended Inverse-PDE Networks (hereby dubbed BIPDE-Nets) and demonstrate its applicability on recovering the variable diffusion coefficient in Poisson problems in one and two spatial dimensions. We also show that this approach is robust to noise.