Reservoir History Matching of the Norne field with generative exotic priors and a coupled Mixture of Experts -- Physics Informed Neural Operator Forward Model

We developed a novel reservoir characterization workflow that addresses reservoir history matching by coupling a physics-informed neural operator (PINO) forward model with a mixture of experts' approach, termed cluster classify regress (CCR). The inverse modelling is achieved via an adaptive Regularized Ensemble Kalman inversion (aREKI) method, ideal for rapid inverse uncertainty quantification during history matching. We parametrize unknown permeability and porosity fields for non-Gaussian posterior measures using a variational convolution autoencoder and a denoising diffusion implicit model (DDIM) exotic priors. The CCR works as a supervised model with the PINO surrogate to replicate nonlinear Peaceman well equations. The CCR's flexibility allows any independent machine-learning algorithm for each stage. The PINO reservoir surrogate's loss function is derived from supervised data loss and losses from the initial conditions and residual of the governing black oil PDE. The PINO-CCR surrogate outputs pressure, water, and gas saturations, along with oil, water, and gas production rates. The methodology was compared to a standard numerical black oil simulator for a waterflooding case on the Norne field, showing similar outputs. This PINO-CCR surrogate was then used in the aREKI history matching workflow, successfully recovering the unknown permeability, porosity and fault multiplier, with simulations up to 6000 times faster than conventional methods. Training the PINO-CCR surrogate on an NVIDIA H100 with 80G memory takes about 5 hours for 100 samples of the Norne field. This workflow is suitable for ensemble-based approaches, where posterior density sampling, given an expensive likelihood evaluation, is desirable for uncertainty quantification.

A Novel A.I Enhanced Reservoir Characterization with a Combined Mixture of Experts -- NVIDIA Modulus based Physics Informed Neural Operator Forward Model

We have developed an advanced workflow for reservoir characterization, effectively addressing the challenges of reservoir history matching through a novel approach. This method integrates a Physics Informed Neural Operator (PINO) as a forward model within a sophisticated Cluster Classify Regress (CCR) framework. The process is enhanced by an adaptive Regularized Ensemble Kalman Inversion (aREKI), optimized for rapid uncertainty quantification in reservoir history matching. This innovative workflow parameterizes unknown permeability and porosity fields, capturing non-Gaussian posterior measures with techniques such as a variational convolution autoencoder and the CCR. Serving as exotic priors and a supervised model, the CCR synergizes with the PINO surrogate to accurately simulate the nonlinear dynamics of Peaceman well equations. The CCR approach allows for flexibility in applying distinct machine learning algorithms across its stages. Updates to the PINO reservoir surrogate are driven by a loss function derived from supervised data, initial conditions, and residuals of governing black oil PDEs. Our integrated model, termed PINO-Res-Sim, outputs crucial parameters including pressures, saturations, and production rates for oil, water, and gas. Validated against traditional simulators through controlled experiments on synthetic reservoirs and the Norne field, the methodology showed remarkable accuracy. Additionally, the PINO-Res-Sim in the aREKI workflow efficiently recovered unknown fields with a computational speedup of 100 to 6000 times faster than conventional methods. The learning phase for PINO-Res-Sim, conducted on an NVIDIA H100, was impressively efficient, compatible with ensemble-based methods for complex computational tasks.

A Novel Cluster Classify Regress Model Predictive Controller Formulation; CCR-MPC

In this work, we develop a novel data-driven model predictive controller using advanced techniques in the field of machine learning. The objective is to regulate control signals to adjust the desired internal room setpoint temperature, affected indirectly by the external weather states. The methodology involves developing a time-series machine learning model with either a Long Short Term Memory model (LSTM) or a Gradient Boosting Algorithm (XGboost), capable of forecasting this weather states for any desired time horizon and concurrently optimising the control signals to the desired set point. The supervised learning model for mapping the weather states together with the control signals to the room temperature is constructed using a previously developed methodology called Cluster Classify regress (CCR), which is similar in style but scales better to high dimensional dataset than the well-known Mixture-of-Experts. The overall method called CCR-MPC involves a combination of a time series model for weather states prediction, CCR for forwarding and any numerical optimisation method for solving the inverse problem. Forward uncertainty quantification (Forward-UQ) leans towards the regression model in the CCR and is attainable using a Bayesian deep neural network or a Gaussian process (GP). For this work, in the CCR modulation, we employ K-means clustering for Clustering, XGboost classifier for Classification and 5th order polynomial regression for Regression. Inverse UQ can also be obtained by using an I-ES approach for solving the inverse problem or even the well-known Markov chain Monte Carlo (MCMC) approach. The developed CCR-MPC is elegant, and as seen on the numerical experiments is able to optimise the controller to attain the desired setpoint temperature.

Ultra-fast Deep Mixtures of Gaussian Process Experts

Mixtures of experts have become an indispensable tool for flexible modelling in a supervised learning context, and sparse Gaussian processes (GP) have shown promise as a leading candidate for the experts in such models. In the present article, we propose to design the gating network for selecting the experts from such mixtures of sparse GPs using a deep neural network (DNN). This combination provides a flexible, robust, and efficient model which is able to significantly outperform competing models. We furthermore consider efficient approaches to computing maximum a posteriori (MAP) estimators of these models by iteratively maximizing the distribution of experts given allocations and allocations given experts. We also show that a recently introduced method called Cluster-Classify-Regress (CCR) is capable of providing a good approximation of the optimal solution extremely quickly. This approximation can then be further refined with the iterative algorithm.

4D Seismic History Matching Incorporating Unsupervised Learning

The work discussed and presented in this paper focuses on the history matching of reservoirs by integrating 4D seismic data into the inversion process using machine learning techniques. A new integrated scheme for the reconstruction of petrophysical properties with a modified Ensemble Smoother with Multiple Data Assimilation (ES-MDA) in a synthetic reservoir is proposed. The permeability field inside the reservoir is parametrised with an unsupervised learning approach, namely K-means with Singular Value Decomposition (K-SVD). This is combined with the Orthogonal Matching Pursuit (OMP) technique which is very typical for sparsity promoting regularisation schemes. Moreover, seismic attributes, in particular, acoustic impedance, are parametrised with the Discrete Cosine Transform (DCT). This novel combination of techniques from machine learning, sparsity regularisation, seismic imaging and history matching aims to address the ill-posedness of the inversion of historical production data efficiently using ES-MDA. In the numerical experiments provided, I demonstrate that these sparse representations of the petrophysical properties and the seismic attributes enables to obtain better production data matches to the true production data and to quantify the propagating waterfront better compared to more traditional methods that do not use comparable parametrisation techniques.

Cluster, Classify, Regress: A General Method For Learning Discountinous Functions

This paper presents a method for solving the supervised learning problem in which the output is highly nonlinear and discontinuous. It is proposed to solve this problem in three stages: (i) cluster the pairs of input-output data points, resulting in a label for each point; (ii) classify the data, where the corresponding label is the output; and finally (iii) perform one separate regression for each class, where the training data corresponds to the subset of the original input-output pairs which have that label according to the classifier. It has not yet been proposed to combine these 3 fundamental building blocks of machine learning in this simple and powerful fashion. This can be viewed as a form of deep learning, where any of the intermediate layers can itself be deep. The utility and robustness of the methodology is illustrated on some toy problems, including one example problem arising from simulation of plasma fusion in a tokamak.