Neural Network Verification through Replication

A system identification based approach to neural network model replication is presented and the application of model replication to verification of fundamental, single hidden layer, neural network systems is demonstrated. The presented approach serves as a means to partially address the problem of verifying that a neural network implementation meets a provided specification given only grey-box access to the implemented network. The procedure developed involves stimulating a neural network with a chosen signal, extracting a replicated model from the response, and systematically checking that the replicated model is output-equivalent to a specified model in order to verify that the grey-box system under test is implemented to specification without direct access to its hidden parameters. The replication step is introduced to provide an inherent guarantee that the stimulus signals employed yield sufficient test coverage. This method is investigated as a neural network focused nonlinear counterpart to the traditional verification of circuits through system identification. A strategy for choosing the stimulus is provided and an algorithm for verifying that the resulting response is indicative of a specification-compliant neural network system under test is derived. We find that the method can reliably detect defects in small neural networks or in small sub-circuits within larger neural networks.