An Exact Poly-Time Membership-Queries Algorithm for Extraction a three-Layer ReLU Network

As machine learning increasingly becomes more prevalent in our everyday life, many organizations offer neural-networks based services as a black-box. The reasons for hiding a learning model may vary: e.g., preventing copying of its behavior or keeping back an adversarial from reverse-engineering its mechanism and revealing sensitive information about its training data. However, even as a black-box, some information can still be discovered by specific queries. In this work, we show a polynomial-time algorithm that uses a polynomial number of queries to mimic precisely the behavior of a three-layer neural network that uses ReLU activation.