On Logical Extrapolation for Mazes with Recurrent and Implicit Networks

Recent work has suggested that certain neural network architectures-particularly recurrent neural networks (RNNs) and implicit neural networks (INNs) are capable of logical extrapolation. That is, one may train such a network on easy instances of a specific task and then apply it successfully to more difficult instances of the same task. In this paper, we revisit this idea and show that (i) The capacity for extrapolation is less robust than previously suggested. Specifically, in the context of a maze-solving task, we show that while INNs (and some RNNs) are capable of generalizing to larger maze instances, they fail to generalize along axes of difficulty other than maze size. (ii) Models that are explicitly trained to converge to a fixed point (e.g. the INN we test) are likely to do so when extrapolating, while models that are not (e.g. the RNN we test) may exhibit more exotic limiting behaviour such as limit cycles, even when they correctly solve the problem. Our results suggest that (i) further study into why such networks extrapolate easily along certain axes of difficulty yet struggle with others is necessary, and (ii) analyzing the dynamics of extrapolation may yield insights into designing more efficient and interpretable logical extrapolators.

A Two-Stage Deep Learning Detection Classifier for the ATLAS Asteroid Survey

In this paper we present a two-step neural network model to separate detections of solar system objects from optical and electronic artifacts in data obtained with the "Asteroid Terrestrial-impact Last Alert System" (ATLAS), a near-Earth asteroid sky survey system [arXiv:1802.00879]. A convolutional neural network [arXiv:1807.10912] is used to classify small "postage-stamp" images of candidate detections of astronomical sources into eight classes, followed by a multi-layered perceptron that provides a probability that a temporal sequence of four candidate detections represents a real astronomical source. The goal of this work is to reduce the time delay between Near-Earth Object (NEO) detections and submission to the Minor Planet Center. Due to the rare and hazardous nature of NEOs [Harris and D'Abramo, 2015], a low false negative rate is a priority for the model. We show that the model reaches 99.6\% accuracy on real asteroids in ATLAS data with a 0.4\% false negative rate. Deployment of this model on ATLAS has reduced the amount of NEO candidates that astronomers must screen by 90%, thereby bringing ATLAS one step closer to full autonomy.