Perceptual-Score: A Psychophysical Measure for Assessing the Biological Plausibility of Visual Recognition Models

For the last decade, convolutional neural networks (CNNs) have vastly superseded their predecessors in nearly all vision tasks in artificial intelligence, including object recognition. However, in spite of abundant advancements, they continue to pale in comparison to biological vision. This chasm has prompted the development of biologically-inspired models that have attempted to mimic the human visual system, primarily at a neural-level, which are evaluated using standard dataset benchmarks. However, more work is needed to understand how these models actually perceive the visual world. This article proposes a state-of-the-art procedure that generates a new metric, Perceptual-Score, which is grounded in visual psychophysics, and is capable of reliably estimating perceptual responses across numerous models -- representing a large range in complexity and biological inspiration. We perform the procedure on twelve models that vary in degree of biological inspiration and complexity, and compare the results against the aggregated results of 2,390 Amazon Mechanical Turk workers who together provided ~2.7 million perceptual responses. Each model's Perceptual-Score is compared against the state-of-the-art neural activity-based metric, Brain-Score. Our study indicates that models with high correlation to human perceptual behavior also have high correlation with the corresponding neural activity.