Zeroth-Order Topological Insights into Iterative Magnitude Pruning

Modern-day neural networks are famously large, yet also highly redundant and compressible; there exist numerous pruning strategies in the deep learning literature that yield over 90% sparser sub-networks of fully-trained, dense architectures while still maintaining their original accuracies. Amongst these many methods though -- thanks to its conceptual simplicity, ease of implementation, and efficacy -- Iterative Magnitude Pruning (IMP) dominates in practice and is the de facto baseline to beat in the pruning community. However, theoretical explanations as to why a simplistic method such as IMP works at all are few and limited. In this work, we leverage the notion of persistent homology to gain insights into the workings of IMP and show that it inherently encourages retention of those weights which preserve topological information in a trained network. Subsequently, we also provide bounds on how much different networks can be pruned while perfectly preserving their zeroth order topological features, and present a modified version of IMP to do the same.

High-Capacity Complex Convolutional Neural Networks For I/Q Modulation Classification

I/Q modulation classification is a unique pattern recognition problem as the data for each class varies in quality, quantified by signal to noise ratio (SNR), and has structure in the complex-plane. Previous work shows treating these samples as complex-valued signals and computing complex-valued convolutions within deep learning frameworks significantly increases the performance over comparable shallow CNN architectures. In this work, we claim state of the art performance by enabling high-capacity architectures containing residual and/or dense connections to compute complex-valued convolutions, with peak classification accuracy of 92.4% on a benchmark classification problem, the RadioML 2016.10a dataset. We show statistically significant improvements in all networks with complex convolutions for I/Q modulation classification. Complexity and inference speed analyses show models with complex convolutions substantially outperform architectures with a comparable number of parameters and comparable speed by over 10% in each case.

Modulation Pattern Detection Using Complex Convolutions in Deep Learning

Transceivers used for telecommunications transmit and receive specific modulation patterns that are represented as sequences of complex numbers. Classifying modulation patterns is challenging because noise and channel impairments affect the signals in complicated ways such that the received signal bears little resemblance to the transmitted signal. Although deep learning approaches have shown great promise over statistical methods in this problem space, deep learning frameworks continue to lag in support for complex-valued data. To address this gap, we study the implementation and use of complex convolutions in a series of convolutional neural network architectures. Replacement of data structure and convolution operations by their complex generalization in an architecture improves performance, with statistical significance, at recognizing modulation patterns in complex-valued signals with high SNR after being trained on low SNR signals. This suggests complex-valued convolutions enables networks to learn more meaningful representations. We investigate this hypothesis by comparing the features learned in each experiment by visualizing the inputs that results in one-hot modulation pattern classification for each network.