Efficient Spatio-Temporal Signal Recognition on Edge Devices Using PointLCA-Net

Recent advancements in machine learning, particularly through deep learning architectures like PointNet, have transformed the processing of three-dimensional (3D) point clouds, significantly improving 3D object classification and segmentation tasks. While 3D point clouds provide detailed spatial information, spatio-temporal signals introduce a dynamic element that accounts for changes over time. However, applying deep learning techniques to spatio-temporal signals and deploying them on edge devices presents challenges, including real-time processing, memory capacity, and power consumption. To address these issues, this paper presents a novel approach that combines PointNet's feature extraction with the in-memory computing capabilities and energy efficiency of neuromorphic systems for spatio-temporal signal recognition. The proposed method consists of a two-stage process: in the first stage, PointNet extracts features from the spatio-temporal signals, which are then stored in non-volatile memristor crossbar arrays. In the second stage, these features are processed by a single-layer spiking neural encoder-decoder that employs the Locally Competitive Algorithm (LCA) for efficient encoding and classification. This work integrates the strengths of both PointNet and LCA, enhancing computational efficiency and energy performance on edge devices. PointLCA-Net achieves high recognition accuracy for spatio-temporal data with substantially lower energy burden during both inference and training than comparable approaches, thus advancing the deployment of advanced neural architectures in energy-constrained environments.

ViT-LCA: A Neuromorphic Approach for Vision Transformers

The recent success of Vision Transformers has generated significant interest in attention mechanisms and transformer architectures. Although existing methods have proposed spiking self-attention mechanisms compatible with spiking neural networks, they often face challenges in effective deployment on current neuromorphic platforms. This paper introduces a novel model that combines vision transformers with the Locally Competitive Algorithm (LCA) to facilitate efficient neuromorphic deployment. Our experiments show that ViT-LCA achieves higher accuracy on ImageNet-1K dataset while consuming significantly less energy than other spiking vision transformer counterparts. Furthermore, ViT-LCA's neuromorphic-friendly design allows for more direct mapping onto current neuromorphic architectures.

Rouser: Robust SNN training using adaptive threshold learning

In Spiking Neural Networks (SNNs), learning rules are based on neuron spiking behavior, that is, if and when spikes are generated due to a neuron's membrane potential exceeding that neuron's firing threshold, and this spike timing encodes vital information. However, the threshold is generally treated as a hyperparameter, and incorrect selection can lead to neurons that do not spike for large portions of the training process, hindering the effective rate of learning. Inspired by homeostatic mechanisms in biological neurons, this work (Rouser) presents a study to rouse training-inactive neurons and improve the SNN training by using an in-loop adaptive threshold learning mechanism. Rouser's adaptive threshold allows for dynamic adjustments based on input data and network hyperparameters, influencing spike timing and improving training. This study focuses primarily on investigating the significance of learning neuron thresholds alongside weights in SNNs. We evaluate the performance of Rouser on the spatiotemporal datasets NMNIST, DVS128 and Spiking Heidelberg Digits (SHD), compare our results with state-of-the-art SNN training techniques, and discuss the strengths and limitations of our approach. Our results suggest that promoting threshold from a hyperparameter to a parameter can effectively address the issue of dead neurons during training, resulting in a more robust training algorithm that leads to improved training convergence, increased test accuracy, and substantial reductions in the number of training epochs needed to achieve viable accuracy. Rouser achieves up to 70% lower training latency while providing up to 2% higher accuracy over state-of-the-art SNNs with similar network architecture on the neuromorphic datasets NMNIST, DVS128 and SHD.