Scalable Superconductor Neuron with Ternary Synaptic Connections for Ultra-Fast SNN Hardware

A novel high-fan-in differential superconductor neuron structure designed for ultra-high-performance Spiking Neural Network (SNN) accelerators is presented. Utilizing a high-fan-in neuron structure allows us to design SNN accelerators with more synaptic connections, enhancing the overall network capabilities. The proposed neuron design is based on superconductor electronics fabric, incorporating multiple superconducting loops, each with two Josephson Junctions. This arrangement enables each input data branch to have positive and negative inductive coupling, supporting excitatory and inhibitory synaptic data. Compatibility with synaptic devices and thresholding operation is achieved using a single flux quantum (SFQ) pulse-based logic style. The neuron design, along with ternary synaptic connections, forms the foundation for a superconductor-based SNN inference. To demonstrate the capabilities of our design, we train the SNN using snnTorch, augmenting the PyTorch framework. After pruning, the demonstrated SNN inference achieves an impressive 96.1% accuracy on MNIST images. Notably, the network exhibits a remarkable throughput of 8.92 GHz while consuming only 1.5 nJ per inference, including the energy consumption associated with cooling to 4K. These results underscore the potential of superconductor electronics in developing high-performance and ultra-energy-efficient neural network accelerator architectures.

Hybrid Synaptic Structure for Spiking Neural Network Realization

Neural networks and neuromorphic computing play pivotal roles in deep learning and machine vision. Due to their dissipative nature and inherent limitations, traditional semiconductor-based circuits face challenges in realizing ultra-fast and low-power neural networks. However, the spiking behavior characteristic of single flux quantum (SFQ) circuits positions them as promising candidates for spiking neural networks (SNNs). Our previous work showcased a JJ-Soma design capable of operating at tens of gigahertz while consuming only a fraction of the power compared to traditional circuits, as documented in [1]. This paper introduces a compact SFQ-based synapse design that applies positive and negative weighted inputs to the JJ-Soma. Using an RSFQ synapse empowers us to replicate the functionality of a biological neuron, a crucial step in realizing a complete SNN. The JJ-Synapse can operate at ultra-high frequencies, exhibits orders of magnitude lower power consumption than CMOS counterparts, and can be conveniently fabricated using commercial Nb processes. Furthermore, the network's flexibility enables modifications by incorporating cryo-CMOS circuits for weight value adjustments. In our endeavor, we have successfully designed, fabricated, and partially tested the JJ-Synapse within our cryocooler system. Integration with the JJ-Soma further facilitates the realization of a high-speed inference SNN.

An On-Chip Trainable Neuron Circuit for SFQ-Based Spiking Neural Networks

We present an on-chip trainable neuron circuit. Our proposed circuit suits bio-inspired spike-based time-dependent data computation for training spiking neural networks (SNN). The thresholds of neurons can be increased or decreased depending on the desired application-specific spike generation rate. This mechanism provides us with a flexible design and scalable circuit structure. We demonstrate the trainable neuron structure under different operating scenarios. The circuits are designed and optimized for the MIT LL SFQ5ee fabrication process. Margin values for all parameters are above 25\% with a 3GHz throughput for a 16-input neuron.