Minimizing the Hidden Cost of Scales: Graph-Guided Ultra-Low-Bit Quantization for Large Language Models

Post-training quantization (PTQ) is critical for the efficient deployment of large language models (LLMs). Recent ultra-low-bit PTQ methods rely on rigid weight-saliency assumptions or position heuristics, introducing substantial hidden scaling overhead. We propose SAGE-PTQ (Saliency-Aware Graph-guided Efficient PTQ), a novel ultra-low-bit quantization framework for LLMs that minimizes hidden scaling cost. SAGE-PTQ separates salient and unsalient weights using distributional statistics, then models subsampled unsalient weights as a sparse graph to estimate the optimal number of groups per layer. SAGE-PTQ applies dual-mode quantization, assigning multi-bit precision to salient weights and binarizing unsalient weights. To reduce scaling overhead, SAGE-PTQ uses one per-channel scale for salient weights and one scalar per unsalient group. Finally, SAGE-PTQ implements adaptive saliency thresholding to select the optimal saliency ratio per matrix. SAGE-PTQ achieves 1.03 weight bits and only 0.004 scaling bits per matrix on average, outperforming state-of-the-art methods such as BiLLM and PB-LLM. On LLaMA-3-8B, SAGE-PTQ achieves 6.74 WikiText2 perplexity, compared to 55.8 for BiLLM, while using less than 50% of BiLLM's GPU memory. On LLaMA-2-70B, SAGE-PTQ provides 1.5x faster decoding on one NVIDIA L40 GPU, demonstrating practical inference efficiency.

Comparative Study of ZF, LMS and RLS Adaptive Equalization for alpha-mu Fading Channels

Wireless communication systems generally endure severe fading and interference caused by the time-dispersive channel. Major transmission distortion is produced by channel multipath propagation and overlap of subsequent symbols. To counteract channel response, equalization techniques are employed to operate on channel output and recover transmitted signal at the receiver. This work investigates equalization of channels undergoing alpha-mu fading distribution, which models multipath propagation in millimeter Wave (mmWave) and sub-Terahertz (sub-THz) frequencies bands. Three equalization algorithms, which are non-adaptive Zero-Forcing (ZF), adaptive Least-Mean-Square (LMS) and adaptive Recursive-Least-Square (RLS) are implemented and addressed in terms of Bit Error Rate (BER), Convergence speed and implementation complexity. Monte Carlo simulations are then carried out to compare algorithms and assess performance by varying multiple parameters such as training lengths, channel order, equalization taps and diverse fading conditions.

Complex-valued Neural Networks -- Theory and Analysis

Complex-valued neural networks (CVNNs) have recently been successful in various pioneering areas which involve wave-typed information and frequency-domain processing. This work addresses different structures and classification of CVNNs. The theory behind complex activation functions, implications related to complex differentiability and special activations for CVNN output layers are presented. The work also discusses CVNN learning and optimization using gradient and non-gradient based algorithms. Complex Backpropagation utilizing complex chain rule is also explained in terms of Wirtinger calculus. Moreover, special modules for building CVNN models, such as complex batch normalization and complex random initialization are also discussed. The work also highlights libraries and software blocks proposed for CVNN implementations and discusses future directions. The objective of this work is to understand the dynamics and most recent developments of CVNNs.

Analysis of Synchrosqueezed Transforms and Application Perspectives

High-resolution time-frequency (TF) analysis plays crucial role in characterizing multicomponent signal (MCSs) and estimating oscillatory properties. Linear time-frequency representations (TFRs) such as classical short-time Fourier transform (STFT) and continuous wavelet transform (CWT) incur constrained TF resolution and energy diffusion in both time and frequency direction. The synchrosqueezing transform (SST) represents a powerful sparse reassignment method that allows component reconstruction. This work introduces SST as extension to STFT and CWT and illustrates corresponding advantages of sharpening TFRs and recovery of instantaneous components. The SST effectiveness is assessed in practical situations that involve comparing STFT-based and CWT-based versions of synthetic data and also applying SST to optimize deep learning (DL) prediction model. It is demonstrated how SST achieves promising results in terms of improving TFR readability and increasing accuracy of DL-based prediction models.