Treating Brain-inspired Memories as Priors for Diffusion Model to Forecast Multivariate Time Series

Forecasting Multivariate Time Series (MTS) involves significant challenges in various application domains. One immediate challenge is modeling temporal patterns with the finite length of the input. These temporal patterns usually involve periodic and sudden events that recur across different channels. To better capture temporal patterns, we get inspiration from humans' memory mechanisms and propose a channel-shared, brain-inspired memory module for MTS. Specifically, brain-inspired memory comprises semantic and episodic memory, where the former is used to capture general patterns, such as periodic events, and the latter is employed to capture special patterns, such as sudden events, respectively. Meanwhile, we design corresponding recall and update mechanisms to better utilize these patterns. Furthermore, acknowledging the capacity of diffusion models to leverage memory as a prior, we present a brain-inspired memory-augmented diffusion model. This innovative model retrieves relevant memories for different channels, utilizing them as distinct priors for MTS predictions. This incorporation significantly enhances the accuracy and robustness of predictions. Experimental results on eight datasets consistently validate the superiority of our approach in capturing and leveraging diverse recurrent temporal patterns across different channels.

A New Channel Model for OAM Wireless Communication at 5.8 and 28 GHz

Orbital angular momentum (OAM) in electromagnetic (EM) waves can significantly enhance spectrum efficiency in wireless communications without requiring additional power, time, or frequency resources. Different OAM modes in EM waves create orthogonal channels, thereby improving spectrum efficiency. Additionally, OAM waves can more easily maintain orthogonality in line-of-sight (LOS) transmissions, offering an advantage over multiple-input and multiple-output (MIMO) technology in LOS scenarios. However, challenges such as divergence and crosstalk hinder OAM's efficiency. Additionally, channel modeling for OAM transmissions is still limited. A reliable channel model with balanced accuracy and complexity is essential for further system analysis. In this paper, we present a quasi-deterministic channel model for OAM channels in the 5.8 GHz and 28 GHz bands based on measurement data. Accurate measurement, especially at high frequencies like millimeter bands, requires synchronized RF channels to maintain phase coherence and purity, which is a major challenge for OAM channel measurement. To address this, we developed an 8-channel OAM generation device at 28 GHz to ensure beam integrity. By measuring and modeling OAM channels at 5.8 GHz and 28 GHz with a modified 3D geometric-based stochastic model (GBSM), this study provides insights into OAM channel characteristics, aiding simulation-based analysis and system optimization.

Channel-wise Influence: Estimating Data Influence for Multivariate Time Series

The influence function, a technique from robust statistics, measures the impact on model parameters or related functions when training data is removed or modified. This effective and valuable post-hoc method allows for studying the interpretability of machine learning models without requiring costly model retraining. It would provide extensions like increasing model performance, improving model generalization, and offering interpretability. Recently, Multivariate Time Series (MTS) analysis has become an important yet challenging task, attracting significant attention. However, there is no preceding research on the influence functions of MTS to shed light on the effects of modifying the channel of training MTS. Given that each channel in an MTS plays a crucial role in its analysis, it is essential to characterize the influence of different channels. To fill this gap, we propose a channel-wise influence function, which is the first method that can estimate the influence of different channels in MTS, utilizing a first-order gradient approximation that leverages the more informative average gradient of the data set. Additionally, we demonstrate how this influence function can be used to estimate the impact of a channel in MTS. Finally, we validated the accuracy and effectiveness of our influence estimation function in critical MTS analysis tasks, such as MTS anomaly detection and MTS forecasting. According to abundant experiments on real-world dataset, the original influence function performs worse than our method and even fail for the channel pruning problem, which demonstrate the superiority and necessity of channel-wise influence function in MTS analysis tasks.

A Non-negative VAE:the Generalized Gamma Belief Network

The gamma belief network (GBN), often regarded as a deep topic model, has demonstrated its potential for uncovering multi-layer interpretable latent representations in text data. Its notable capability to acquire interpretable latent factors is partially attributed to sparse and non-negative gamma-distributed latent variables. However, the existing GBN and its variations are constrained by the linear generative model, thereby limiting their expressiveness and applicability. To address this limitation, we introduce the generalized gamma belief network (Generalized GBN) in this paper, which extends the original linear generative model to a more expressive non-linear generative model. Since the parameters of the Generalized GBN no longer possess an analytic conditional posterior, we further propose an upward-downward Weibull inference network to approximate the posterior distribution of the latent variables. The parameters of both the generative model and the inference network are jointly trained within the variational inference framework. Finally, we conduct comprehensive experiments on both expressivity and disentangled representation learning tasks to evaluate the performance of the Generalized GBN against state-of-the-art Gaussian variational autoencoders serving as baselines.

Fractal OAM Generation and Detection Schemes

Orbital angular momentum (OAM) carried electromagnetic waves have the potential to improve spectrum efficiency in optical and radio-frequency communications due to the orthogonal wavefronts of different OAM modes. However, OAM beams are vortically hollow and divergent, which significantly decreases the capacity of OAM transmissions. In addition, unaligned transceivers in OAM transmissions can result in a high bit error rate (BER). The Talbot effect is a self-imaging phenomenon that can be used to generate optical or radio-frequency OAM beams with periodic repeating structures at multiples of a certain distance along the propagation direction. These periodic structures make it unnecessary for the transceiver antennas to be perfectly aligned and can also alleviate the hollow divergence of OAM beams. In this paper, we propose Talbot-effect-based fractal OAM generation and detection schemes using a uniform circular array (UCA) to significantly improve capacity and BER performance in unaligned OAM transmissions. We first provide a brief overview of fractal OAM. Then, we propose the fractal OAM beam generation and detection schemes. Numerical analysis and simulations verify the effectiveness of our proposed fractal OAM generation scheme and also demonstrate improved capacity and BER performance compared to normal OAM transmissions. We also analyze how the receive UCA radius and the distance between the UCAs impact the capacity and BER performances.

Considering Nonstationary within Multivariate Time Series with Variational Hierarchical Transformer for Forecasting

The forecasting of Multivariate Time Series (MTS) has long been an important but challenging task. Due to the non-stationary problem across long-distance time steps, previous studies primarily adopt stationarization method to attenuate the non-stationary problem of the original series for better predictability. However, existing methods always adopt the stationarized series, which ignores the inherent non-stationarity, and has difficulty in modeling MTS with complex distributions due to the lack of stochasticity. To tackle these problems, we first develop a powerful hierarchical probabilistic generative module to consider the non-stationarity and stochastic characteristics within MTS, and then combine it with transformer for a well-defined variational generative dynamic model named Hierarchical Time series Variational Transformer (HTV-Trans), which recovers the intrinsic non-stationary information into temporal dependencies. Being a powerful probabilistic model, HTV-Trans is utilized to learn expressive representations of MTS and applied to forecasting tasks. Extensive experiments on diverse datasets show the efficiency of HTV-Trans on MTS forecasting tasks