Fitted Q-Iteration via Max-Plus-Linear Approximation

In this study, we consider the application of max-plus-linear approximators for Q-function in offline reinforcement learning of discounted Markov decision processes. In particular, we incorporate these approximators to propose novel fitted Q-iteration (FQI) algorithms with provable convergence. Exploiting the compatibility of the Bellman operator with max-plus operations, we show that the max-plus-linear regression within each iteration of the proposed FQI algorithm reduces to simple max-plus matrix-vector multiplications. We also consider the variational implementation of the proposed algorithm which leads to a per-iteration complexity that is independent of the number of samples.

ProCNS: Progressive Prototype Calibration and Noise Suppression for Weakly-Supervised Medical Image Segmentation

Weakly-supervised segmentation (WSS) has emerged as a solution to mitigate the conflict between annotation cost and model performance by adopting sparse annotation formats (e.g., point, scribble, block, etc.). Typical approaches attempt to exploit anatomy and topology priors to directly expand sparse annotations into pseudo-labels. However, due to a lack of attention to the ambiguous edges in medical images and insufficient exploration of sparse supervision, existing approaches tend to generate erroneous and overconfident pseudo proposals in noisy regions, leading to cumulative model error and performance degradation. In this work, we propose a novel WSS approach, named ProCNS, encompassing two synergistic modules devised with the principles of progressive prototype calibration and noise suppression. Specifically, we design a Prototype-based Regional Spatial Affinity (PRSA) loss to maximize the pair-wise affinities between spatial and semantic elements, providing our model of interest with more reliable guidance. The affinities are derived from the input images and the prototype-refined predictions. Meanwhile, we propose an Adaptive Noise Perception and Masking (ANPM) module to obtain more enriched and representative prototype representations, which adaptively identifies and masks noisy regions within the pseudo proposals, reducing potential erroneous interference during prototype computation. Furthermore, we generate specialized soft pseudo-labels for the noisy regions identified by ANPM, providing supplementary supervision. Extensive experiments on three medical image segmentation tasks involving different modalities demonstrate that the proposed framework significantly outperforms representative state-of-the-art methods

Densifying MIMO: Channel Modeling, Physical Constraints, and Performance Evaluation for Holographic Communications

As the backbone of the fifth-generation (5G) cellular network, massive multiple-input multiple-output (MIMO) encounters a significant challenge in practical applications: how to deploy a large number of antenna elements within limited spaces. Recently, holographic communication has emerged as a potential solution to this issue. It employs dense antenna arrays and provides a tractable model. Nevertheless, some challenges must be addressed to actualize this innovative concept. One is the mutual coupling among antenna elements within an array. When the element spacing is small, near-field coupling becomes the dominant factor that strongly restricts the array performance. Another is the polarization of electromagnetic waves. As an intrinsic property, it was not fully considered in the previous channel modeling of holographic communication. The third is the lack of real-world experiments to show the potential and possible defects of a holographic communication system. In this paper, we propose an electromagnetic channel model based on the characteristics of electromagnetic waves. This model encompasses the impact of mutual coupling in the transceiver sides and the depolarization in the propagation environment. Furthermore, by approximating an infinite array, the performance restrictions of large-scale dense antenna arrays are also studied theoretically to exploit the potential of the proposed channel. In addition, numerical simulations and a channel measurement experiment are conducted. The findings reveal that within limited spaces, the coupling effect, particularly for element spacing smaller than half of the wavelength, is the primary factor leading to the inflection point for the performance of holographic communications.

Multiple-Step Quantized Triplet STDP Implemented with Memristive Synapse

As an extension of the pairwise spike-timingdependent plasticity (STDP) learning rule, the triplet STDP is provided with greater capability in characterizing the synaptic changes in the biological neural cell. In this work, a novel mixedsignal circuit scheme, called multiple-step quantized triplet STDP, is designed to provide a precise and flexible implementation of coactivation triplet STDP learning rule in memristive synapse spiking neural network. The robustness of the circuit is greatly improved through the utilization of pulse-width encoded weight modulation signals. The circuit performance is studied through the simulations which are carried out in MATLAB Simulink & Simscape, and assessment is given by comparing the results of circuits with the algorithmic approaches.

Joint Transmit Waveform and Passive Beamforming Design for RIS-Aided DFRC Systems

Reconfigurable intelligent surface (RIS) is a promising technology for 6G networks owing to its superior ability to enhance the capacity and coverage of wireless communications by smartly creating a favorable propagation environment. In this paper, we investigate the potential of employing RIS in dual-functional radar-communication (DFRC) systems for improving both radar sensing and communication functionalities. In particular, we consider a RIS-assisted DFRC system in which the multi-antenna base station (BS) simultaneously performs both multi-input multi-output (MIMO) radar sensing and multi-user multi-input single-output (MU-MISO) communications using the same hardware platform. We aim to jointly design the dual-functional transmit waveform and the passive beamforming of RIS to maximize the radar output signal-to-interference-plus-noise ratio (SINR) achieved by space-time adaptive processing (STAP), while satisfying the communication quality-of-service (QoS) requirement under one of three metrics, the constant-modulus constraint on the transmit waveform, and the restrict of RIS reflecting coefficients. An efficient algorithm framework based on the alternative direction method of multipliers (ADMM) and majorization-minimization (MM) methods is developed to solve the complicated non-convex optimization problem. Simulation results verify the advancement of the proposed RIS-assisted DRFC scheme and the effectiveness of the developed ADMM-MM-based joint transmit waveform and passive beamforming design algorithm.

Machine Learning Empowered Resource Allocation in IRS Aided MISO-NOMA Networks

A novel framework of intelligent reflecting surface (IRS)-aided multiple-input single-output (MISO) non-orthogonal multiple access (NOMA) network is proposed, where a base station (BS) serves multiple clusters with unfixed number of users in each cluster. The goal is to maximize the sum rate of all users by jointly optimizing the passive beamforming vector at the IRS, decoding order, power allocation coefficient vector and number of clusters, subject to the rate requirements of users. In order to tackle the formulated problem, a three-step approach is proposed. More particularly, a long short-term memory (LSTM) based algorithm is first adopted for predicting the mobility of users. Secondly, a K-means based Gaussian mixture model (K-GMM) algorithm is proposed for user clustering. Thirdly, a deep Q-network (DQN) based algorithm is invoked for jointly determining the phase shift matrix and power allocation policy. Simulation results are provided for demonstrating that the proposed algorithm outperforms the benchmarks, while the throughput gain of 35% can be achieved by invoking NOMA technique instead of orthogonal multiple access (OMA).

Hybrid Machine Learning Models of Classifying Residential Requests for Smart Dispatching

This paper presents a hybrid machine learning method of classifying residential requests in natural language to responsible departments that provide timely responses back to residents under the vision of digital government services in smart cities. Residential requests in natural language descriptions cover almost every aspect of a city's daily operation. Hence the responsible departments are fine-grained to even the level of local communities. There are no specific general categories or labels for each request sample. This causes two issues for supervised classification solutions, namely (1) the request sample data is unbalanced and (2) lack of specific labels for training. To solve these issues, we investigate a hybrid machine learning method that generates meta-class labels by means of unsupervised clustering algorithms; applies two-word embedding methods with three classifiers (including two hierarchical classifiers and one residual convolutional neural network); and selects the best performing classifier as the classification result. We demonstrate our approach performing better classification tasks compared to two benchmarking machine learning models, Naive Bayes classifier and a Multiple Layer Perceptron (MLP). In addition, the hierarchical classification method provides insights into the source of classification errors.