Enhanced Information Extraction from Cylindrical Visual-Tactile Sensors via Image Fusion

Vision-based tactile sensors equipped with planar contact structures acquire the shape, force, and motion states of objects in contact. The limited planar contact area presents a challenge in acquiring information about larger target objects. In contrast, vision-based tactile sensors with cylindrical contact structures could extend the contact area by rolling, which can acquire much tactile information that exceeds the sensing projection area in a single contact. However, the tactile data acquired by cylindrical structures does not consistently correspond to the same depth level. Therefore, stitching and analyzing the data in an extended contact area is a challenging problem. In this work, we propose an image fusion method based on cylindrical vision-based tactile sensors. The method takes advantage of the changing characteristics of the contact depth of cylindrical structures, extracts the effective information of different contact depths in the frequency domain, and performs differential fusion for the information characteristics. The results show that in object contact confronting an area larger than single sensing, the images fused with our proposed method have higher information and structural similarity compared with the method of stitching based on motion distance sampling. Meanwhile, it is robust to sampling time. We complement this method with a deep neural network to illustrate its potential for fusing and recognizing object contact information using cylindrical vision-based tactile sensors.

A Vision-Based Tactile Sensing System for Multimodal Contact Information Perception via Neural Network

In general, robotic dexterous hands are equipped with various sensors for acquiring multimodal contact information such as position, force, and pose of the grasped object. This multi-sensor-based design adds complexity to the robotic system. In contrast, vision-based tactile sensors employ specialized optical designs to enable the extraction of tactile information across different modalities within a single system. Nonetheless, the decoupling design for different modalities in common systems is often independent. Therefore, as the dimensionality of tactile modalities increases, it poses more complex challenges in data processing and decoupling, thereby limiting its application to some extent. Here, we developed a multimodal sensing system based on a vision-based tactile sensor, which utilizes visual representations of tactile information to perceive the multimodal contact information of the grasped object. The visual representations contain extensive content that can be decoupled by a deep neural network to obtain multimodal contact information such as classification, position, posture, and force of the grasped object. The results show that the tactile sensing system can perceive multimodal tactile information using only one single sensor and without different data decoupling designs for different modal tactile information, which reduces the complexity of the tactile system and demonstrates the potential for multimodal tactile integration in various fields such as biomedicine, biology, and robotics.

Waveforms for xG Non-stationary Channels

Waveform design aims to achieve orthogonality among data signals/symbols across all available Degrees of Freedom (DoF) to avoid interference while transmitted over the channel. In general, precoding decomposes the channel matrix into desirable components in order to construct a precoding matrix, which is combined with the data signal to orthogonality in the spatial dimension. On the other hand, modulation uses orthogonal carriers in a certain signal space to carry data symbols with minimal interference from other symbols. However, it is widely evident that next Generation (xG) wireless systems will experience very high mobility, density and time-varying multi-path propagation that will result in a highly non-stationarity of the channel states. Conventional precoding methods using SVD or QR decomposition, are unable to capture these joint spatio-temporal variations as those techniques treat the space-time-varying channel as separate independent spatial channel matrices and hence fail to achieve joint spatio-temporal orthogonality. Meanwhile, the carriers in OFDM and OTFS modulations are unable to maintain the orthogonality in the frequency and delay-Doppler domain respectively, due to the higher order physical variation like velocity (Doppler effect) or acceleration (time-varying Doppler effect). In this article, we review a recent method called High Order Generalized Mercer's Theorem (HOGMT) for orthogonal decomposition of higher dimensional, non-stationary channels and its application to MU-MIMO precoding and modulation. We conclude by identifying some practical challenges and the future directions for waveform design for MU-MIMO non-stationary channels based on HOGMT.

Multidimensional Eigenwave Multiplexing Modulation for Non-Stationary Channels

While interference in time domain (caused by path difference) is mitigated by OFDM modulation, interference in frequency domain (due to velocity difference), can be mitigated by OTFS modulation. However, in non-stationary channels, the relative difference in acceleration will cause Inter-Doppler Interference (IDI) and a modulation method for mitigating IDI does not exist in the literature. Both methods in the literature use carriers in a specific domain which achieve orthogonality in the target domain to mitigate interference. Moreover, those modulation cannot directly incorporate space domain, which requires additional precoding technique to mitigate inter-user interference (IUI) for MU-MIMO channels. This work presents a generalized modulation for any multidimensional channel. Recently, Higher Order Mercer's Theorem (HOGMT) [1] has been proposed to decompose multi-user non-stationary channels into independent fading subchannels (Eigenwaves). Based on HOGMT decomposition, we develop Multidimensional Eigenwaves Multiplexing (MEM) modulation which uses jointly orthogonal eigenwaves, decomposed from the multidimensional channel as subcarriers. Data symbols modulated by these eigenwaves can achieve orthogonality across each degree of freedom(e.g. space (users/antennas), time-frequency and delay-Doppler). Consequently, the transmitted remain independent over the high dimensional channel, thereby avoiding interference from other symbols.

Low Complexity Dirty Paper Coding for MU-MIMO Channels

Dirty Paper Coding (DPC) is considered as the optimal precoding which achieves capacity for the Gaussian Multiple-Input Multiple-Output (MIMO) broadcast channel (BC). However, to find the optimal precoding order, it needs to repeat N! times for N users as there are N! possible precoding orders. This extremely high complexity limits its practical use in modern wireless networks. In this paper, we show the equivalence of DPC and the recently proposed Higher Order Mercer's Theorem (HOGMT) precoding [1] in 2-D (spatial) case, which provides an alternate implementation for DPC. Furthermore, we show that the proposed implementation method is linear over the permutation operator when permuting over multi-user channels. Therefore, we present a low complexity algorithm that optimizes the precoding order for DPC with beamforming, eliminating repeated computation of DPC for each precoding order. Simulations show that our method can achieve the same result as conventional DPC with near 30 dB lower complexity for N = 10 users.

Joint Spatio-Temporal Precoding for Practical Non-Stationary Wireless Channels

The high mobility, density and multi-path evident in modern wireless systems makes the channel highly non-stationary. This causes temporal variation in the channel distribution that leads to the existence of time-varying joint interference across multiple degrees of freedom (DoF, e.g., users, antennas, frequency and symbols), which renders conventional precoding sub-optimal in practice. In this work, we derive a High-Order Generalization of Mercer's Theorem (HOGMT), which decomposes the multi-user non-stationary channel into two (dual) sets of jointly orthogonal subchannels (eigenfunctions), that result in the other set when one set is transmitted through the channel. This duality and joint orthogonality of eigenfuntions ensure transmission over independently flat-fading subchannels. Consequently, transmitting these eigenfunctions with optimally derived coefficients eventually mitigates any interference across its degrees of freedoms and forms the foundation of the proposed joint spatio-temporal precoding. The transferred dual eigenfuntions and coefficients directly reconstruct the data symbols at the receiver upon demodulation, thereby significantly reducing its computational burden, by alleviating the need for any complementary post-coding. Additionally, the eigenfunctions decomposed from the time-frequency delay-Doppler channel kernel are paramount to extracting the second-order channel statistics, and therefore completely characterize the underlying channel. We evaluate this using a realistic non-stationary channel framework built in Matlab and show that our precoding achieves ${\geqslant}$4 orders of reduction in BER at SNR${\geqslant}15$dB in OFDM systems for higher-order modulations and less complexity compared to the state-of-the-art precoding.

Optimizing towards the best insertion-based error-tolerating joints

We present an optimization-based design process that can generate the best insertion-based joints with respect to different errors, including manipulation error, manufacturing error, and sensing error. We separate the analysis into two stages, the insertion and the after-insertion stability. Each sub-process is discretized into different modes of contacts. The transitions among the contact modes form a directed graph and the connectivity of the graph is achieved and maintained through the manipulation of the socket edge-angle and peg contact-point locations. The analysis starts in 2D with the assumption of point-edge contacts. During the optimization, the edges of the socket are rotated and the points on the peg are moved along the edges to ensure the successful insertion and the stability after insertion. We show in simulation that our proposed method can generate insertion-based joints that are tolerant to the given errors. and we present a few simple 3D projections to show that the analysis is still effective beyond 2D cases.

Unified Characterization and Precoding for Non-Stationary Channels

Modern wireless channels are increasingly dense and mobile making the channel highly non-stationary. The time-varying distribution and the existence of joint interference across multiple degrees of freedom (e.g., users, antennas, frequency and symbols) in such channels render conventional precoding sub-optimal in practice, and have led to historically poor characterization of their statistics. The core of our work is the derivation of a high-order generalization of Mercer's Theorem to decompose the non-stationary channel into constituent fading sub-channels (2-D eigenfunctions) that are jointly orthogonal across its degrees of freedom. Consequently, transmitting these eigenfunctions with optimally derived coefficients eventually mitigates any interference across these dimensions and forms the foundation of the proposed joint spatio-temporal precoding. The precoded symbols directly reconstruct the data symbols at the receiver upon demodulation, thereby significantly reducing its computational burden, by alleviating the need for any complementary decoding. These eigenfunctions are paramount to extracting the second-order channel statistics, and therefore completely characterize the underlying channel. Theory and simulations show that such precoding leads to ${>}10^4{\times}$ BER improvement (at 20dB) over existing methods for non-stationary channels.

Proofs and Supplementary Material: Unified Characterization and Precoding for Non-Stationary Channels

This document provides the supplementary material including a comprehensive related work, the complete proofs and extended evaluation results that support the manuscript, "Unified Characterization and Precoding for Non-Stationary Channels", that was accepted for publication at IEEE International Conference on Communications (ICC) 2022. Equations (1)--(34) refer to the equations from the main manuscript, and the Theorem, Lemma and Corollaries correspond to those from the manuscript.