Robust Environment Perception for Automated Driving: A Unified Learning Pipeline for Visual-Infrared Object Detection

The RGB complementary metal-oxidesemiconductor (CMOS) sensor works within the visible light spectrum. Therefore it is very sensitive to environmental light conditions. On the contrary, a long-wave infrared (LWIR) sensor operating in 8-14 micro meter spectral band, functions independent of visible light. In this paper, we exploit both visual and thermal perception units for robust object detection purposes. After delicate synchronization and (cross-) labeling of the FLIR [1] dataset, this multi-modal perception data passes through a convolutional neural network (CNN) to detect three critical objects on the road, namely pedestrians, bicycles, and cars. After evaluation of RGB and infrared (thermal and infrared are often used interchangeably) sensors separately, various network structures are compared to fuse the data at the feature level effectively. Our RGB-thermal (RGBT) fusion network, which takes advantage of a novel entropy-block attention module (EBAM), outperforms the state-of-the-art network [2] by 10% with 82.9% mAP.

Contact-less Material Probing with Distributed Sensors: Joint Sensing and Communication Optimization

The utilization of RF signals to probe material properties of objects is of huge interest both in academia as well as industry. To this end, a setup is investigated, in which a transmitter equipped with a two-dimensional multi-antenna array dispatches a signal, which hits objects in the environment and the reflections from the objects are captured by distributed sensors. The received signal at those sensors are then amplified and forwarded to a multiple antenna fusion center, which performs space-time post-processing in order to optimize the information extraction. In this process, optimal design of power allocation per object alongside sensors amplifications is of crucial importance. Here, the power allocation and sensors amplifications is jointly optimized, given maximum-ratio combining (MRC) at the fusion center. We formulate this challenge as a sum-power minimization under per-object SINR constraints, a sum-power constraint at the transmitter and individual power constraints at the sensors. Moreover, the advantage of deploying zero-forcing (ZF) and minimum mean-squared error (MMSE) at the fusion center is discussed. Asymptotic analysis is also provided for the case that large number of sensors are deployed in the sensing environment.