AI-Driven HSI: Multimodality, Fusion, Challenges, and the Deep Learning Revolution

Hyperspectral imaging (HSI) captures spatial and spectral data, enabling analysis of features invisible to conventional systems. The technology is vital in fields such as weather monitoring, food quality control, counterfeit detection, healthcare diagnostics, and extending into defense, agriculture, and industrial automation at the same time. HSI has advanced with improvements in spectral resolution, miniaturization, and computational methods. This study provides an overview of the HSI, its applications, challenges in data fusion and the role of deep learning models in processing HSI data. We discuss how integration of multimodal HSI with AI, particularly with deep learning, improves classification accuracy and operational efficiency. Deep learning enhances HSI analysis in areas like feature extraction, change detection, denoising unmixing, dimensionality reduction, landcover mapping, data augmentation, spectral construction and super resolution. An emerging focus is the fusion of hyperspectral cameras with large language models (LLMs), referred as highbrain LLMs, enabling the development of advanced applications such as low visibility crash detection and face antispoofing. We also highlight key players in HSI industry, its compound annual growth rate and the growing industrial significance. The purpose is to offer insight to both technical and non-technical audience, covering HSI's images, trends, and future directions, while providing valuable information on HSI datasets and software libraries.

Holistic random encoding for imaging through multimode fibers

The input numerical aperture (NA) of multimode fiber (MMF) can be effectively increased by placing turbid media at the input end of the MMF. This provides the potential for high-resolution imaging through the MMF. While the input NA is increased, the number of propagation modes in the MMF and hence the output NA remains the same. This makes the image reconstruction process underdetermined and may limit the quality of the image reconstruction. In this paper, we aim to improve the signal to noise ratio (SNR) of the image reconstruction in imaging through MMF. We notice that turbid media placed in the input of the MMF transforms the incoming waves into a better format for information transmission and information extraction. We call this transformation as holistic random (HR) encoding of turbid media. By exploiting the HR encoding, we make a considerable improvement on the SNR of the image reconstruction. For efficient utilization of the HR encoding, we employ sparse representation (SR), a relatively new signal reconstruction framework when it is provided with a HR encoded signal. This study shows for the first time to our knowledge the benefit of utilizing the HR encoding of turbid media for recovery in the optically underdetermined systems where the output NA of it is smaller than the input NA for imaging through MMF.