Multiple-beam Interference Spectroscopy: Instrument Analysis and Spectrum Reconstruction

Hyperspectral imaging systems based on multiple-beam interference (MBI), such as Fabry-Perot interferometry, are attracting interest due to their compact design, high throughput, and fine resolution. Unlike dispersive devices, which measure spectra directly, the desired spectra in interferometric systems are reconstructed from measured interferograms. Although the response of MBI devices is modeled by the Airy function, existing reconstruction techniques are often limited to Fourier-transform spectroscopy, which is tailored for two-beam interference (TBI). These methods impose limitations for MBI and are susceptible to non-idealities like irregular sampling and noise, highlighting the need for an in-depth numerical framework. To fill this gap, we propose a rigorous taxonomy of the TBI and MBI instrument description and propose a unified Bayesian formulation which both embeds the description of existing literature works and adds some of the real-world non-idealities of the acquisition process. Under this framework, we provide a comprehensive review of spectroscopy forward and inverse models. In the forward model, we propose a thorough analysis of the discretization of the continuous model and the ill-posedness of the problem. In the inverse model, we extend the range of existing solutions for spectrum reconstruction, framing them as an optimization problem. Specifically, we provide a progressive comparative analysis of reconstruction methods from more specific to more general scenarios, up to employing the proposed Bayesian framework with prior knowledge, such as sparsity constraints. Experiments on simulated and real data demonstrate the framework's flexibility and noise robustness. The code is available at https://github.com/mhmdjouni/inverspyctrometry.

Implementation of hyperspectral inversion algorithms on FPGA: Hardware comparison using High Level Synthesis

Hyperspectral imaging is gathering significant attention due to its potential in various domains such as geology, agriculture, ecology, and surveillance. However, the associated processing algorithms, which are essential for enhancing output quality and extracting relevant information, are often computationally intensive and have to deal with substantial data volumes. Our focus lies on reconfigurable hardware, particularly recent FPGAs. While FPGA design can be complex, High Level Synthesis (HLS) workflows have emerged as a solution, abstracting low-level design intricacies and enhancing productivity. Despite successful prior efforts using HLS for hyperspectral imaging acceleration, we lack a comprehensive research to benchmark various algorithms and architectures within a unified framework. This study aims to quantitatively evaluate performance across different inversion algorithms and design architectures, providing insights for optimal trade-offs for specific applications. We apply this analysis to the case study of spectrum reconstruction processed from interferometric acquisitions taken by Fourier transform spectrometers.

Spectro-spatial hyperspectral image reconstruction from interferometric acquisitions

In the last decade, novel hyperspectral cameras have been developed with particularly desirable characteristics of compactness and short acquisition time, retaining their potential to obtain spectral/spatial resolution competitive with respect to traditional cameras. However, a computational effort is required to recover an interpretable data cube. In this work we focus our attention on imaging spectrometers based on interferometry, for which the raw acquisition is an image whose spectral component is expressed as an interferogram. Previous works have focused on the inversion of such acquisition on a pixel-by-pixel basis within a Bayesian framework, leaving behind critical information on the spatial structure of the image data cube. In this work, we address this problem by integrating a spatial regularization for image reconstruction, showing that the combination of spectral and spatial regularizers leads to enhanced performances with respect to the pixelwise case. We compare our results with Plug-and-Play techniques, as its strategy to inject a set of denoisers from the literature can be implemented seamlessly with our physics-based formulation of the optimization problem.

Model-based demosaicking for acquisitions by a RGBW color filter array

Microsatellites and drones are often equipped with digital cameras whose sensing system is based on color filter arrays (CFAs), which define a pattern of color filter overlaid over the focal plane. Recent commercial cameras have started implementing RGBW patterns, which include some filters with a wideband spectral response together with the more classical RGB ones. This allows for additional light energy to be captured by the relevant pixels and increases the overall SNR of the acquisition. Demosaicking defines reconstructing a multi-spectral image from the raw image and recovering the full color components for all pixels. However, this operation is often tailored for the most widespread patterns, such as the Bayer pattern. Consequently, less common patterns that are still employed in commercial cameras are often neglected. In this work, we present a generalized framework to represent the image formation model of such cameras. This model is then exploited by our proposed demosaicking algorithm to reconstruct the datacube of interest with a Bayesian approach, using a total variation regularizer as prior. Some preliminary experimental results are also presented, which apply to the reconstruction of acquisitions of various RGBW cameras.

The ImSPOC snapshot imaging spectrometer: image formation model and device characterization

In recent years, the demand for capturing spectral information with finer detail has increased, requiring hyperspectral imaging devices capable of acquiring the required information with increased temporal, spatial and spectral resolution. In this work, we present the image acquisition model of the Image SPectrometer On Chip (ImSPOC), a novel compact snapshot image spectrometer based on the interferometry of Fabry-Perot. Additionally, we propose the interferometer response characterization algorithm (IRCA), a robust three-step procedure to characterize the ImSPOC device that estimates the optical parameters of the composing interferometers' transfer function. The proposed algorithm processes the image output from a set of monochromatic light sources, refining the results through nonlinear regression after an ad-hoc initialization. Experimental analysis confirms the performances of the proposed approach for the characterization of four different ImSPOC prototypes. The source code associated to this paper is available at https://github.com/danaroth83/irca.

Joint Demosaicing and Fusion of Multiresolution Compressed Acquisitions: Image Formation and Reconstruction Methods

Novel optical imaging devices allow for hybrid acquisition modalities such as compressed acquisitions with locally different spatial and spectral resolutions captured by the same focal plane array. In this work, we propose to model a multiresolution compressed acquisition (MRCA) in a generic framework, which natively includes acquisitions by conventional systems such as those based on spectral/color filter arrays, compressed coded apertures, and multiresolution sensing. We propose a model-based image reconstruction algorithm performing a joint demosaicing and fusion (JoDeFu) of any acquisition modeled in the MRCA framework. The JoDeFu reconstruction algorithm solves an inverse problem with a proximal splitting technique and is able to reconstruct an uncompressed image datacube at the highest available spatial and spectral resolution. An implementation of the code is available at https://github.com/danaroth83/jodefu.