Bayesian reconstruction of sparse raster-scanned mid-infrared optoacoustic signals enables fast, label-free chemical microscopy

Hyperspectral optoacoustic microscopy (OAM) enables obtaining images with label-free biomolecular contrast, offering excellent perspectives as a diagnostic tool to assess freshly excised and unprocessed tissues. However, time-consuming raster-scanning image formation currently limits the translation potential of OAM into the clinical setting-for instance, in intraoperative histopathological assessments-where micrographs of excised tissue need to be taken within a few minutes for fast clinical decision-making. Here, we present a non-data-driven computational framework tailored to enable fast OAM by sparse data acquisition and model-based image reconstruction, termed Bayesian raster-computed optoacoustic microscopy (BayROM). Unlike conventional machine learning, BayROM doesn't require training datasets, but instead, it employs 1) optomechanical system properties to define a forward model and 2) prior knowledge of the imaged samples to facilitate reconstructing images based on the sparsely acquired data. We show that BayROM enables acquiring micrographs ten times faster and with structural similarity (SSIM) indices greater than 0.93 compared to conventional raster scanning microscopy, thus facilitating the clinical translation of OAM for fast, label-free intraoperative histopathology.

A Method for Accurate Spatial Focusing Simulation via Numerical Integration and its Application in Optoacoustic Tomography

The spatial sensitivity of an ultrasound transducer, which strongly influences its suitability for different applications, depends on the shape of the transducer surface. Accurate simulation of these spatial effects is important for transducer characterization and design, and for system response modelling in imaging applications. In optoacoustic imaging, broadband transducers are used to capitalize on the rich frequency content of the signals, but their usage makes highly accurate simulations with general wave equation solvers prohibitively memory- and time-intensive. Therefore, specialized tools for simulating the isolated spatial focusing properties described by the spatial impulse response (SIR) have been developed. However, the challenging numerics of the SIR and the necessity to convolve the SIR with the wave shape generated by the optoacoustic absorber to simulate the system response lead to numerical inaccuracies of SIR-based methods. In addition, the approximation error of these methods cannot be controlled a priori. To circumvent the problems associated with the explicit calculation of SIR, we propose directly computing the convolution of the required wave shape with the SIR, which we call the spatial pulse response (SPR). We demonstrate that by utilizing an h-adaptive cubature algorithm, SPR can be computed with significantly higher accuracy than an SIR-based reference method, and the approximation error can be controlled with a tolerance parameter. In addition, the integration of accurate SPR simulations into model-based optoacoustic image reconstruction is shown to improve image contrast and reduce noise artifacts. Precise system characterization and simulation leads to improved imaging performance, ultimately increasing the value of optoacoustic imaging systems for clinical applications.

Spotlight on nerves: Portable multispectral optoacoustic imaging of peripheral nerve vascularization and morphology

Various morphological and functional parameters of peripheral nerves and their vascular supply are indicative of pathological changes due to injury or disease. Based on recent improvements in optoacoustic image quality, we explore the ability of multispectral optoacoustic tomography, in tandem with ultrasound imaging (OPUS), to investigate the vascular environment and morphology of peripheral nerves in vivo in a pilot study on healthy volunteers. We showcase the unique ability of optoacoustic imaging to visualize the vasa nervorum by observing intraneurial vessels in healthy nerves in vivo for the first time. In addition, we demonstrate that the label-free spectral optoacoustic contrast of the perfused connective tissue of peripheral nerves can be linked to the endogenous contrast of haemoglobin and collagen. We introduce metrics to analyze the composition of tissue based on its optoacoustic contrast and show that the high-resolution spectral contrast reveals specific differences between nervous tissue and reference tissue in the nerve's surrounding. We discuss how this showcased extraction of peripheral nerve characteristics using multispectral optoacoustic and ultrasound imaging can offer new insights into the pathophysiology of nerve damage and neuropathies, for example, in the context of diabetes.

DeepMB: Deep neural network for real-time model-based optoacoustic image reconstruction with adjustable speed of sound

Multispectral optoacoustic tomography (MSOT) is a high-resolution functional imaging modality that can non-invasively access a broad range of pathophysiological phenomena by quantifying the contrast of endogenous chromophores in tissue. Real-time imaging is imperative to translate MSOT into clinical imaging, visualize dynamic pathophysiological changes associated with disease progression, and enable in situ diagnoses. Model-based reconstruction affords state-of-the-art optoacoustic images; however, the advanced image quality provided by model-based reconstruction remains inaccessible during real-time imaging because the algorithm is iterative and computationally demanding. Deep-learning may afford faster reconstructions for real-time optoacoustic imaging, but existing approaches only support oversimplified imaging settings and fail to generalize to in vivo data. In this work, we introduce a novel deep-learning framework, termed DeepMB, to learn the model-based reconstruction operator and infer optoacoustic images with state-of-the-art quality in less than 10 ms per image. DeepMB accurately generalizes to in vivo data after training on synthesized sinograms that are derived from real-world images. The framework affords in-focus images for a broad range of anatomical locations because it supports dynamic adjustment of the reconstruction speed of sound during imaging. Furthermore, DeepMB is compatible with the data rates and image sizes of modern multispectral optoacoustic tomography scanners. We evaluate DeepMB on a diverse dataset of in vivo images and demonstrate that the framework reconstructs images 3000 times faster than the iterative model-based reference method while affording near-identical image qualities. Accurate and real-time image reconstructions with DeepMB can enable full access to the high-resolution and multispectral contrast of handheld optoacoustic tomography.

Self-Supervised Learning from Unlabeled Fundus Photographs Improves Segmentation of the Retina

Fundus photography is the primary method for retinal imaging and essential for diabetic retinopathy prevention. Automated segmentation of fundus photographs would improve the quality, capacity, and cost-effectiveness of eye care screening programs. However, current segmentation methods are not robust towards the diversity in imaging conditions and pathologies typical for real-world clinical applications. To overcome these limitations, we utilized contrastive self-supervised learning to exploit the large variety of unlabeled fundus images in the publicly available EyePACS dataset. We pre-trained an encoder of a U-Net, which we later fine-tuned on several retinal vessel and lesion segmentation datasets. We demonstrate for the first time that by using contrastive self-supervised learning, the pre-trained network can recognize blood vessels, optic disc, fovea, and various lesions without being provided any labels. Furthermore, when fine-tuned on a downstream blood vessel segmentation task, such pre-trained networks achieve state-of-the-art performance on images from different datasets. Additionally, the pre-training also leads to shorter training times and an improved few-shot performance on both blood vessel and lesion segmentation tasks. Altogether, our results showcase the benefits of contrastive self-supervised pre-training which can play a crucial role in real-world clinical applications requiring robust models able to adapt to new devices with only a few annotated samples.

Disentangling the frequency content in optoacoustics

Signals acquired by optoacoustic tomography systems have broadband frequency content that encodes information about structures on different physical scales. Concurrent processing and rendering of such broadband signals may result in images with poor contrast and fidelity due to a bias towards low frequency contributions from larger structures. This problem cannot be addressed by filtering different frequency bands and reconstructing them individually, as this procedure leads to artefacts due to its incompatibility with the entangled frequency content of signals generated by structures of different sizes. Here we introduce frequency-band model-based (fbMB) reconstruction to separate frequency-band-specific optoacoustic image components during image formation, thereby enabling structures of all sizes to be rendered with high fidelity. In order to disentangle the overlapping frequency content of image components, fbMB uses soft priors to achieve an optimal trade-off between localization of the components in frequency bands and their structural integrity. We demonstrate that fbMB produces optoacoustic images with improved contrast and fidelity, which reveal anatomical structures in in vivo images of mice in unprecedented detail. These enhancements further improve the accuracy of spectral unmixing in small vasculature. By offering a precise treatment of the frequency components of optoacoustic signals, fbMB improves the quality, accuracy, and quantification of optoacoustic images and provides a method of choice for optoacoustic reconstructions.

Deep learning based electrical noise removal enables high spectral optoacoustic contrast in deep tissue

Image contrast in multispectral optoacoustic tomography (MSOT) can be severely reduced by electrical noise and interference in the acquired optoacoustic signals. Signal processing techniques have proven insufficient to remove the effects of electrical noise because they typically rely on simplified models and fail to capture complex characteristics of signal and noise. Moreover, they often involve time-consuming processing steps that are unsuited for real-time imaging applications. In this work, we develop and demonstrate a discriminative deep learning (DL) approach to separate electrical noise from optoacoustic signals prior to image reconstruction. The proposed DL algorithm is based on two key features. First, it learns spatiotemporal correlations in both noise and signal by using the entire optoacoustic sinogram as input. Second, it employs training based on a large dataset of experimentally acquired pure noise and synthetic optoacoustic signals. We validated the ability of the trained model to accurately remove electrical noise on synthetic data and on optoacoustic images of a phantom and the human breast. We demonstrate significant enhancements of morphological and spectral optoacoustic images reaching 19% higher blood vessel contrast and localized spectral contrast at depths of more than 2 cm for images acquired in vivo. We discuss how the proposed denoising framework is applicable to clinical multispectral optoacoustic tomography and suitable for real-time operation.

A distance-based loss for smooth and continuous skin layer segmentation in optoacoustic images

Raster-scan optoacoustic mesoscopy (RSOM) is a powerful, non-invasive optical imaging technique for functional, anatomical, and molecular skin and tissue analysis. However, both the manual and the automated analysis of such images are challenging, because the RSOM images have very low contrast, poor signal to noise ratio, and systematic overlaps between the absorption spectra of melanin and hemoglobin. Nonetheless, the segmentation of the epidermis layer is a crucial step for many downstream medical and diagnostic tasks, such as vessel segmentation or monitoring of cancer progression. We propose a novel, shape-specific loss function that overcomes discontinuous segmentations and achieves smooth segmentation surfaces while preserving the same volumetric Dice and IoU. Further, we validate our epidermis segmentation through the sensitivity of vessel segmentation. We found a 20 $\%$ improvement in Dice for vessel segmentation tasks when the epidermis mask is provided as additional information to the vessel segmentation network.

Maximum entropy based non-negative optoacoustic tomographic image reconstruction

Optoacoustic (photoacoustic) tomography reconstructs maps of the initial pressure rise induced by the absorption of light pulses in tissue. In practice, due to inaccurate assumptions in the forward model employed, noise and other experimental factors, the images often contain errors, occasionally manifested as negative values. We present optoacoustic tomography based on an entropy maximization algorithm that uses logarithmic regularization as a potent method for imparting non-negative image reconstruction. We experimentally investigate the performance achieved by the entropy maximization scheme on phantoms and in vivo samples. The findings demonstrate that the proposed scheme reconstructs physically relevant image values devoid of unwanted negative contrast, thus improving quantitative imaging performance.

Eigenspectra optoacoustic tomography achieves quantitative blood oxygenation imaging deep in tissues

Light propagating in tissue attains a spectrum that varies with location due to wavelength-dependent fluence attenuation by tissue optical properties, an effect that causes spectral corruption. Predictions of the spectral variations of light fluence in tissue are challenging since the spatial distribution of optical properties in tissue cannot be resolved in high resolution or with high accuracy by current methods. Spectral corruption has fundamentally limited the quantification accuracy of optical and optoacoustic methods and impeded the long sought-after goal of imaging blood oxygen saturation (sO2) deep in tissues; a critical but still unattainable target for the assessment of oxygenation in physiological processes and disease. We discover a new principle underlying light fluence in tissues, which describes the wavelength dependence of light fluence as an affine function of a few reference base spectra, independently of the specific distribution of tissue optical properties. This finding enables the introduction of a previously undocumented concept termed eigenspectra Multispectral Optoacoustic Tomography (eMSOT) that can effectively account for wavelength dependent light attenuation without explicit knowledge of the tissue optical properties. We validate eMSOT in more than 2000 simulations and with phantom and animal measurements. We find that eMSOT can quantitatively image tissue sO2 reaching in many occasions a better than 10-fold improved accuracy over conventional spectral optoacoustic methods. Then, we show that eMSOT can spatially resolve sO2 in muscle and tumor; revealing so far unattainable tissue physiology patterns. Last, we related eMSOT readings to cancer hypoxia and found congruence between eMSOT tumor sO2 images and tissue perfusion and hypoxia maps obtained by correlative histological analysis.