Bayesian reconstruction of sparse raster-scanned mid-infrared optoacoustic signals enables fast, label-free chemical microscopy

Hyperspectral optoacoustic microscopy (OAM) enables obtaining images with label-free biomolecular contrast, offering excellent perspectives as a diagnostic tool to assess freshly excised and unprocessed tissues. However, time-consuming raster-scanning image formation currently limits the translation potential of OAM into the clinical setting-for instance, in intraoperative histopathological assessments-where micrographs of excised tissue need to be taken within a few minutes for fast clinical decision-making. Here, we present a non-data-driven computational framework tailored to enable fast OAM by sparse data acquisition and model-based image reconstruction, termed Bayesian raster-computed optoacoustic microscopy (BayROM). Unlike conventional machine learning, BayROM doesn't require training datasets, but instead, it employs 1) optomechanical system properties to define a forward model and 2) prior knowledge of the imaged samples to facilitate reconstructing images based on the sparsely acquired data. We show that BayROM enables acquiring micrographs ten times faster and with structural similarity (SSIM) indices greater than 0.93 compared to conventional raster scanning microscopy, thus facilitating the clinical translation of OAM for fast, label-free intraoperative histopathology.

A Method for Accurate Spatial Focusing Simulation via Numerical Integration and its Application in Optoacoustic Tomography

The spatial sensitivity of an ultrasound transducer, which strongly influences its suitability for different applications, depends on the shape of the transducer surface. Accurate simulation of these spatial effects is important for transducer characterization and design, and for system response modelling in imaging applications. In optoacoustic imaging, broadband transducers are used to capitalize on the rich frequency content of the signals, but their usage makes highly accurate simulations with general wave equation solvers prohibitively memory- and time-intensive. Therefore, specialized tools for simulating the isolated spatial focusing properties described by the spatial impulse response (SIR) have been developed. However, the challenging numerics of the SIR and the necessity to convolve the SIR with the wave shape generated by the optoacoustic absorber to simulate the system response lead to numerical inaccuracies of SIR-based methods. In addition, the approximation error of these methods cannot be controlled a priori. To circumvent the problems associated with the explicit calculation of SIR, we propose directly computing the convolution of the required wave shape with the SIR, which we call the spatial pulse response (SPR). We demonstrate that by utilizing an h-adaptive cubature algorithm, SPR can be computed with significantly higher accuracy than an SIR-based reference method, and the approximation error can be controlled with a tolerance parameter. In addition, the integration of accurate SPR simulations into model-based optoacoustic image reconstruction is shown to improve image contrast and reduce noise artifacts. Precise system characterization and simulation leads to improved imaging performance, ultimately increasing the value of optoacoustic imaging systems for clinical applications.

Spotlight on nerves: Portable multispectral optoacoustic imaging of peripheral nerve vascularization and morphology

Various morphological and functional parameters of peripheral nerves and their vascular supply are indicative of pathological changes due to injury or disease. Based on recent improvements in optoacoustic image quality, we explore the ability of multispectral optoacoustic tomography, in tandem with ultrasound imaging (OPUS), to investigate the vascular environment and morphology of peripheral nerves in vivo in a pilot study on healthy volunteers. We showcase the unique ability of optoacoustic imaging to visualize the vasa nervorum by observing intraneurial vessels in healthy nerves in vivo for the first time. In addition, we demonstrate that the label-free spectral optoacoustic contrast of the perfused connective tissue of peripheral nerves can be linked to the endogenous contrast of haemoglobin and collagen. We introduce metrics to analyze the composition of tissue based on its optoacoustic contrast and show that the high-resolution spectral contrast reveals specific differences between nervous tissue and reference tissue in the nerve's surrounding. We discuss how this showcased extraction of peripheral nerve characteristics using multispectral optoacoustic and ultrasound imaging can offer new insights into the pathophysiology of nerve damage and neuropathies, for example, in the context of diabetes.

DeepMB: Deep neural network for real-time model-based optoacoustic image reconstruction with adjustable speed of sound

Multispectral optoacoustic tomography (MSOT) is a high-resolution functional imaging modality that can non-invasively access a broad range of pathophysiological phenomena by quantifying the contrast of endogenous chromophores in tissue. Real-time imaging is imperative to translate MSOT into clinical imaging, visualize dynamic pathophysiological changes associated with disease progression, and enable in situ diagnoses. Model-based reconstruction affords state-of-the-art optoacoustic images; however, the advanced image quality provided by model-based reconstruction remains inaccessible during real-time imaging because the algorithm is iterative and computationally demanding. Deep-learning may afford faster reconstructions for real-time optoacoustic imaging, but existing approaches only support oversimplified imaging settings and fail to generalize to in vivo data. In this work, we introduce a novel deep-learning framework, termed DeepMB, to learn the model-based reconstruction operator and infer optoacoustic images with state-of-the-art quality in less than 10 ms per image. DeepMB accurately generalizes to in vivo data after training on synthesized sinograms that are derived from real-world images. The framework affords in-focus images for a broad range of anatomical locations because it supports dynamic adjustment of the reconstruction speed of sound during imaging. Furthermore, DeepMB is compatible with the data rates and image sizes of modern multispectral optoacoustic tomography scanners. We evaluate DeepMB on a diverse dataset of in vivo images and demonstrate that the framework reconstructs images 3000 times faster than the iterative model-based reference method while affording near-identical image qualities. Accurate and real-time image reconstructions with DeepMB can enable full access to the high-resolution and multispectral contrast of handheld optoacoustic tomography.

Self-Supervised Learning from Unlabeled Fundus Photographs Improves Segmentation of the Retina

Fundus photography is the primary method for retinal imaging and essential for diabetic retinopathy prevention. Automated segmentation of fundus photographs would improve the quality, capacity, and cost-effectiveness of eye care screening programs. However, current segmentation methods are not robust towards the diversity in imaging conditions and pathologies typical for real-world clinical applications. To overcome these limitations, we utilized contrastive self-supervised learning to exploit the large variety of unlabeled fundus images in the publicly available EyePACS dataset. We pre-trained an encoder of a U-Net, which we later fine-tuned on several retinal vessel and lesion segmentation datasets. We demonstrate for the first time that by using contrastive self-supervised learning, the pre-trained network can recognize blood vessels, optic disc, fovea, and various lesions without being provided any labels. Furthermore, when fine-tuned on a downstream blood vessel segmentation task, such pre-trained networks achieve state-of-the-art performance on images from different datasets. Additionally, the pre-training also leads to shorter training times and an improved few-shot performance on both blood vessel and lesion segmentation tasks. Altogether, our results showcase the benefits of contrastive self-supervised pre-training which can play a crucial role in real-world clinical applications requiring robust models able to adapt to new devices with only a few annotated samples.

Disentangling the frequency content in optoacoustics

Signals acquired by optoacoustic tomography systems have broadband frequency content that encodes information about structures on different physical scales. Concurrent processing and rendering of such broadband signals may result in images with poor contrast and fidelity due to a bias towards low frequency contributions from larger structures. This problem cannot be addressed by filtering different frequency bands and reconstructing them individually, as this procedure leads to artefacts due to its incompatibility with the entangled frequency content of signals generated by structures of different sizes. Here we introduce frequency-band model-based (fbMB) reconstruction to separate frequency-band-specific optoacoustic image components during image formation, thereby enabling structures of all sizes to be rendered with high fidelity. In order to disentangle the overlapping frequency content of image components, fbMB uses soft priors to achieve an optimal trade-off between localization of the components in frequency bands and their structural integrity. We demonstrate that fbMB produces optoacoustic images with improved contrast and fidelity, which reveal anatomical structures in in vivo images of mice in unprecedented detail. These enhancements further improve the accuracy of spectral unmixing in small vasculature. By offering a precise treatment of the frequency components of optoacoustic signals, fbMB improves the quality, accuracy, and quantification of optoacoustic images and provides a method of choice for optoacoustic reconstructions.

Deep learning based electrical noise removal enables high spectral optoacoustic contrast in deep tissue

Image contrast in multispectral optoacoustic tomography (MSOT) can be severely reduced by electrical noise and interference in the acquired optoacoustic signals. Signal processing techniques have proven insufficient to remove the effects of electrical noise because they typically rely on simplified models and fail to capture complex characteristics of signal and noise. Moreover, they often involve time-consuming processing steps that are unsuited for real-time imaging applications. In this work, we develop and demonstrate a discriminative deep learning (DL) approach to separate electrical noise from optoacoustic signals prior to image reconstruction. The proposed DL algorithm is based on two key features. First, it learns spatiotemporal correlations in both noise and signal by using the entire optoacoustic sinogram as input. Second, it employs training based on a large dataset of experimentally acquired pure noise and synthetic optoacoustic signals. We validated the ability of the trained model to accurately remove electrical noise on synthetic data and on optoacoustic images of a phantom and the human breast. We demonstrate significant enhancements of morphological and spectral optoacoustic images reaching 19% higher blood vessel contrast and localized spectral contrast at depths of more than 2 cm for images acquired in vivo. We discuss how the proposed denoising framework is applicable to clinical multispectral optoacoustic tomography and suitable for real-time operation.