Reducing the Human Effort in Developing PET-CT Registration

We aim to reduce the tedious nature of developing and evaluating methods for aligning PET-CT scans from multiple patient visits. Current methods for registration rely on correspondences that are created manually by medical experts with 3D manipulation, or assisted alignments done by utilizing mutual information across CT scans that may not be consistent when transferred to the PET images. Instead, we propose to label multiple key points across several 2D slices, which we then fit a key curve to. This removes the need for creating manual alignments in 3D and makes the labelling process easier. We use these key curves to define an error metric for the alignments that can be computed efficiently. While our metric is non-differentiable, we further show that we can utilize it during the training of our deep model via a novel method. Specifically, instead of relying on detailed geometric labels -- e.g., manual 3D alignments -- we use synthetically generated deformations of real data. To incorporate robustness to changes that occur between visits other than geometric changes, we enforce consistency across visits in the deep network's internal representations. We demonstrate the potential of our method via qualitative and quantitative experiments.

Functional connectivity patterns of autism spectrum disorder identified by deep feature learning

Autism spectrum disorder (ASD) is regarded as a brain disease with globally disrupted neuronal networks. Even though fMRI studies have revealed abnormal functional connectivity in ASD, they have not reached a consensus of the disrupted patterns. Here, a deep learning-based feature extraction method identifies multivariate and nonlinear functional connectivity patterns of ASD. Resting-state fMRI data of 972 subjects (465 ASD 507 normal controls) acquired from the Autism Brain Imaging Data Exchange were used. A functional connectivity matrix of each subject was generated using 90 predefined brain regions. As a data-driven feature extraction method without prior knowledge such as subjects diagnosis, variational autoencoder (VAE) summarized the functional connectivity matrix into 2 features. Those feature values of ASD patients were statistically compared with those of controls. A feature was significantly different between ASD and normal controls. The extracted features were visualized by VAE-based generator which can produce virtual functional connectivity matrices. The ASD-related feature was associated with frontoparietal connections, interconnections of the dorsal medial frontal cortex and corticostriatal connections. It also showed a trend of negative correlation with full-scale IQ. A data-driven feature extraction based on deep learning could identify complex patterns of functional connectivity of ASD. This approach will help discover complex patterns of abnormalities in brain connectivity in various brain disorders.

Predicting Cognitive Decline with Deep Learning of Brain Metabolism and Amyloid Imaging

For effective treatment of Alzheimer disease (AD), it is important to identify subjects who are most likely to exhibit rapid cognitive decline. Herein, we developed a novel framework based on a deep convolutional neural network which can predict future cognitive decline in mild cognitive impairment (MCI) patients using flurodeoxyglucose and florbetapir positron emission tomography (PET). The architecture of the network only relies on baseline PET studies of AD and normal subjects as the training dataset. Feature extraction and complicated image preprocessing including nonlinear warping are unnecessary for our approach. Accuracy of prediction (84.2%) for conversion to AD in MCI patients outperformed conventional feature-based quantification approaches. ROC analyses revealed that performance of CNN-based approach was significantly higher than that of the conventional quantification methods (p < 0.05). Output scores of the network were strongly correlated with the longitudinal change in cognitive measurements. These results show the feasibility of deep learning as a tool for predicting disease outcome using brain images.