Reliable Prostate Cancer Risk Mapping From MRI Using Targeted and Systematic Core Needle Biopsy Histopathology

OBJECTIVE

To compute a dense prostate cancer risk map for the individual patient post-biopsy from magnetic resonance imaging (MRI) and to provide a more reliable evaluation of its fitness in prostate regions that were not identified as suspicious for cancer by a human-reader in pre- and intra-biopsy imaging analysis.

METHODS

Low-level pre-biopsy MRI biomarkers from targeted and non-targeted biopsy locations were extracted and statistically tested for representativeness against biomarkers from non-biopsied prostate regions. A probabilistic machine learning classifier was optimized to map biomarkers to their core-level pathology, followed by extrapolation of pathology scores to non-biopsied prostate regions. Goodness-of-fit was assessed at targeted and non-targeted biopsy locations for the post-biopsy individual patient.

RESULTS

Our experiments showed high predictability of imaging biomarkers in differentiating histopathology scores in thousands of non-targeted core-biopsy locations (ROC-AUCs: 0.85-0.88), but also high variability between patients (Median ROC-AUC [IQR]: 0.81-0.89 [0.29-0.40]).

CONCLUSION

The sparseness of prostate biopsy data makes the validation of a whole gland risk mapping a non-trivial task. Previous studies i) focused on targeted-biopsy locations although biopsy-specimens drawn from systematically scattered locations across the prostate constitute a more representative sample to non-biopsied regions, and ii) estimated prediction-power across predicted instances (e.g., biopsy specimens) with no patient distinction, which may lead to unreliable estimation of model fitness to the individual patient due to variation between patients in instance count, imaging characteristics, and pathologies.

SIGNIFICANCE

This study proposes a personalized whole-gland prostate cancer risk mapping post-biopsy to allow clinicians to better stage and personalize focal therapy treatment plans.

Automated MRI liver segmentation for anatomical segmentation, liver volumetry, and the extraction of radiomics

OBJECTIVES

To develop and evaluate a deep convolutional neural network (DCNN) for automated liver segmentation, volumetry, and radiomic feature extraction on contrast-enhanced portal venous phase magnetic resonance imaging (MRI).

MATERIALS AND METHODS

This retrospective study included hepatocellular carcinoma patients from an institutional database with portal venous MRI. After manual segmentation, the data was randomly split into independent training, validation, and internal testing sets. From a collaborating institution, de-identified scans were used for external testing. The public LiverHccSeg dataset was used for further external validation. A 3D DCNN was trained to automatically segment the liver. Segmentation accuracy was quantified by the Dice similarity coefficient (DSC) with respect to manual segmentation. A Mann-Whitney U test was used to compare the internal and external test sets. Agreement of volumetry and radiomic features was assessed using the intraclass correlation coefficient (ICC).

RESULTS

In total, 470 patients met the inclusion criteria (63.9±8.2 years; 376 males) and 20 patients were used for external validation (41±12 years; 13 males). DSC segmentation accuracy of the DCNN was similarly high between the internal (0.97±0.01) and external (0.96±0.03) test sets (p=0.28) and demonstrated robust segmentation performance on public testing (0.93±0.03). Agreement of liver volumetry was satisfactory in the internal (ICC, 0.99), external (ICC, 0.97), and public (ICC, 0.85) test sets. Radiomic features demonstrated excellent agreement in the internal (mean ICC, 0.98±0.04), external (mean ICC, 0.94±0.10), and public (mean ICC, 0.91±0.09) datasets.

CONCLUSION

Automated liver segmentation yields robust and generalizable segmentation performance on MRI data and can be used for volumetry and radiomic feature extraction.

CLINICAL RELEVANCE STATEMENT

Liver volumetry, anatomic localization, and extraction of quantitative imaging biomarkers require accurate segmentation, but manual segmentation is time-consuming. A deep convolutional neural network demonstrates fast and accurate segmentation performance on T1-weighted portal venous MRI.

KEY POINTS

• This deep convolutional neural network yields robust and generalizable liver segmentation performance on internal, external, and public testing data. • Automated liver volumetry demonstrated excellent agreement with manual volumetry. • Automated liver segmentations can be used for robust and reproducible radiomic feature extraction.

LiverHccSeg: A publicly available multiphasic MRI dataset with liver and HCC tumor segmentations and inter-rater agreement analysis

Accurate segmentation of liver and tumor regions in medical imaging is crucial for the diagnosis, treatment, and monitoring of hepatocellular carcinoma (HCC) patients. However, manual segmentation is time-consuming and subject to inter- and intra-rater variability. Therefore, automated methods are necessary but require rigorous validation of high-quality segmentations based on a consensus of raters. To address the need for reliable and comprehensive data in this domain, we present LiverHccSeg, a dataset that provides liver and tumor segmentations on multiphasic contrast-enhanced magnetic resonance imaging from two board-approved abdominal radiologists, along with an analysis of inter-rater agreement. LiverHccSeg provides a curated resource for liver and HCC tumor segmentation tasks. The dataset includes a scientific reading and co-registered contrast-enhanced multiphasic magnetic resonance imaging (MRI) scans with corresponding manual segmentations by two board-approved abdominal radiologists and relevant metadata and offers researchers a comprehensive foundation for external validation, and benchmarking of liver and tumor segmentation algorithms. The dataset also provides an analysis of the agreement between the two sets of liver and tumor segmentations. Through the calculation of appropriate segmentation metrics, we provide insights into the consistency and variability in liver and tumor segmentations among the radiologists. A total of 17 cases were included for liver segmentation and 14 cases for HCC tumor segmentation. Liver segmentations demonstrates high segmentation agreement (mean Dice, 0.95 ± 0.01 [standard deviation]) and HCC tumor segmentations showed higher variation (mean Dice, 0.85 ± 0.16 [standard deviation]). The applications of LiverHccSeg can be manifold, ranging from testing machine learning algorithms on public external data to radiomic feature analyses. Leveraging the inter-rater agreement analysis within the dataset, researchers can investigate the impact of variability on segmentation performance and explore methods to enhance the accuracy and robustness of liver and tumor segmentation algorithms in HCC patients. By making this dataset publicly available, LiverHccSeg aims to foster collaborations, facilitate innovative solutions, and ultimately improve patient outcomes in the diagnosis and treatment of HCC.

Fast Reconstruction for Deep Learning PET Head Motion Correction

Head motion correction is an essential component of brain PET imaging, in which even motion of small magnitude can greatly degrade image quality and introduce artifacts. Building upon previous work, we propose a new head motion correction framework taking fast reconstructions as input. The main characteristics of the proposed method are: (i) the adoption of a high-resolution short-frame fast reconstruction workflow; (ii) the development of a novel encoder for PET data representation extraction; and (iii) the implementation of data augmentation techniques. Ablation studies are conducted to assess the individual contributions of each of these design choices. Furthermore, multi-subject studies are conducted on an 18F-FPEB dataset, and the method performance is qualitatively and quantitatively evaluated by MOLAR reconstruction study and corresponding brain Region of Interest (ROI) Standard Uptake Values (SUV) evaluation. Additionally, we also compared our method with a conventional intensity-based registration method. Our results demonstrate that the proposed method outperforms other methods on all subjects, and can accurately estimate motion for subjects out of the training set. All code is publicly available on GitHub: https://github.com/OnofreyLab/dl-hmc_fast_recon_miccai2023.

Cross-Attention for Improved Motion Correction in Brain PET

Head movement during long scan sessions degrades the quality of reconstruction in positron emission tomography (PET) and introduces artifacts, which limits clinical diagnosis and treatment. Recent deep learning-based motion correction work utilized raw PET list-mode data and hardware motion tracking (HMT) to learn head motion in a supervised manner. However, motion prediction results were not robust to testing subjects outside the training data domain. In this paper, we integrate a cross-attention mechanism into the supervised deep learning network to improve motion correction across test subjects. Specifically, cross-attention learns the spatial correspondence between the reference images and moving images to explicitly focus the model on the most correlative inherent information - the head region the motion correction. We validate our approach on brain PET data from two different scanners: HRRT without time of flight (ToF) and mCT with ToF. Compared with traditional and deep learning benchmarks, our network improved the performance of motion correction by 58% and 26% in translation and rotation, respectively, in multi-subject testing in HRRT studies. In mCT studies, our approach improved performance by 66% and 64% for translation and rotation, respectively. Our results demonstrate that cross-attention has the potential to improve the quality of brain PET image reconstruction without the dependence on HMT. All code will be released on GitHub: https://github.com/OnofreyLab/dl_hmc_attention_mlcn2023.

Deep learning of image-derived measures of body composition in pediatric, adolescent, and young adult lymphoma: association with late treatment effects

OBJECTIVES

The objective of this study was to translate a deep learning (DL) approach for semiautomated analysis of body composition (BC) measures from standard of care CT images to investigate the prognostic value of BC in pediatric, adolescent, and young adult (AYA) patients with lymphoma.

METHODS

This 10-year retrospective, single-site study of 110 pediatric and AYA patients with lymphoma involved manual segmentation of fat and muscle tissue from 260 CT imaging datasets obtained as part of routine imaging at initial staging and first therapeutic follow-up. A DL model was trained to perform semiautomated image segmentation of adipose and muscle tissue. The association between BC measures and the occurrence of 3-year late effects was evaluated using Cox proportional hazards regression analyses.

RESULTS

DL-guided measures of BC were in close agreement with those obtained by a human rater, as demonstrated by high Dice scores (≥ 0.95) and correlations (r > 0.99) for each tissue of interest. Cox proportional hazards regression analyses revealed that patients with elevated subcutaneous adipose tissue at baseline and first follow-up, along with patients who possessed lower volumes of skeletal muscle at first follow-up, have increased risk of late effects compared to their peers.

CONCLUSIONS

DL provides rapid and accurate quantification of image-derived measures of BC that are associated with risk for treatment-related late effects in pediatric and AYA patients with lymphoma. Image-based monitoring of BC measures may enhance future opportunities for personalized medicine for children with lymphoma by identifying patients at the highest risk for late effects of treatment.

KEY POINTS

• Deep learning-guided CT image analysis of body composition measures achieved high agreement level with manual image analysis. • Pediatric patients with more fat and less muscle during the course of cancer treatment were more likely to experience a serious adverse event compared to their clinical counterparts. • Deep learning of body composition may add value to routine CT imaging by offering real-time monitoring of pediatric, adolescent, and young adults at high risk for late effects of cancer treatment.

DuSFE: Dual-Channel Squeeze-Fusion-Excitation co-attention for cross-modality registration of cardiac SPECT and CT

Myocardial perfusion imaging (MPI) using single-photon emission computed tomography (SPECT) is widely applied for the diagnosis of cardiovascular diseases. Attenuation maps (μ-maps) derived from computed tomography (CT) are utilized for attenuation correction (AC) to improve the diagnostic accuracy of cardiac SPECT. However, in clinical practice, SPECT and CT scans are acquired sequentially, potentially inducing misregistration between the two images and further producing AC artifacts. Conventional intensity-based registration methods show poor performance in the cross-modality registration of SPECT and CT-derived μ-maps since the two imaging modalities might present totally different intensity patterns. Deep learning has shown great potential in medical imaging registration. However, existing deep learning strategies for medical image registration encoded the input images by simply concatenating the feature maps of different convolutional layers, which might not fully extract or fuse the input information. In addition, deep-learning-based cross-modality registration of cardiac SPECT and CT-derived μ-maps has not been investigated before. In this paper, we propose a novel Dual-Channel Squeeze-Fusion-Excitation (DuSFE) co-attention module for the cross-modality rigid registration of cardiac SPECT and CT-derived μ-maps. DuSFE is designed based on the co-attention mechanism of two cross-connected input data streams. The channel-wise or spatial features of SPECT and μ-maps are jointly encoded, fused, and recalibrated in the DuSFE module. DuSFE can be flexibly embedded at multiple convolutional layers to enable gradual feature fusion in different spatial dimensions. Our studies using clinical patient MPI studies demonstrated that the DuSFE-embedded neural network generated significantly lower registration errors and more accurate AC SPECT images than existing methods. We also showed that the DuSFE-embedded network did not over-correct or degrade the registration performance of motion-free cases. The source code of this work is available at https://github.com/XiongchaoChen/DuSFE_CrossRegistration.

MULTI-TASK DEEP LEARNING AND UNCERTAINTY ESTIMATION FOR PET HEAD MOTION CORRECTION

Head motion occurring during brain positron emission tomography images acquisition leads to a decrease in image quality and induces quantification errors. We have previously introduced a Deep Learning Head Motion Correction (DL-HMC) method based on supervised learning of gold-standard Polaris Vicra motion tracking device and showed the potential of this method. In this study, we upgrade our network to a multi-task architecture in order to include image appearance prediction in the learning process. This multi-task Deep Learning Head Motion Correction (mtDL-HMC) model was trained on 21 subjects and showed enhanced motion prediction performance compared to our previous DL-HMC method on both quantitative and qualitative results for 5 testing subjects. We also evaluate the trustworthiness of network predictions by performing Monte Carlo Dropout at inference on testing subjects. We discard the data associated with a great motion prediction uncertainty and show that this does not harm the quality of reconstructed images, and can even improve it.

INTEGRATING PROSTATE SPECIFIC ANTIGEN DENSITY BIOMARKER INTO DEEP LEARNING PROSTATE MRI LESION SEGMENTATION MODELS

Prostate cancer lesion segmentation in multi-parametric magnetic resonance imaging (mpMRI) is crucial for pre-biopsy diagnosis and targeted biopsy guidance. Deep convolution neural networks have been widely utilized for lesion segmentation. However, these methods fail to achieve a high Dice coefficient because of the large variations in lesion size and location within the gland. To address this problem, we integrate the clinically-meaningful prostate specific antigen density (PSAD) biomarker into the deep learning model using feature-wise transformations to condition the features in latent space, and thus control the size of lesion prediction. We tested our models on a public dataset with 214 annotated mpMRI scans and compared the segmentation performance to a baseline 3D U-Net model. Results demonstrate that integrating the PSAD biomarker significantly improves segmentation performance in both Dice coefficient and centroid distance metric.

Inter-pass motion correction for whole-body dynamic PET and parametric imaging

Whole-body dynamic FDG-PET imaging through continuous-bed-motion (CBM) mode multi-pass acquisition protocol is a promising metabolism measurement. However, inter-pass misalignment originating from body movement could degrade parametric quantification. We aim to apply a non-rigid registration method for inter-pass motion correction in whole-body dynamic PET. 27 subjects underwent a 90-min whole-body FDG CBM PET scan on a Biograph mCT (Siemens Healthineers), acquiring 9 over-the-heart single-bed passes and subsequently 19 CBM passes (frames). The inter-pass motion correction was executed using non-rigid image registration with multi-resolution, B-spline free-form deformations. The parametric images were then generated by Patlak analysis. The overlaid Patlak slope Ki and y-intercept Vb images were visualized to qualitatively evaluate motion impact and correction effect. The normalized weighted mean squared Patlak fitting errors (NFE) were compared in the whole body, head, and hypermetabolic regions of interest (ROI). In Ki images, ROI statistics were collected and malignancy discrimination capacity was estimated by the area under the receiver operating characteristic curve (AUC). After the inter-pass motion correction was applied, the spatial misalignment appearance between Ki and Vb images was successfully reduced. Voxel-wise normalized fitting error maps showed global error reduction after motion correction. The NFE in the whole body (p = 0.0013), head (p = 0.0021), and ROIs (p = 0.0377) significantly decreased. The visual performance of each hypermetabolic ROI in Ki images was enhanced, while 3.59% and 3.67% average absolute percentage changes were observed in mean and maximum Ki values, respectively, across all evaluated ROIs. The estimated mean Ki values had substantial changes with motion correction (p = 0.0021). The AUC of both mean Ki and maximum Ki after motion correction increased, possibly suggesting the potential of enhancing oncological discrimination capacity through inter-pass motion correction.

Co-attention spatial transformer network for unsupervised motion tracking and cardiac strain analysis in 3D echocardiography

Myocardial ischemia/infarction causes wall-motion abnormalities in the left ventricle. Therefore, reliable motion estimation and strain analysis using 3D+time echocardiography for localization and characterization of myocardial injury is valuable for early detection and targeted interventions. Previous unsupervised cardiac motion tracking methods rely on heavily-weighted regularization functions to smooth out the noisy displacement fields in echocardiography. In this work, we present a Co-Attention Spatial Transformer Network (STN) for improved motion tracking and strain analysis in 3D echocardiography. Co-Attention STN aims to extract inter-frame dependent features between frames to improve the motion tracking in otherwise noisy 3D echocardiography images. We also propose a novel temporal constraint to further regularize the motion field to produce smooth and realistic cardiac displacement paths over time without prior assumptions on cardiac motion. Our experimental results on both synthetic and in vivo 3D echocardiography datasets demonstrate that our Co-Attention STN provides superior performance compared to existing methods. Strain analysis from Co-Attention STNs also correspond well with the matched SPECT perfusion maps, demonstrating the clinical utility for using 3D echocardiography for infarct localization.

Deep learning-based attenuation map generation with simultaneously reconstructed PET activity and attenuation and low-dose application

Objective. In PET/CT imaging, CT is used for positron emission tomography (PET) attenuation correction (AC). CT artifacts or misalignment between PET and CT can cause AC artifacts and quantification errors in PET. Simultaneous reconstruction (MLAA) of PET activity (λ-MLAA) and attenuation (μ-MLAA) maps was proposed to solve those issues using the time-of-flight PET raw data only. However,λ-MLAA still suffers from quantification error as compared to reconstruction using the gold-standard CT-based attenuation map (μ-CT). Recently, a deep learning (DL)-based framework was proposed to improve MLAA by predictingμ-DL fromλ-MLAA andμ-MLAA using an image domain loss function (IM-loss). However, IM-loss does not directly measure the AC errors according to the PET attenuation physics. Our preliminary studies showed that an additional physics-based loss function can lead to more accurate PET AC. The main objective of this study is to optimize the attenuation map generation framework for clinical full-dose¹⁸F-FDG studies. We also investigate the effectiveness of the optimized network on predicting attenuation maps for synthetic low-dose oncological PET studies.Approach. We optimized the proposed DL framework by applying different preprocessing steps and hyperparameter optimization, including patch size, weights of the loss terms and number of angles in the projection-domain loss term. The optimization was performed based on 100 skull-to-toe¹⁸F-FDG PET/CT scans with minimal misalignment. The optimized framework was further evaluated on 85 clinical full-dose neck-to-thigh¹⁸F-FDG cancer datasets as well as synthetic low-dose studies with only 10% of the full-dose raw data.Main results. Clinical evaluation of tumor quantification as well as physics-based figure-of-merit metric evaluation validated the promising performance of our proposed method. For both full-dose and low-dose studies, the proposed framework achieved <1% error in tumor standardized uptake value measures.Significance. It is of great clinical interest to achieve CT-less PET reconstruction, especially for low-dose PET studies.

An objective evaluation method for head motion estimation in PET-Motion corrected centroid-of-distribution

Head motion presents a continuing problem in brain PET studies. A wealth of motion correction (MC) algorithms had been proposed in the past, including both hardware-based methods and data-driven methods. However, in most real brain PET studies, in the absence of ground truth or gold standard of motion information, it is challenging to objectively evaluate MC quality. For MC evaluation, image-domain metrics, e.g., standardized uptake value (SUV) change before and after MC are commonly used, but this measure lacks objectivity because 1) other factors, e.g., attenuation correction, scatter correction and parameters used in the reconstruction, will confound MC effectiveness; 2) SUV only reflects final image quality, and it cannot precisely inform when an MC method performed well or poorly during the scan time period; 3) SUV is tracer-dependent and head motion may cause increases or decreases in SUV for different tracers, so evaluating MC effectiveness is complicated. Here, we present a new algorithm, i.e., motion corrected centroid-of-distribution (MCCOD) to perform objective quality control for measured or estimated rigid motion information. MCCOD is a three-dimensional surrogate trace of the center of tracer distribution after performing rigid MC using the existing motion information. MCCOD is used to inform whether the motion information is accurate, using the PET raw data only, i.e., without PET image reconstruction, where inaccurate motion information typically leads to abrupt changes in the MCCOD trace. MCCOD was validated using simulation studies and was tested on real studies acquired from both time-of-flight (TOF) and non-TOF scanners. A deep learning-based brain mask segmentation was implemented, which is shown to be necessary for non-TOF MCCOD generation. MCCOD is shown to be effective in detecting abrupt translation motion errors in slowly varying tracer distribution caused by the motion tracking hardware and can be used to compare different motion estimation methods as well as to improve existing motion information.

Supervised Deep Learning for Head Motion Correction in PET

Head movement is a major limitation in brain positron emission tomography (PET) imaging, which results in image artifacts and quantification errors. Head motion correction plays a critical role in quantitative image analysis and diagnosis of nervous system diseases. However, to date, there is no approach that can track head motion continuously without using an external device. Here, we develop a deep learning-based algorithm to predict rigid motion for brain PET by lever-aging existing dynamic PET scans with gold-standard motion measurements from external Polaris Vicra tracking. We propose a novel Deep Learning for Head Motion Correction (DL-HMC) methodology that consists of three components: (i) PET input data encoder layers; (ii) regression layers to estimate the six rigid motion transformation parameters; and (iii) feature-wise transformation (FWT) layers to condition the network to tracer time-activity. The input of DL-HMC is sampled pairs of one-second 3D cloud representations of the PET data and the output is the prediction of six rigid transformation motion parameters. We trained this network in a supervised manner using the Vicra motion tracking information as gold-standard. We quantitatively evaluate DL-HMC by comparing to gold-standard Vicra measurements and qualitatively evaluate the reconstructed images as well as perform region of interest standard uptake value (SUV) measurements. An algorithm ablation study was performed to determine the contributions of each of our DL-HMC design choices to network performance. Our results demonstrate accurate motion prediction performance for brain PET using a data-driven registration approach without external motion tracking hardware. All code is publicly available on GitHub: https://github.com/OnofreyLab/dl-hmc_miccai2022.

Prediction of Adverse Pathology at Radical Prostatectomy in Grade Group 2 and 3 Prostate Biopsies Using Machine Learning

PURPOSE

There is ongoing clinical need to improve estimates of disease outcome in prostate cancer. Machine learning (ML) approaches to pathologic diagnosis and prognosis are a promising and increasingly used strategy. In this study, we use an ML algorithm for prediction of adverse outcomes at radical prostatectomy (RP) using whole-slide images (WSIs) of prostate biopsies with Grade Group (GG) 2 or 3 disease.

METHODS

We performed a retrospective review of prostate biopsies collected at our institution which had corresponding RP, GG 2 or 3 disease one or more cores, and no biopsies with higher than GG 3 disease. A hematoxylin and eosin-stained core needle biopsy from each site with GG 2 or 3 disease was scanned and used as the sole input for the algorithm. The ML pipeline had three phases: image preprocessing, feature extraction, and adverse outcome prediction. First, patches were extracted from each biopsy scan. Subsequently, the pre-trained Visual Geometry Group-16 convolutional neural network was used for feature extraction. A representative feature vector was then used as input to an Extreme Gradient Boosting classifier for predicting the binary adverse outcome. We subsequently assessed patient clinical risk using CAPRA score for comparison with the ML pipeline results.

RESULTS

The data set included 361 WSIs from 107 patients (56 with adverse pathology at RP). The area under the receiver operating characteristic curves for the ML classification were 0.72 (95% CI, 0.62 to 0.81), 0.65 (95% CI, 0.53 to 0.79) and 0.89 (95% CI, 0.79 to 1.00) for the entire cohort, and GG 2 and GG 3 patients, respectively, similar to the performance of the CAPRA clinical risk assessment.

CONCLUSION

We provide evidence for the potential of ML algorithms to use WSIs of needle core prostate biopsies to estimate clinically relevant prostate cancer outcomes.

Deep learning-based attenuation correction for whole-body PET - a multi-tracer study with 18F-FDG, 68 Ga-DOTATATE, and 18F-Fluciclovine

A novel deep learning (DL)-based attenuation correction (AC) framework was applied to clinical whole-body oncology studies using 18F-FDG, 68 Ga-DOTATATE, and 18F-Fluciclovine. The framework used activity (λ-MLAA) and attenuation (µ-MLAA) maps estimated by the maximum likelihood reconstruction of activity and attenuation (MLAA) algorithm as inputs to a modified U-net neural network with a novel imaging physics-based loss function to learn a CT-derived attenuation map (µ-CT).

METHODS

Clinical whole-body PET/CT datasets of 18F-FDG (N = 113), 68 Ga-DOTATATE (N = 76), and 18F-Fluciclovine (N = 90) were used to train and test tracer-specific neural networks. For each tracer, forty subjects were used to train the neural network to predict attenuation maps (µ-DL). µ-DL and µ-MLAA were compared to the gold-standard µ-CT. PET images reconstructed using the OSEM algorithm with µ-DL (OSEMDL) and µ-MLAA (OSEMMLAA) were compared to the CT-based reconstruction (OSEMCT). Tumor regions of interest were segmented by two radiologists and tumor SUV and volume measures were reported, as well as evaluation using conventional image analysis metrics.

RESULTS

µ-DL yielded high resolution and fine detail recovery of the attenuation map, which was superior in quality as compared to µ-MLAA in all metrics for all tracers. Using OSEMCT as the gold-standard, OSEMDL provided more accurate tumor quantification than OSEMMLAA for all three tracers, e.g., error in SUVmax for OSEMMLAA vs. OSEMDL: - 3.6 ± 4.4% vs. - 1.7 ± 4.5% for 18F-FDG (N = 152), - 4.3 ± 5.1% vs. 0.4 ± 2.8% for 68 Ga-DOTATATE (N = 70), and - 7.3 ± 2.9% vs. - 2.8 ± 2.3% for 18F-Fluciclovine (N = 44). OSEMDL also yielded more accurate tumor volume measures than OSEMMLAA, i.e., - 8.4 ± 14.5% (OSEMMLAA) vs. - 3.0 ± 15.0% for 18F-FDG, - 14.1 ± 19.7% vs. 1.8 ± 11.6% for 68 Ga-DOTATATE, and - 15.9 ± 9.1% vs. - 6.4 ± 6.4% for 18F-Fluciclovine.

CONCLUSIONS

The proposed framework provides accurate and robust attenuation correction for whole-body 18F-FDG, 68 Ga-DOTATATE and 18F-Fluciclovine in tumor SUV measures as well as tumor volume estimation. The proposed method provides clinically equivalent quality as compared to CT in attenuation correction for the three tracers.

Adaptive data-driven motion detection and optimized correction for brain PET

Head motion during PET scans causes image quality degradation, decreased concentration in regions with high uptake and incorrect outcome measures from kinetic analysis of dynamic datasets. Previously, we proposed a data-driven method, center of tracer distribution (COD), to detect head motion without an external motion tracking device. There, motion was detected using one dimension of the COD trace with a semiautomatic detection algorithm, requiring multiple user defined parameters and manual intervention. In this study, we developed a new data-driven motion detection algorithm, which is automatic, self-adaptive to noise level, does not require user-defined parameters and uses all three dimensions of the COD trace (3DCOD). 3DCOD was first validated and tested using 30 simulation studies (¹⁸F-FDG, N = 15; ¹¹C-raclopride (RAC), N = 15) with large motion. The proposed motion correction method was tested on 22 real human datasets, with 20 acquired from a high resolution research tomograph (HRRT) scanner (¹⁸F-FDG, N = 10; ¹¹C-RAC, N = 10) and 2 acquired from the Siemens Biograph mCT scanner. Real-time hardware-based motion tracking information (Vicra) was available for all real studies and was used as the gold standard. 3DCOD was compared to Vicra, no motion correction (NMC), one-direction COD (our previous method called 1DCOD) and two conventional frame-based image registration (FIR) algorithms, i.e., FIR1 (based on predefined frames reconstructed with attenuation correction) and FIR2 (without attenuation correction) for both simulation and real studies. For the simulation studies, 3DCOD yielded -2.3 ± 1.4% (mean ± standard deviation across all subjects and 11 brain regions) error in region of interest (ROI) uptake for ¹⁸F-FDG (-3.4 ± 1.7% for ¹¹C-RAC across all subjects and 2 regions) as compared to Vicra (perfect correction) while NMC, FIR1, FIR2 and 1DCOD yielded -25.4 ± 11.1% (-34.5 ± 16.1% for ¹¹C- RAC), -13.4 ± 3.5% (-16.1 ± 4.6%), -5.7 ± 3.6% (-8.0 ± 4.5%) and -2.6 ± 1.5% (-5.1 ± 2.7%), respectively. For real HRRT studies, 3DCOD yielded -0.3 ± 2.8% difference for ¹⁸F-FDG (-0.4 ± 3.2% for ¹¹C-RAC) as compared to Vicra while NMC, FIR1, FIR2 and 1DCOD yielded -14.9 ± 9.0% (-24.5 ± 14.6%), -3.6 ± 4.9% (-13.4 ± 14.3%), -0.6 ± 3.4% (-6.7 ± 5.3%) and -1.5 ± 4.2% (-2.2 ± 4.1%), respectively. In summary, the proposed motion correction method yielded comparable performance to the hardware-based motion tracking method for multiple tracers, including very challenging cases with large frequent head motion, in studies performed on a non-TOF scanner.

MVnet: automated time-resolved tracking of the mitral valve plane in CMR long-axis cine images with residual neural networks: a multi-center, multi-vendor study

BACKGROUND

Mitral annular plane systolic excursion (MAPSE) and left ventricular (LV) early diastolic velocity (e') are key metrics of systolic and diastolic function, but not often measured by cardiovascular magnetic resonance (CMR). Its derivation is possible with manual, precise annotation of the mitral valve (MV) insertion points along the cardiac cycle in both two and four-chamber long-axis cines, but this process is highly time-consuming, laborious, and prone to errors. A fully automated, consistent, fast, and accurate method for MV plane tracking is lacking. In this study, we propose MVnet, a deep learning approach for MV point localization and tracking capable of deriving such clinical metrics comparable to human expert-level performance, and validated it in a multi-vendor, multi-center clinical population.

METHODS

The proposed pipeline first performs a coarse MV point annotation in a given cine accurately enough to apply an automated linear transformation task, which standardizes the size, cropping, resolution, and heart orientation, and second, tracks the MV points with high accuracy. The model was trained and evaluated on 38,854 cine images from 703 patients with diverse cardiovascular conditions, scanned on equipment from 3 main vendors, 16 centers, and 7 countries, and manually annotated by 10 observers. Agreement was assessed by the intra-class correlation coefficient (ICC) for both clinical metrics and by the distance error in the MV plane displacement. For inter-observer variability analysis, an additional pair of observers performed manual annotations in a randomly chosen set of 50 patients.

RESULTS

MVnet achieved a fast segmentation (<1 s/cine) with excellent ICCs of 0.94 (MAPSE) and 0.93 (LV e') and a MV plane tracking error of -0.10 ± 0.97 mm. In a similar manner, the inter-observer variability analysis yielded ICCs of 0.95 and 0.89 and a tracking error of -0.15 ± 1.18 mm, respectively.

CONCLUSION

A dual-stage deep learning approach for automated annotation of MV points for systolic and diastolic evaluation in CMR long-axis cine images was developed. The method is able to carefully track these points with high accuracy and in a timely manner. This will improve the feasibility of CMR methods which rely on valve tracking and increase their utility in a clinical setting.

Improved performance and consistency of deep learning 3D liver segmentation with heterogeneous cancer stages in magnetic resonance imaging

PURPOSE

Accurate liver segmentation is key for volumetry assessment to guide treatment decisions. Moreover, it is an important pre-processing step for cancer detection algorithms. Liver segmentation can be especially challenging in patients with cancer-related tissue changes and shape deformation. The aim of this study was to assess the ability of state-of-the-art deep learning 3D liver segmentation algorithms to generalize across all different Barcelona Clinic Liver Cancer (BCLC) liver cancer stages.

METHODS

This retrospective study, included patients from an institutional database that had arterial-phase T1-weighted magnetic resonance images with corresponding manual liver segmentations. The data was split into 70/15/15% for training/validation/testing each proportionally equal across BCLC stages. Two 3D convolutional neural networks were trained using identical U-net-derived architectures with equal sized training datasets: one spanning all BCLC stages ("All-Stage-Net": AS-Net), and one limited to early and intermediate BCLC stages ("Early-Intermediate-Stage-Net": EIS-Net). Segmentation accuracy was evaluated by the Dice Similarity Coefficient (DSC) on a dataset spanning all BCLC stages and a Wilcoxon signed-rank test was used for pairwise comparisons.

RESULTS

219 subjects met the inclusion criteria (170 males, 49 females, 62.8±9.1 years) from all BCLC stages. Both networks were trained using 129 subjects: AS-Net training comprised 19, 74, 18, 8, and 10 BCLC 0, A, B, C, and D patients, respectively; EIS-Net training comprised 21, 86, and 22 BCLC 0, A, and B patients, respectively. DSCs (mean±SD) were 0.954±0.018 and 0.946±0.032 for AS-Net and EIS-Net (p<0.001), respectively. The AS-Net 0.956±0.014 significantly outperformed the EIS-Net 0.941±0.038 on advanced BCLC stages (p<0.001) and yielded similarly good segmentation performance on early and intermediate stages (AS-Net: 0.952±0.021; EIS-Net: 0.949±0.027; p = 0.107).

CONCLUSION

To ensure robust segmentation performance across cancer stages that is independent of liver shape deformation and tumor burden, it is critical to train deep learning models on heterogeneous imaging data spanning all BCLC stages.

Magnetic resonance image connectivity analysis provides evidence of central nervous system mode of action for parasacral transcutaneous electro neural stimulation - A pilot study

INTRODUCTION

Parasacral transcutaneous electriconeural stimulation (pTENS) is a common treatment modality for patients with overactive bladder (OAB). Its mechanism of effectiveness has yet to be elucidated. Recent studies with fMRI in adults with implanted sacral nerve stimulators impute its effectiveness on changes in the brain involving the anterior cingulate cortex (ACC) and prefrontal cortex (PFC).

AIM

The study set out to evaluate brain connectivity utilizing functional MRI to the outline the mechanism of action of pTENS in the brain.

METHODS

Ten adult volunteers without urinary tract symptoms underwent fMRI. Electrodes were placed on the skin at sacral level (S2) (Experimental Stimulation - pTENS) and on the right scapular region (Sham Stimulation - sTENS). Stimulation was done twice on each site for 6 min at a frequency of 10 Hz and pulse width of 260 μs and intensity determined by the motor threshold. A 6 min resting state fMRI was also done twice as control. Functional connectivity data was acquired during each state (resting, pTENS and sTENS). Standard functional connectivity preprocessing was performed. Seed connectivity was examined to investigate changes in ACC functional connectivity between the stimulations and resting-state conditions. Significance was assessed at p < 0.05 corrected for multiple comparisons.

RESULTS

For all conditions (pTENS, sTENS, and rest), standard patterns of ACC connectivity were detectable with strong connectivity between the ACC and subcortical regions and between the ACC and the frontal lobe. Functional connectivity between ACC seed and the dorsal lateral prefrontal cortex (DLPFC) was significantly increased during pTENS compared to rest. sTENS did not increase connectivity between the ACC seed and DLPFC when compared to rest.

DISCUSSION

Preliminary results indicate that ACC is a major site of activation during pTENS. Increased connectivity between ACC and DLPFC may be a possible mechanism of pTENS effectiveness, which appears to be specific to pTENS compared to sTENS. This study is limited to the small size at this time which prevents further investigation at other sites in the brain.

CONCLUSIONS

The study confirms our original aim which was to define if parasacral TENS actually has a central effect.

Noise reduction with cross-tracer and cross-protocol deep transfer learning for low-dose PET

Previous studies have demonstrated the feasibility of reducing noise with deep learning-based methods for low-dose fluorodeoxyglucose (FDG) positron emission tomography (PET). This work aimed to investigate the feasibility of noise reduction for tracers without sufficient training datasets using a deep transfer learning approach, which can utilize existing networks trained by the widely available FDG datasets. In this study, the deep transfer learning strategy based on a fully 3D patch-based U-Net was investigated on a 18F-fluoromisonidazole (18F-FMISO) dataset using single-bed scanning and a 68Ga-DOTATATE dataset using whole-body scanning. The datasets of 18F-FDG by single-bed scanning and whole-body scanning were used to obtain pre-trained U-Nets separately for subsequent cross-tracer and cross-protocol transfer learning. The full-dose PET images were used as the labels while low-dose PET images from 10% counts were used as the inputs. Three types of U-Nets were obtained: a U-Net trained by FDG dataset, a pre-trained FDG U-Net fine-tuned by another less-available tracer (FMISO/DOATATE), and a U-Net completely trained by a large number of less-available tracer datasets (FMISO/DOATATE), used as the reference U-Net. The denoising performance of the three types of U-Nets was evaluated on single-bed 18F-FMISO and whole-body 68Ga-DOTATATE separately and compared using normalized root-mean-square error (NRMSE), signal-to-noise ratio (SNR), and relative bias of region of interest (ROI). For cross-tracer transfer learning, all the U-Nets provided denoised images with similar quality for both tracers. There was no significant difference in terms of NRMSE and SNR when comparing the former two U-Nets with the reference U-Net. The ROI biases for these U-Nets were similar. For cross-tracer and cross-protocol transfer learning, the pre-trained single-bed FDG U-Net fine-tuned by whole-body DOTATATE data provided the most consistent images with the reference U-Net. Fine-tuning significantly reduced the NRMSE and the ROI bias and improved the SNR when comparing the fine-tuned U-Net with the U-Net trained by single-bed FDG only (NRMSE: 96.3% ± 21.1% versus 120.6% ± 18.5%, ROI bias: -10.5% ± 13.0% versus -14.7% ± 6.4%, SNR: 4.2 ± 1.4 versus 3.9 ± 1.6, for fine-tuned U-Net and the U-Net trained by single-bed FDG, respectively, with p < 0.01 in all cases). This work demonstrated that it is feasible to utilize existing networks well-trained by FDG datasets to reduce the noise for other less-available tracers and other scanning protocols by using the fine-tuning strategy.

Sparse Data-Driven Learning for Effective and Efficient Biomedical Image Segmentation

Sparsity is a powerful concept to exploit for high-dimensional machine learning and associated representational and computational efficiency. Sparsity is well suited for medical image segmentation. We present a selection of techniques that incorporate sparsity, including strategies based on dictionary learning and deep learning, that are aimed at medical image segmentation and related quantification.

Noninvasive In Vivo Quantification of Adeno-Associated Virus Serotype 9-Mediated Expression of the Sodium/Iodide Symporter Under Hindlimb Ischemia and Neuraminidase Desialylation in Skeletal Muscle Using Single-Photon Emission Computed Tomography/Computed Tomography

BACKGROUND

We propose micro single-photon emission computed tomography/computed tomography imaging of the hNIS (human sodium/iodide symporter) to noninvasively quantify adeno-associated virus 9 (AAV9)-mediated gene expression in a murine model of peripheral artery disease.

METHODS

AAV9-hNIS (2×10¹¹ viral genome particles) was injected into nonischemic or ischemic gastrocnemius muscles of C57Bl/6J mice following unilateral hindlimb ischemia ± the α-sialidase NA (neuraminidase). Control nonischemic limbs were injected with phosphate buffered saline or remained noninjected. Twelve mice underwent micro single-photon emission computed tomography/computed tomography imaging after serial injection of pertechnetate (^99mTcO₄^-), a NIS substrate, up to 28 days after AAV9-hNIS injection. Twenty four animals were euthanized at selected times over 1 month for ex vivo validation. Forty-two animals were imaged with ^99mTcO₄^- ± the selective NIS inhibitor perchlorate on day 10, to ascertain specificity of radiotracer uptake. Tissue was harvested for ex vivo validation. A modified version of the U-Net deep learning algorithm was used for image quantification.

RESULTS

As quantitated by standardized uptake value, there was a gradual temporal increase in ^99mTcO₄^- uptake in muscles treated with AAV9-hNIS. Hindlimb ischemia, NA, and hindlimb ischemia plus NA increased the magnitude of ^99mTcO₄^- uptake by 4- to 5-fold compared with nonischemic muscle treated with only AAV9-hNIS. Perchlorate treatment significantly reduced ^99mTcO₄^- uptake in AAV9-hNIS-treated muscles, demonstrating uptake specificity. The imaging results correlated well with ex vivo well counting (r²=0.9375; P<0.0001) and immunoblot analysis of NIS protein (r²=0.65; P<0.0001).

CONCLUSIONS

Micro single-photon emission computed tomography/computed tomography imaging of hNIS-mediated ^99mTcO₄^- uptake allows for accurate in vivo quantification of AAV9-driven gene expression, which increases under ischemic conditions or neuraminidase desialylation in skeletal muscle.

GENERALIZABLE MULTI-SITE TRAINING AND TESTING OF DEEP NEURAL NETWORKS USING IMAGE NORMALIZATION

The ability of medical image analysis deep learning algorithms to generalize across multiple sites is critical for clinical adoption of these methods. Medical imging data, especially MRI, can have highly variable intensity characteristics across different individuals, scanners, and sites. However, it is not practical to train algorithms with data from all imaging equipment sources at all possible sites. Intensity normalization methods offer a potential solution for working with multi-site data. We evaluate five different image normalization methods on training a deep neural network to segment the prostate gland in MRI. Using 600 MRI prostate gland segmentations from two different sites, our results show that both intra-site and inter-site evaluation is critical for assessing the robustness of trained models and that training with single-site data produces models that fail to fully generalize across testing data from sites not included in the training.

Data-driven voluntary body motion detection and non-rigid event-by-event correction for static and dynamic PET

PET has the potential to perform absolute in vivo radiotracer quantitation. This potential can be compromised by voluntary body motion (BM), which degrades image resolution, alters apparent tracer uptakes, introduces CT-based attenuation correction mismatch artifacts and causes inaccurate parameter estimates in dynamic studies. Existing body motion correction (BMC) methods include frame-based image-registration (FIR) approaches and real-time motion tracking using external measurement devices. FIR does not correct for motion occurring within a pre-defined frame and the device-based method is generally not practical in routine clinical use, since it requires attaching a tracking device to the patient and additional device set up time. In this paper, we proposed a data-driven algorithm, centroid of distribution (COD), to detect BM. In this algorithm, the central coordinate of the time-of-flight (TOF) bin, which can be used as a reasonable surrogate for the annihilation point, is calculated for every event, and averaged over a certain time interval to generate a COD trace. We hypothesized that abrupt changes on the COD trace in lateral direction represent BMs. After detection, BM is estimated using non-rigid image registrations and corrected through list-mode reconstruction. The COD-based BMC approach was validated using a monkey study and was evaluated against FIR using four human and one dog studies with multiple tracers. The proposed approach successfully detected BMs and yielded superior correction results over conventional FIR approaches.

Segmenting the Brain Surface From CT Images With Artifacts Using Locally Oriented Appearance and Dictionary Learning

The accurate segmentation of the brain surface in post-surgical computed tomography (CT) images is critical for image-guided neurosurgical procedures in epilepsy patients. Following surgical implantation of intracranial electrodes, surgeons require accurate registration of the post-implantation CT images to the pre-implantation functional and structural magnetic resonance imaging to guide surgical resection of epileptic tissue. One way to perform the registration is via surface matching. The key challenge in this setup is the CT segmentation, where the extraction of the cortical surface is difficult due to the missing parts of the skull and artifacts introduced from the electrodes. In this paper, we present a dictionary learning-based method to segment the brain surface in post-surgical CT images of epilepsy patients following surgical implantation of electrodes. We propose learning a model of locally oriented appearance that captures both the normal tissue and the artifacts found along this brain surface boundary. Utilizing a database of clinical epilepsy imaging data to train and test our approach, we demonstrate that our method using locally oriented image appearance both more accurately extracts the brain surface and better localizes electrodes on the post-operative brain surface compared to standard, non-oriented appearance modeling. In addition, we compare our method to a standard atlas-based segmentation approach and to a U-Net-based deep convolutional neural network segmentation method.

Respiratory Motion Compensation for PET/CT with Motion Information Derived from Matched Attenuation-Corrected Gated PET Data

Respiratory motion degrades the detection and quantification capabilities of PET/CT imaging. Moreover, mismatch between a fast helical CT image and a time-averaged PET image due to respiratory motion results in additional attenuation correction artifacts and inaccurate localization. Current motion compensation approaches typically have 3 limitations: the mismatch among respiration-gated PET images and the CT attenuation correction (CTAC) map can introduce artifacts in the gated PET reconstructions that can subsequently affect the accuracy of the motion estimation; sinogram-based correction approaches do not correct for intragate motion due to intracycle and intercycle breathing variations; and the mismatch between the PET motion compensation reference gate and the CT image can cause an additional CT-mismatch artifact. In this study, we established a motion correction framework to address these limitations. Methods: In the proposed framework, the combined emission-transmission reconstruction algorithm was used for phase-matched gated PET reconstructions to facilitate the motion model building. An event-by-event nonrigid respiratory motion compensation method with correlations between internal organ motion and external respiratory signals was used to correct both intracycle and intercycle breathing variations. The PET reference gate was automatically determined by a newly proposed CT-matching algorithm. We applied the new framework to 13 human datasets with 3 different radiotracers and 323 lesions and compared its performance with CTAC and non-attenuation correction (NAC) approaches. Validation using 4-dimensional CT was performed for one lung cancer dataset. Results: For the 10 ¹⁸F-FDG studies, the proposed method outperformed (P < 0.006) both the CTAC and the NAC methods in terms of region-of-interest-based SUV_mean, SUV_max, and SUV ratio improvements over no motion correction (SUV_mean: 19.9% vs. 14.0% vs. 13.2%; SUV_max: 15.5% vs. 10.8% vs. 10.6%; SUV ratio: 24.1% vs. 17.6% vs. 16.2%, for the proposed, CTAC, and NAC methods, respectively). The proposed method increased SUV ratios over no motion correction for 94.4% of lesions, compared with 84.8% and 86.4% using the CTAC and NAC methods, respectively. For the 2 ¹⁸F-fluoropropyl-(+)-dihydrotetrabenazine studies, the proposed method reduced the CT-mismatch artifacts in the lower lung where the CTAC approach failed and maintained the quantification accuracy of bone marrow where the NAC approach failed. For the ¹⁸F-FMISO study, the proposed method outperformed both the CTAC and the NAC methods in terms of motion estimation accuracy at 2 lung lesion locations. Conclusion: The proposed PET/CT respiratory event-by-event motion-correction framework with motion information derived from matched attenuation-corrected PET data provides image quality superior to that of the CTAC and NAC methods for multiple tracers.

Learning Non-rigid Deformations for Robust, Constrained Point-based Registration in Image-Guided MR-TRUS Prostate Intervention

Accurate and robust non-rigid registration of pre-procedure magnetic resonance (MR) imaging to intra-procedure trans-rectal ultrasound (TRUS) is critical for image-guided biopsies of prostate cancer. Prostate cancer is one of the most prevalent forms of cancer and the second leading cause of cancer-related death in men in the United States. TRUS-guided biopsy is the current clinical standard for prostate cancer diagnosis and assessment. State-of-the-art, clinical MR-TRUS image fusion relies upon semi-automated segmentations of the prostate in both the MR and the TRUS images to perform non-rigid surface-based registration of the gland. Segmentation of the prostate in TRUS imaging is itself a challenging task and prone to high variability. These segmentation errors can lead to poor registration and subsequently poor localization of biopsy targets, which may result in false-negative cancer detection. In this paper, we present a non-rigid surface registration approach to MR-TRUS fusion based on a statistical deformation model (SDM) of intra-procedural deformations derived from clinical training data. Synthetic validation experiments quantifying registration volume of interest overlaps of the PI-RADS parcellation standard and tests using clinical landmark data demonstrate that our use of an SDM for registration, with median target registration error of 2.98 mm, is significantly more accurate than the current clinical method. Furthermore, we show that the low-dimensional SDM registration results are robust to segmentation errors that are not uncommon in clinical TRUS data.

Learning intervention-induced deformations for non-rigid MR-CT registration and electrode localization in epilepsy patients

This paper describes a framework for learning a statistical model of non-rigid deformations induced by interventional procedures. We make use of this learned model to perform constrained non-rigid registration of pre-procedural and post-procedural imaging. We demonstrate results applying this framework to non-rigidly register post-surgical computed tomography (CT) brain images to pre-surgical magnetic resonance images (MRIs) of epilepsy patients who had intra-cranial electroencephalography electrodes surgically implanted. Deformations caused by this surgical procedure, imaging artifacts caused by the electrodes, and the use of multi-modal imaging data make non-rigid registration challenging. Our results show that the use of our proposed framework to constrain the non-rigid registration process results in significantly improved and more robust registration performance compared to using standard rigid and non-rigid registration methods.

LEARNING NONRIGID DEFORMATIONS FOR CONSTRAINED POINT-BASED REGISTRATION FOR IMAGE-GUIDED MR-TRUS PROSTATE INTERVENTION

This paper presents and validates a low-dimensional nonrigid registration method for fusing magnetic resonance imaging (MRI) and trans-rectal ultrasound (TRUS) in image-guided prostate biopsy. Prostate cancer is one of the most prevalent forms of cancer and the second leading cause of cancer-related death in men in the United States. Conventional clinical practice uses TRUS to guide prostate biopsies when there is a suspicion of cancer. Pre-procedural MRI information can reveal lesions and may be fused with intra-procedure TRUS imaging to provide patient-specific, localization of lesions for targeting. The state-of-the-art MRI-TRUS nonrigid image fusion process relies upon semi-automated segmentation of the prostate in both the MRI and TRUS images. In this paper, we develop a fast, automated nonrigid registration approach to MRI-TRUS fusion based on a statistical deformation model of intra-procedural deformations derived from a clinical sample.

Low-Dimensional Non-Rigid Image Registration Using Statistical Deformation Models From Semi-Supervised Training Data

Accurate and robust image registration is a fundamental task in medical image analysis applications, and requires non-rigid transformations with a large number of degrees of freedom. Statistical deformation models (SDMs) attempt to learn the distribution of non-rigid deformations, and can be used both to reduce the transformation dimensionality and to constrain the registration process. However, high-dimensional SDMs are difficult to train given orders of magnitude fewer training samples. In this paper, we utilize both a small set of annotated imaging data and a large set of unlabeled data to effectively learn an SDM of non-rigid transformations in a semi-supervised training (SST) framework. We demonstrate results applying this framework towards inter-subject registration of skull-stripped, magnetic resonance (MR) brain images. Our approach makes use of 39 labeled MR datasets to create a set of supervised registrations, which we augment with a set of over 1200 unsupervised registrations using unlabeled MRIs. Through leave-one-out cross validation, we show that SST of a non-rigid SDM results in a robust registration algorithm with significantly improved accuracy compared to standard, intensity-based registration, and does so with a 99% reduction in transformation dimensionality.

Segmenting the Brain Surface from CT Images with Artifacts Using Dictionary Learning for Non-rigid MR-CT Registration

This paper presents a dictionary learning-based method to segment the brain surface in post-surgical CT images of epilepsy patients following surgical implantation of electrodes. Using the electrodes identified in the post-implantation CT, surgeons require accurate registration with pre-implantation functional and structural MR imaging to guide surgical resection of epileptic tissue. In this work, we use a surface-based registration method to align the MR and CT brain surfaces. The key challenge here is not the registration, but rather the extraction of the cortical surface from the CT image, which includes missing parts of the skull and artifacts introduced by the electrodes. To segment the brain from these images, we propose learning a model of appearance that captures both the normal tissue and the artifacts found along this brain surface boundary. Using clinical data, we demonstrate that our method both accurately extracts the brain surface and better localizes electrodes than intensity-based rigid and non-rigid registration methods.

Learning nonrigid deformations for constrained multi-modal image registration

We present a new strategy to constrain nonrigid registrations of multi-modal images using a low-dimensional statistical deformation model and test this in registering pre-operative and post-operative images from epilepsy patients. For those patients who may undergo surgical resection for treatment, the current gold-standard to identify regions of seizure involves craniotomy and implantation of intracranial electrodes. To guide surgical resection, surgeons utilize pre-op anatomical and functional MR images in conjunction with post-electrode implantation MR and CT images. The electrode positions from the CT image need to be registered to pre-op functional and structural MR images. The post-op MRI serves as an intermediate registration step between the pre-op MR and CT images. In this work, we propose to bypass the post-op MR image registration step and directly register the pre-op MR and post-op CT images using a low-dimensional nonrigid registration that captures the gross deformation after electrode implantation. We learn the nonrigid deformation characteristics from a principal component analysis of a set of training deformations and demonstrate results using clinical data. We show that our technique significantly outperforms both standard rigid and nonrigid intensity-based registration methods in terms of mean and maximum registration error.

FAST NONRIGID IMAGE REGISTRATION USING STATISTICAL DEFORMATION MODELS LEARNED FROM RICHLY-ANNOTATED DATA

Nonrigid image registrations require a large number of degrees of freedom (DoFs) to capture intersubject anatomical variations. With such high DoFs and lack of anatomical correspondences, algorithms may not converge to the globally optimal solution. In this work, we propose a fast, two-step nonrigid registration procedure with low DoFs to accurately register brain images. Our method makes use of a statistical deformation model based upon a principal component analysis of deformations learned from a manually-segmented dataset to perform an initial registration. We then follow with a low DoF nonrigid transformation to complete the registration. Our results show the same registration accuracy in terms of volume of interest overlap as high DoF transformations, but with a 96% reduction in DoF and 98% decrease in computation time.