Co-Clinical Imaging Resource Program (CIRP): Bridging the Translational Divide to Advance Precision Medicine

Deep learning generation of preclinical positron emission tomography (PET) images from low-count PET with task-based performance assessment

BACKGROUND

Preclinical low-count positron emission tomography (LC-PET) imaging offers numerous advantages such as facilitating imaging logistics, enabling longitudinal studies of long- and short-lived isotopes as well as increasing scanner throughput. However, LC-PET is characterized by reduced photon-count levels resulting in low signal-to-noise ratio (SNR), segmentation difficulties, and quantification uncertainties.

PURPOSE

We developed and evaluated a novel deep-learning (DL) architecture-Attention based Residual-Dilated Net (ARD-Net)-to generate standard-count PET (SC-PET) images from LC-PET images. The performance of the ARD-Net framework was evaluated for numerous low count realizations using fidelity-based qualitative metrics, task-based segmentation, and quantitative metrics.

METHOD

Patient Derived tumor Xenograft (PDX) with tumors implanted in the mammary fat-pad were subjected to preclinical [18F]-Fluorodeoxyglucose (FDG)-PET/CT imaging. SC-PET images were derived from a 10 min static FDG-PET acquisition, 50 min post administration of FDG, and were resampled to generate four distinct LC-PET realizations corresponding to 10%, 5%, 1.6%, and 0.8% of SC-PET count-level. ARD-Net was trained and optimized using 48 preclinical FDG-PET datasets, while 16 datasets were utilized to assess performance. Further, the performance of ARD-Net was benchmarked against two leading DL-based methods (Residual UNet, RU-Net; and Dilated Network, D-Net) and non-DL methods (Non-Local Means, NLM; and Block Matching 3D Filtering, BM3D). The performance of the framework was evaluated using traditional fidelity-based image quality metrics such as Structural Similarity Index Metric (SSIM) and Normalized Root Mean Square Error (NRMSE), as well as human observer-based tumor segmentation performance (Dice Score and volume bias) and quantitative analysis of Standardized Uptake Value (SUV) measurements. Additionally, radiomics-derived features were utilized as a measure of quality assurance (QA) in comparison to true SC-PET. Finally, a performance ensemble score (EPS) was developed by integrating fidelity-based and task-based metrics. Concordance Correlation Coefficient (CCC) was utilized to determine concordance between measures. The non-parametric Friedman Test with Bonferroni correction was used to compare the performance of ARD-Net against benchmarked methods with significance at adjusted p-value ≤0.01.

RESULTS

ARD-Net-generated SC-PET images exhibited significantly better (p ≤ 0.01 post Bonferroni correction) overall image fidelity scores in terms of SSIM and NRMSE at majority of photon-count levels compared to benchmarked DL and non-DL methods. In terms of task-based quantitative accuracy evaluated by SUVMean and SUVPeak, ARD-Net exhibited less than 5% median absolute bias for SUVMean compared to true SC-PET and lower degree of variability compared to benchmarked DL and non-DL based methods in generating SC-PET. Additionally, ARD-Net-generated SC-PET images displayed higher degree of concordance to SC-PET images in terms of radiomics features compared to non-DL and other DL approaches. Finally, the ensemble score suggested that ARD-Net exhibited significantly superior performance compared to benchmarked algorithms (p ≤ 0.01 post Bonferroni correction).

CONCLUSION

ARD-Net provides a robust framework to generate SC-PET from LC-PET images. ARD-Net generated SC-PET images exhibited superior performance compared other DL and non-DL approaches in terms of image-fidelity based metrics, task-based segmentation metrics, and minimal bias in terms of task-based quantification performance for preclinical PET imaging.

Co-Clinical Imaging Metadata Information (CIMI) for Cancer Research to Promote Open Science, Standardization, and Reproducibility in Preclinical Imaging

Preclinical imaging is a critical component in translational research with significant complexities in workflow and site differences in deployment. Importantly, the National Cancer Institute's (NCI) precision medicine initiative emphasizes the use of translational co-clinical oncology models to address the biological and molecular bases of cancer prevention and treatment. The use of oncology models, such as patient-derived tumor xenografts (PDX) and genetically engineered mouse models (GEMMs), has ushered in an era of co-clinical trials by which preclinical studies can inform clinical trials and protocols, thus bridging the translational divide in cancer research. Similarly, preclinical imaging fills a translational gap as an enabling technology for translational imaging research. Unlike clinical imaging, where equipment manufacturers strive to meet standards in practice at clinical sites, standards are neither fully developed nor implemented in preclinical imaging. This fundamentally limits the collection and reporting of metadata to qualify preclinical imaging studies, thereby hindering open science and impacting the reproducibility of co-clinical imaging research. To begin to address these issues, the NCI co-clinical imaging research program (CIRP) conducted a survey to identify metadata requirements for reproducible quantitative co-clinical imaging. The enclosed consensus-based report summarizes co-clinical imaging metadata information (CIMI) to support quantitative co-clinical imaging research with broad implications for capturing co-clinical data, enabling interoperability and data sharing, as well as potentially leading to updates to the preclinical Digital Imaging and Communications in Medicine (DICOM) standard.

The National Cancer Institute's Co-Clinical Quantitative Imaging Research Resources for Precision Medicine in Preclinical and Clinical Settings

Genetically engineered mouse models (GEMMs) and patient-derived xenograft mouse models (PDXs) can recapitulate important biological features of cancer. They are often part of precision medicine studies in a co-clinical setting, in which therapeutic investigations are conducted in patients and in parallel (or sequentially) in cohorts of GEMMs or PDXs. Employing radiology-based quantitative imaging in these studies allows in vivo assessment of disease response in real time, providing an important opportunity to bridge precision medicine from the bench to the bedside. The Co-Clinical Imaging Research Resource Program (CIRP) of the National Cancer Institute focuses on the optimization of quantitative imaging methods to improve co-clinical trials. The CIRP supports 10 different co-clinical trial projects, spanning diverse tumor types, therapeutic interventions, and imaging modalities. Each CIRP project is tasked to deliver a unique web resource to support the cancer community with the necessary methods and tools to conduct co-clinical quantitative imaging studies. This review provides an update of the CIRP web resources, network consensus, technology advances, and a perspective on the future of the CIRP. The presentations in this special issue of Tomography were contributed by the CIRP working groups, teams, and associate members.

Toward Practical Integration of Omic and Imaging Data in Co-Clinical Trials

Co-clinical trials are the concurrent or sequential evaluation of therapeutics in both patients clinically and patient-derived xenografts (PDX) pre-clinically, in a manner designed to match the pharmacokinetics and pharmacodynamics of the agent(s) used. The primary goal is to determine the degree to which PDX cohort responses recapitulate patient cohort responses at the phenotypic and molecular levels, such that pre-clinical and clinical trials can inform one another. A major issue is how to manage, integrate, and analyze the abundance of data generated across both spatial and temporal scales, as well as across species. To address this issue, we are developing MIRACCL (molecular and imaging response analysis of co-clinical trials), a web-based analytical tool. For prototyping, we simulated data for a co-clinical trial in "triple-negative" breast cancer (TNBC) by pairing pre- (T0) and on-treatment (T1) magnetic resonance imaging (MRI) from the I-SPY2 trial, as well as PDX-based T0 and T1 MRI. Baseline (T0) and on-treatment (T1) RNA expression data were also simulated for TNBC and PDX. Image features derived from both datasets were cross-referenced to omic data to evaluate MIRACCL functionality for correlating and displaying MRI-based changes in tumor size, vascularity, and cellularity with changes in mRNA expression as a function of treatment.

Animal Models and Their Role in Imaging-Assisted Co-Clinical Trials

The availability of high-fidelity animal models for oncology research has grown enormously in recent years, enabling preclinical studies relevant to prevention, diagnosis, and treatment of cancer to be undertaken. This has led to increased opportunities to conduct co-clinical trials, which are studies on patients that are carried out parallel to or sequentially with animal models of cancer that mirror the biology of the patients' tumors. Patient-derived xenografts (PDX) and genetically engineered mouse models (GEMM) are considered to be the models that best represent human disease and have high translational value. Notably, one element of co-clinical trials that still needs significant optimization is quantitative imaging. The National Cancer Institute has organized a Co-Clinical Imaging Resource Program (CIRP) network to establish best practices for co-clinical imaging and to optimize translational quantitative imaging methodologies. This overview describes the ten co-clinical trials of investigators from eleven institutions who are currently supported by the CIRP initiative and are members of the Animal Models and Co-clinical Trials (AMCT) Working Group. Each team describes their corresponding clinical trial, type of cancer targeted, rationale for choice of animal models, therapy, and imaging modalities. The strengths and weaknesses of the co-clinical trial design and the challenges encountered are considered. The rich research resources generated by the members of the AMCT Working Group will benefit the broad research community and improve the quality and translational impact of imaging in co-clinical trials.

Repeatability of Quantitative Magnetic Resonance Imaging Biomarkers in the Tibia Bone Marrow of a Murine Myelofibrosis Model

Quantitative MRI biomarkers are sought to replace painful and invasive sequential bone-marrow biopsies routinely used for myelofibrosis (MF) cancer monitoring and treatment assessment. Repeatability of MRI-based quantitative imaging biomarker (QIB) measurements was investigated for apparent diffusion coefficient (ADC), proton density fat fraction (PDFF), and magnetization transfer ratio (MTR) in a JAK2 V617F hematopoietic transplant model of MF. Repeatability coefficients (RCs) were determined for three defined tibia bone-marrow sections (2-9 mm; 10-12 mm; and 12.5-13.5 mm from the knee joint) across 15 diseased mice from 20-37 test-retest pairs. Scans were performed on consecutive days every two weeks for a period of 10 weeks starting 3-4 weeks after transplant. The mean RC with (95% confidence interval (CI)) for these sections, respectively, were for ADC: 0.037 (0.031, 0.050), 0.087 (0.069, 0.116), and 0.030 (0.022, 0.044) μm²/ms; for PDFF: 1.6 (1.3, 2.0), 15.5 (12.5, 20.2), and 25.5 (12.0, 33.0)%; and for MTR: 0.16 (0.14, 0.19), 0.11 (0.09, 0.15), and 0.09 (0.08, 0.15). Change-trend analysis of these QIBs identified a dynamic section within the mid-tibial bone marrow in which confident changes (exceeding RC) could be observed after a four-week interval between scans across all measured MRI-based QIBs. Our results demonstrate the capability to derive quantitative imaging metrics from mouse tibia bone marrow for monitoring significant longitudinal MF changes.

Evaluation of Apparent Diffusion Coefficient Repeatability and Reproducibility for Preclinical MRIs Using Standardized Procedures and a Diffusion-Weighted Imaging Phantom

Relevant to co-clinical trials, the goal of this work was to assess repeatability, reproducibility, and bias of the apparent diffusion coefficient (ADC) for preclinical MRIs using standardized procedures for comparison to performance of clinical MRIs. A temperature-controlled phantom provided an absolute reference standard to measure spatial uniformity of these performance metrics. Seven institutions participated in the study, wherein diffusion-weighted imaging (DWI) data were acquired over multiple days on 10 preclinical scanners, from 3 vendors, at 6 field strengths. Centralized versus site-based analysis was compared to illustrate incremental variance due to processing workflow. At magnet isocenter, short-term (intra-exam) and long-term (multiday) repeatability were excellent at within-system coefficient of variance, wCV [±CI] = 0.73% [0.54%, 1.12%] and 1.26% [0.94%, 1.89%], respectively. The cross-system reproducibility coefficient, RDC [±CI] = 0.188 [0.129, 0.343] µm2/ms, corresponded to 17% [12%, 31%] relative to the reference standard. Absolute bias at isocenter was low (within 4%) for 8 of 10 systems, whereas two high-bias (>10%) scanners were primary contributors to the relatively high RDC. Significant additional variance (>2%) due to site-specific analysis was observed for 2 of 10 systems. Base-level technical bias, repeatability, reproducibility, and spatial uniformity patterns were consistent with human MRIs (scaled for bore size). Well-calibrated preclinical MRI systems are capable of highly repeatable and reproducible ADC measurements.

Co-clinical FDG-PET radiomic signature in predicting response to neoadjuvant chemotherapy in triple-negative breast cancer

Purpose

We sought to exploit the heterogeneity afforded by patient-derived tumor xenografts (PDX) to first, optimize and identify robust radiomic features to predict response to therapy in subtype-matched triple negative breast cancer (TNBC) PDX, and second, to implement PDX-optimized image features in a TNBC co-clinical study to predict response to therapy using machine learning (ML) algorithms.

Methods

TNBC patients and subtype-matched PDX were recruited into a co-clinical FDG-PET imaging trial to predict response to therapy. One hundred thirty-one imaging features were extracted from PDX and human-segmented tumors. Robust image features were identified based on reproducibility, cross-correlation, and volume independence. A rank importance of predictors using ReliefF was used to identify predictive radiomic features in the preclinical PDX trial in conjunction with ML algorithms: classification and regression tree (CART), Naïve Bayes (NB), and support vector machines (SVM). The top four PDX-optimized image features, defined as radiomic signatures (RadSig), from each task were then used to predict or assess response to therapy. Performance of RadSig in predicting/assessing response was compared to SUV_mean, SUV_max, and lean body mass-normalized SUL_peak measures.

Results

Sixty-four out of 131 preclinical imaging features were identified as robust. NB-RadSig performed highest in predicting and assessing response to therapy in the preclinical PDX trial. In the clinical study, the performance of SVM-RadSig and NB-RadSig to predict and assess response was practically identical and superior to SUV_mean, SUV_max, and SUL_peak measures.

Conclusions

We optimized robust FDG-PET radiomic signatures (RadSig) to predict and assess response to therapy in the context of a co-clinical imaging trial.

Supplementary Information

The online version contains supplementary material available at 10.1007/s00259-021-05489-8.

Deep Learning Segmentation of Triple-Negative Breast Cancer (TNBC) Patient Derived Tumor Xenograft (PDX) and Sensitivity of Radiomic Pipeline to Tumor Probability Boundary

Simple Summary

Co-clinical trials are an emerging area of investigation in which a clinical trial is coupled with a corresponding preclinical trial to inform the corresponding clinical trial. The preclinical arm aids in assessing therapeutic efficacy, patient stratification, and designing optimal imaging strategies. There is much interest in harmonizing preclinical and clinical quantitative imaging pipelines. Radiomics is widely explored in clinical imaging to predict response to therapy. In preclinical imaging, high-throughput radiomic analysis is limited by manual delineation of tumor boundaries, which is labor intensive with poor reproducibility. Our proposed deep-learning-based system was trained to automatically segment tumors from multi-contrast MR images and extract radiomic features. The proposed method is highly reproducible with significant correlation in radiomic features. The deployment of this pipeline in the preclinical arm would provide high throughput and reproducible radiomic analysis.

Abstract

Preclinical magnetic resonance imaging (MRI) is a critical component in a co-clinical research pipeline. Importantly, segmentation of tumors in MRI is a necessary step in tumor phenotyping and assessment of response to therapy. However, manual segmentation is time-intensive and suffers from inter- and intra- observer variability and lack of reproducibility. This study aimed to develop an automated pipeline for accurate localization and delineation of TNBC PDX tumors from preclinical T1w and T2w MR images using a deep learning (DL) algorithm and to assess the sensitivity of radiomic features to tumor boundaries. We tested five network architectures including U-Net, dense U-Net, Res-Net, recurrent residual UNet (R2UNet), and dense R2U-Net (D-R2UNet), which were compared against manual delineation by experts. To mitigate bias among multiple experts, the simultaneous truth and performance level estimation (STAPLE) algorithm was applied to create consensus maps. Performance metrics (F1-Score, recall, precision, and AUC) were used to assess the performance of the networks. Multi-contrast D-R2UNet performed best with F1-score = 0.948; however, all networks scored within 1–3% of each other. Radiomic features extracted from D-R2UNet were highly corelated to STAPLE-derived features with 67.13% of T1w and 53.15% of T2w exhibiting correlation ρ ≥ 0.9 (p ≤ 0.05). D-R2UNet-extracted features exhibited better reproducibility relative to STAPLE with 86.71% of T1w and 69.93% of T2w features found to be highly reproducible (CCC ≥ 0.9, p ≤ 0.05). Finally, 39.16% T1w and 13.9% T2w features were identified as insensitive to tumor boundary perturbations (Spearman correlation (−0.4 ≤ ρ ≤ 0.4). We developed a highly reproducible DL algorithm to circumvent manual segmentation of T1w and T2w MR images and identified sensitivity of radiomic features to tumor boundaries.

Respiratory Motion Mitigation and Repeatability of Two Diffusion-Weighted MRI Methods Applied to a Murine Model of Spontaneous Pancreatic Cancer

Respiratory motion and increased susceptibility effects at high magnetic fields pose challenges for quantitative diffusion-weighted MRI (DWI) of a mouse abdomen on preclinical MRI systems. We demonstrate the first application of radial k-space-sampled (RAD) DWI of a mouse abdomen using a genetically engineered mouse model of pancreatic ductal adenocarcinoma (PDAC) on a 4.7 T preclinical scanner equipped with moderate gradient capability. RAD DWI was compared with the echo-planar imaging (EPI)-based DWI method with similar voxel volumes and acquisition times over a wide range of b-values (0.64, 535, 1071, 1478, and 2141 mm²/s). The repeatability metrics are assessed in a rigorous test-retest study (n = 10 for each DWI protocol). The four-shot EPI DWI protocol leads to higher signal-to-noise ratio (SNR) in diffusion-weighted images with persisting ghosting artifacts, whereas the RAD DWI protocol produces relatively artifact-free images over all b-values examined. Despite different degrees of motion mitigation, both RAD DWI and EPI DWI allow parametric maps of apparent diffusion coefficients (ADC) to be produced, and the ADC of the PDAC tumor estimated by the two methods are 1.3 ± 0.24 and 1.5 ± 0.28 × 10^-3 mm²/s, respectively (p = 0.075, n = 10), and those of a water phantom are 3.2 ± 0.29 and 2.8 ± 0.15 × 10^-3 mm²/s, respectively (p = 0.001, n = 10). Bland-Altman plots and probability density function reveal good repeatability for both protocols, whose repeatability metrics do not differ significantly. In conclusion, RAD DWI enables a more effective respiratory motion mitigation but lower SNR, while the performance of EPI DWI is expected to improve with more advanced gradient hardware.