SEMPAI: a Self-Enhancing Multi-Photon Artificial Intelligence for Prior-Informed Assessment of Muscle Function and Pathology

Deep learning (DL) shows notable success in biomedical studies. However, most DL algorithms work as black boxes, exclude biomedical experts, and need extensive data. This is especially problematic for fundamental research in the laboratory, where often only small and sparse data are available and the objective is knowledge discovery rather than automation. Furthermore, basic research is usually hypothesis-driven and extensive prior knowledge (priors) exists. To address this, the Self-Enhancing Multi-Photon Artificial Intelligence (SEMPAI) that is designed for multiphoton microscopy (MPM)-based laboratory research is presented. It utilizes meta-learning to optimize prior (and hypothesis) integration, data representation, and neural network architecture simultaneously. By this, the method allows hypothesis testing with DL and provides interpretable feedback about the origin of biological information in 3D images. SEMPAI performs multi-task learning of several related tasks to enable prediction for small datasets. SEMPAI is applied on an extensive MPM database of single muscle fibers from a decade of experiments, resulting in the largest joint analysis of pathologies and function for single muscle fibers to date. It outperforms state-of-the-art biomarkers in six of seven prediction tasks, including those with scarce data. SEMPAI's DL models with integrated priors are superior to those without priors and to prior-only approaches.

How scan parameter choice affects deep learning-based coronary artery disease assessment from computed tomography

Recently, algorithms capable of assessing the severity of Coronary Artery Disease (CAD) in form of the Coronary Artery Disease-Reporting and Data System (CAD-RADS) grade from Coronary Computed Tomography Angiography (CCTA) scans using Deep Learning (DL) were proposed. Before considering to apply these algorithms in clinical practice, their robustness regarding different commonly used Computed Tomography (CT)-specific image formation parameters-including denoising strength, slab combination, and reconstruction kernel-needs to be evaluated. For this study, we reconstructed a data set of 500 patient CCTA scans under seven image formation parameter configurations. We select one default configuration and evaluate how varying individual parameters impacts the performance and stability of a typical algorithm for automated CAD assessment from CCTA. This algorithm consists of multiple preprocessing and a DL prediction step. We evaluate the influence of the parameter changes on the entire pipeline and additionally on only the DL step by propagating the centerline extraction results of the default configuration to all others. We consider the standard deviation of the CAD severity prediction grade difference between the default and variation configurations to assess the stability w.r.t. parameter changes. For the full pipeline we observe slight instability (± 0.226 CAD-RADS) for all variations. Predictions are more stable with centerlines propagated from the default to the variation configurations (± 0.122 CAD-RADS), especially for differing denoising strengths (± 0.046 CAD-RADS). However, stacking slabs with sharp boundaries instead of mixing slabs in overlapping regions (called true stack ± 0.313 CAD-RADS) and increasing the sharpness of the reconstruction kernel (± 0.150 CAD-RADS) leads to unstable predictions. Regarding the clinically relevant tasks of excluding CAD (called rule-out; AUC default 0.957, min 0.937) and excluding obstructive CAD (called hold-out; AUC default 0.971, min 0.964) the performance remains on a high level for all variations. Concluding, an influence of reconstruction parameters on the predictions is observed. Especially, scans reconstructed with the true stack parameter need to be treated with caution when using a DL-based method. Also, reconstruction kernels which are underrepresented in the training data increase the prediction uncertainty.

Myofibrillar Lattice Remodeling Is a Structural Cytoskeletal Predictor of Diaphragm Muscle Weakness in a Fibrotic mdx (mdx Cmah-/-) Model

Duchenne muscular dystrophy (DMD) is a degenerative genetic myopathy characterized by complete absence of dystrophin. Although the mdx mouse lacks dystrophin, its phenotype is milder compared to DMD patients. The incorporation of a null mutation in the Cmah gene led to a more DMD-like phenotype (i.e., more fibrosis). Although fibrosis is thought to be the major determinant of ‘structural weakness’, intracellular remodeling of myofibrillar geometry was shown to be a major cellular determinant thereof. To dissect the respective contribution to muscle weakness, we assessed biomechanics and extra- and intracellular architecture of whole muscle and single fibers from extensor digitorum longus (EDL) and diaphragm. Despite increased collagen contents in both muscles, passive stiffness in mdx Cmah−/− diaphragm was similar to wt mice (EDL muscles were twice as stiff). Isometric twitch and tetanic stresses were 50% reduced in mdx Cmah−/− diaphragm (15% in EDL). Myofibrillar architecture was severely compromised in mdx Cmah−/− single fibers of both muscle types, but more pronounced in diaphragm. Our results show that the mdx Cmah−/− genotype reproduces DMD-like fibrosis but is not associated with changes in passive visco-elastic muscle stiffness. Furthermore, detriments in active isometric force are compatible with the pronounced myofibrillar disarray of the dystrophic background.

Label-free analysis of inflammatory tissue remodeling in murine lung tissue based on multiphoton microscopy, Raman spectroscopy and machine learning

Inflammatory fibrotic tissue remodeling can lead to severe morbidity. Histopathology grading requires extraction of biopsies and elaborate tissue processing. Label-free optical technologies can provide diagnostic readout without preparation and artificial stainings and show potential for in vivo applications. Here, we present an integration of Raman spectroscopy (RS) and multiphoton microscopy for joint investigation of the bio-chemical composition and morphological features related to cellular components and connective tissue. Both modalities show that collagen signatures were significantly increased in a murine fibrosis model. Furthermore, autofluorescence signatures assigned to immune cells show high correlation with disease severity. RS indicates increased levels of elastin and lipids. Further, we investigated the effect of joint data sets on prediction performance in machine learning models. Although binary classification did not benefit from adding more features, multi-class classification was improved by integrated data sets.

Quantitative Imaging Biomarkers of the Whole Liver Tumor Burden Improve Survival Prediction in Metastatic Pancreatic Cancer

Simple Summary

Finding prognostic biomarkers and associated models with high accuracy in patients with pancreatic cancer remains a challenge. The aim of this study was to analyze whether the combination of quantitative imaging biomarkers based on geometric and radiomics analysis of whole liver tumor burden and established clinical parameters improves the prediction of survival in patients with metastatic pancreatic cancer. In this retrospective study a total of 75 patients with pancreatic cancer and liver metastases were analyzed. Segmentations of whole liver tumor burden from baseline contrast-enhanced CT images were used to derive different quantitative imaging biomarkers. For comparison, we chose two clinical prognostic models from the literature. We found that a combined clinical and imaging-based model has a significantly higher predictive performance to discriminate survival than the underlying clinical models alone (p < 0.003).

Abstract

Finding prognostic biomarkers with high accuracy in patients with pancreatic cancer (PC) remains a challenging problem. To improve the prediction of survival and to investigate the relevance of quantitative imaging biomarkers (QIB) we combined QIB with established clinical parameters. In this retrospective study a total of 75 patients with metastatic PC and liver metastases were analyzed. Segmentations of whole liver tumor burden (WLTB) from baseline contrast-enhanced CT images were used to derive QIBs. The benefits of QIBs in multivariable Cox models were analyzed in comparison with two clinical prognostic models from the literature. To discriminate survival, the two clinical models had concordance indices of 0.61 and 0.62 in a statistical setting. Combined clinical and imaging-based models achieved concordance indices of 0.74 and 0.70 with WLTB volume, tumor burden score (TBS), and bilobar disease being the three WLTB parameters that were kept by backward elimination. These combined clinical and imaging-based models have significantly higher predictive performance in discriminating survival than the underlying clinical models alone (p < 0.003). Radiomics and geometric WLTB analysis of patients with metastatic PC with liver metastases enhances the modeling of survival compared with models based on clinical parameters alone.

Unraveling the interplay of image formation, data representation and learning in CT-based COPD phenotyping automation: The need for a meta-strategy

PURPOSE

In the literature on automated phenotyping of chronic obstructive pulmonary disease (COPD), there is a multitude of isolated classical machine learning and deep learning techniques, mostly investigating individual phenotypes, with small study cohorts and heterogeneous meta-parameters, e.g., different scan protocols or segmented regions. The objective is to compare the impact of different experimental setups, i.e., varying meta-parameters related to image formation and data representation, with the impact of the learning technique for subtyping automation for a variety of phenotypes. The identified associations of these parameters with automation performance and their interactions might be a first step towards a determination of optimal meta-parameters, i.e., a meta-strategy.

METHODS

A clinical cohort of 981 patients (53.8 ± 15.1 years, 554 male) was examined. The inspiratory CT images were analyzed to automate the diagnosis of 13 COPD phenotypes given by two radiologists. A benchmark feature set that integrates many quantitative criteria was extracted from the lung and trained a variety of learning algorithms on the first 654 patients (two thirds) and the respective algorithm retrospectively assessed the remaining 327 patients (one third). The automation performance was evaluated by the area under the receiver operating characteristic curve (AUC). 1717 experiments were conducted with varying meta-parameters such as reconstruction kernel, segmented regions and input dimensionality, i.e., number of extracted features. The association of the meta-parameters with the automation performance was analyzed by multivariable general linear model decomposition of the automation performance in the contributions of meta-parameters and the learning technique.

RESULTS

The automation performance varied strongly for varying meta-parameters. For emphysema-predominant phenotypes, an AUC of 93%-95% could be achieved for the best meta-configuration. The airways-predominant phenotypes led to a lower performance of 65%-85%, while smooth kernel configurations on average were unexpectedly superior to those with sharp kernels. The performance impact of meta-parameters, even that of often neglected ones like the missing-data imputation, was in general larger than that of the learning technique. Advanced learning techniques like 3D deep learning or automated machine learning yielded inferior automation performance for non-optimal meta-configurations in comparison to simple techniques with suitable meta-configurations. The best automation performance was achieved by a combination of modern learning techniques and a suitable meta-configuration.

CONCLUSIONS

Our results indicate that for COPD phenotype automation, study design parameters such as reconstruction kernel and the model input dimensionality should be adapted to the learning technique and may be more important than the technique itself. To achieve optimal automation and prediction results, the interaction between input those meta-parameters and the learning technique should be considered. This might be particularly relevant for the development of specific scan protocols for novel learning algorithms, and towards an understanding of good study design for automated phenotyping.

The relevance of CT-based geometric and radiomics analysis of whole liver tumor burden to predict survival of patients with metastatic colorectal cancer

OBJECTIVES

To investigate the prediction of 1-year survival (1-YS) in patients with metastatic colorectal cancer with use of a systematic comparative analysis of quantitative imaging biomarkers (QIBs) based on the geometric and radiomics analysis of whole liver tumor burden (WLTB) in comparison to predictions based on the tumor burden score (TBS), WLTB volume alone, and a clinical model.

METHODS

A total of 103 patients (mean age: 61.0 ± 11.2 years) with colorectal liver metastases were analyzed in this retrospective study. Automatic segmentations of WLTB from baseline contrast-enhanced CT images were used. Established biomarkers as well as a standard radiomics model building were used to derive 3 prognostic models. The benefits of a geometric metastatic spread (GMS) model, the Aerts radiomics prior model of the WLTB, and the performance of TBS and WLTB volume alone were assessed. All models were analyzed in both statistical and predictive machine learning settings in terms of AUC.

RESULTS

TBS showed the best discriminative performance in a statistical setting to discriminate 1-YS (AUC = 0.70, CI: [0.56, 0.90]). For the machine learning-based prediction for unseen patients, both a model of the GMS of WLTB (0.73, CI: [0.60, 0.84]) and the Aerts radiomics prior model (0.76, CI: [0.65, 0.86]) applied on the WLTB showed a numerically higher predictive performance than TBS (0.68, CI: [0.54, 0.79]), radiomics (0.65, CI: [0.55, 0.78]), WLTB volume alone (0.53, CI: [0.40. 0.66]), or the clinical model (0.56, CI: [0.43, 0.67]).

CONCLUSIONS

The imaging-based GMS model may be a first step towards a more fine-grained machine learning extension of the TBS concept for risk stratification in mCRC patients without the vulnerability to technical variance of radiomics.

KEY POINTS

• CT-based geometric distribution and radiomics analysis of whole liver tumor burden in metastatic colorectal cancer patients yield prognostic information. • Differences in survival are possibly attributable to the spatial distribution of metastatic lesions and the geometric metastatic spread analysis of all liver metastases may serve as robust imaging biomarker invariant to technical variation. • Imaging-based prediction models outperform clinical models for 1-year survival prediction in metastatic colorectal cancer patients with liver metastases.

Three-dimensional Distribution of Muscle and Adipose Tissue of the Thigh at CT: Association with Acute Hip Fracture

Purpose To evaluate determinants of hip fracture by assessing soft-tissue composition of the upper thigh at CT. Materials and Methods In this retrospective analysis of prospectively collected data, CT studies in 55 female control participants (mean age, 73.1 years ± 9.3 [standard deviation]) were compared with those in 40 female patients (mean age, 80.2 years ± 11.0) with acute hip fractures. Eighty-seven descriptors of the soft-tissue composition were determined. A multivariable best subsets analysis was used to extract parameters best associated with hip fracture. Results were adjusted for age, height, and weight. Results of soft-tissue parameters were compared with bone mineral density (BMD) and cortical bone thickness. Areas under the receiver operating characteristic curve (AUCs) adjusted for multiple comparisons were determined to discriminate fracture. Results The hip fracture group was characterized by lower BMD, lower cortical thickness, lower relative adipose tissue volume of the upper thigh, and higher extramyocellular lipid (EML) surface density. The relative volume of adipose tissue combined with EML surface density (model S1) was associated with hip fracture (AUC, 0.85; 95% confidence interval [CI]: 0.78, 0.93), as well as trochanteric trabecular BMD combined with neck cortical thickness (model B2) (AUC, 0.84; 95% CI: 0.75, 0.92). The model including all four parameters provided significantly better (P < .01) discrimination (AUC, 0.92; 95% CI: 0.86, 0.97) than model S1 or B2. Conclusion In addition to bone mineral density and geometry of the proximal femur, the amount of adipose tissue of the upper thigh and the distribution of the adipocytes in the muscles are significantly associated with acute hip fracture at CT. © RSNA, 2018 Online supplemental material is available for this article.

A reproducible semi-automatic method to quantify the muscle-lipid distribution in clinical 3D CT images of the thigh

Many studies use threshold-based techniques to assess in vivo the muscle, bone and adipose tissue distribution of the legs using computed tomography (CT) imaging. More advanced techniques divide the legs into subcutaneous adipose tissue (SAT), anatomical muscle (muscle tissue and adipocytes within the muscle border) and intra- and perimuscular adipose tissue. In addition, a so-called muscle density directly derived from the CT-values is often measured. We introduce a new integrated approach to quantify the muscle-lipid system (MLS) using quantitative CT in patients with sarcopenia or osteoporosis. The analysis targets the thigh as many CT studies of the hip do not include entire legs The framework consists of an anatomic coordinate system, allowing delineation of reproducible volumes of interest, a robust semi-automatic 3D segmentation of the fascia and a comprehensive method to quantify of the muscle and lipid distribution within the fascia. CT density-dependent features are calibrated using subject-specific internal CT values of the SAT and external CT values of an in scan calibration phantom. Robustness of the framework with respect to operator interaction, image noise and calibration was evaluated. Specifically, the impact of inter- and intra-operator reanalysis precision and addition of Gaussian noise to simulate lower radiation exposure on muscle and AT volumes, muscle density and 3D texture features quantifying MLS within the fascia, were analyzed. Existing data of 25 subjects (age: 75.6 ± 8.7) with porous and low-contrast muscle structures were included in the analysis. Intra- and inter-operator reanalysis precision errors were below 1% and mostly comparable to 1% of cohort variation of the corresponding features. Doubling the noise changed most 3D texture features by up to 15% of the cohort variation but did not affect density and volume measurements. The application of the novel technique is easy with acceptable processing time. It can thus be employed for a comprehensive quantification of the muscle-lipid system enabling radiomics approaches to musculoskeletal disorders.