BOA: A CT-Based Body and Organ Analysis for Radiologists at the Point of Care

PURPOSE

The study aimed to develop the open-source body and organ analysis (BOA), a comprehensive computed tomography (CT) image segmentation algorithm with a focus on workflow integration.

METHODS

The BOA combines 2 segmentation algorithms: body composition analysis (BCA) and TotalSegmentator. The BCA was trained with the nnU-Net framework using a dataset including 300 CT examinations. The CTs were manually annotated with 11 semantic body regions: subcutaneous tissue, muscle, bone, abdominal cavity, thoracic cavity, glands, mediastinum, pericardium, breast implant, brain, and spinal cord. The models were trained using 5-fold cross-validation, and at inference time, an ensemble was used. Afterward, the segmentation efficiency was evaluated on a separate test set comprising 60 CT scans. In a postprocessing step, a tissue segmentation (muscle, subcutaneous adipose tissue, visceral adipose tissue, intermuscular adipose tissue, epicardial adipose tissue, and paracardial adipose tissue) is created by subclassifying the body regions. The BOA combines this algorithm and the open-source segmentation software TotalSegmentator to have an all-in-one comprehensive selection of segmentations. In addition, it integrates into clinical workflows as a DICOM node-triggered service using the open-source Orthanc research PACS (Picture Archiving and Communication System) server to make the automated segmentation algorithms available to clinicians. The BCA model's performance was evaluated using the Sørensen-Dice score. Finally, the segmentations from the 3 different tools (BCA, TotalSegmentator, and BOA) were compared by assessing the overall percentage of the segmented human body on a separate cohort of 150 whole-body CT scans.

RESULTS

The results showed that the BCA outperformed the previous publication, achieving a higher Sørensen-Dice score for the previously existing classes, including subcutaneous tissue (0.971 vs 0.962), muscle (0.959 vs 0.933), abdominal cavity (0.983 vs 0.973), thoracic cavity (0.982 vs 0.965), bone (0.961 vs 0.942), and an overall good segmentation efficiency for newly introduced classes: brain (0.985), breast implant (0.943), glands (0.766), mediastinum (0.880), pericardium (0.964), and spinal cord (0.896). All in all, it achieved a 0.935 average Sørensen-Dice score, which is comparable to the one of the TotalSegmentator (0.94). The TotalSegmentator had a mean voxel body coverage of 31% ± 6%, whereas BCA had a coverage of 75% ± 6% and BOA achieved 93% ± 2%.

CONCLUSIONS

The open-source BOA merges different segmentation algorithms with a focus on workflow integration through DICOM node integration, offering a comprehensive body segmentation in CT images with a high coverage of the body volume.

SAROS: A dataset for whole-body region and organ segmentation in CT imaging

The Sparsely Annotated Region and Organ Segmentation (SAROS) dataset was created using data from The Cancer Imaging Archive (TCIA) to provide a large open-access CT dataset with high-quality annotations of body landmarks. In-house segmentation models were employed to generate annotation proposals on randomly selected cases from TCIA. The dataset includes 13 semantic body region labels (abdominal/thoracic cavity, bones, brain, breast implant, mediastinum, muscle, parotid/submandibular/thyroid glands, pericardium, spinal cord, subcutaneous tissue) and six body part labels (left/right arm/leg, head, torso). Case selection was based on the DICOM series description, gender, and imaging protocol, resulting in 882 patients (438 female) for a total of 900 CTs. Manual review and correction of proposals were conducted in a continuous quality control cycle. Only every fifth axial slice was annotated, yielding 20150 annotated slices from 28 data collections. For the reproducibility on downstream tasks, five cross-validation folds and a test set were pre-defined. The SAROS dataset serves as an open-access resource for training and evaluating novel segmentation models, covering various scanner vendors and diseases.

Elexacaftor/tezacaftor/ivacaftor influences body composition in adults with cystic fibrosis: a fully automated CT-based analysis

A poor nutritional status is associated with worse pulmonary function and survival in people with cystic fibrosis (pwCF). CF transmembrane conductance regulator modulators can improve pulmonary function and body weight, but more data is needed to evaluate its effects on body composition. In this retrospective study, a pre-trained deep-learning network was used to perform a fully automated body composition analysis on chest CTs from 66 adult pwCF before and after receiving elexacaftor/tezacaftor/ivacaftor (ETI) therapy. Muscle and adipose tissues were quantified and divided by bone volume to obtain body size-adjusted ratios. After receiving ETI therapy, marked increases were observed in all adipose tissue ratios among pwCF, including the total adipose tissue ratio (+ 46.21%, p < 0.001). In contrast, only small, but statistically significant increases of the muscle ratio were measured in the overall study population (+ 1.63%, p = 0.008). Study participants who were initially categorized as underweight experienced more pronounced effects on total adipose tissue ratio (p = 0.002), while gains in muscle ratio were equally distributed across BMI categories (p = 0.832). Our findings suggest that ETI therapy primarily affects adipose tissues, not muscle tissue, in adults with CF. These effects are primarily observed among pwCF who were initially underweight. Our findings may have implications for the future nutritional management of pwCF.

Body composition impacts outcome of bronchoscopic lung volume reduction in patients with severe emphysema: a fully automated CT-based analysis

Chronic Obstructive Pulmonary Disease (COPD) is characterized by progressive and irreversible airflow limitation, with individual body composition influencing disease severity. Severe emphysema worsens symptoms through hyperinflation, which can be relieved by bronchoscopic lung volume reduction (BLVR). To investigate how body composition, assessed through CT scans, impacts outcomes in emphysema patients undergoing BLVR. Fully automated CT-based body composition analysis (BCA) was performed in patients with end-stage emphysema receiving BLVR with valves. Post-interventional muscle and adipose tissues were quantified, body size-adjusted, and compared to baseline parameters. Between January 2015 and December 2022, 300 patients with severe emphysema underwent endobronchial valve treatment. Significant improvements were seen in outcome parameters, which were defined as changes in pulmonary function, physical performance, and quality of life (QoL) post-treatment. Muscle volume remained stable (1.632 vs. 1.635 for muscle bone adjusted ratio (BAR) at baseline and after 6 months respectively), while bone adjusted adipose tissue volumes, especially total and pericardial adipose tissue, showed significant increase (2.86 vs. 3.00 and 0.16 vs. 0.17, respectively). Moderate to strong correlations between bone adjusted muscle volume and weaker correlations between adipose tissue volumes and outcome parameters (pulmonary function, QoL and physical performance) were observed. Particularly after 6-month, bone adjusted muscle volume changes positively corresponded to improved outcomes (ΔForced expiratory volume in 1 s [FEV1], r = 0.440; ΔInspiratory vital capacity [IVC], r = 0.397; Δ6Minute walking distance [6MWD], r = 0.509 and ΔCOPD assessment test [CAT], r = -0.324; all p < 0.001). Group stratification by bone adjusted muscle volume changes revealed that groups with substantial muscle gain experienced a greater clinical benefit in pulmonary function improvements, QoL and physical performance (ΔFEV1%, 5.5 vs. 39.5; ΔIVC%, 4.3 vs. 28.4; Δ6MWDm, 14 vs. 110; ΔCATpts, -2 vs. -3.5 for groups with ΔMuscle, BAR% < -10 vs. > 10, respectively). BCA results among patients divided by the minimal clinically important difference for forced expiratory volume of the first second (FEV1) showed significant differences in bone-adjusted muscle and intramuscular adipose tissue (IMAT) volumes and their respective changes after 6 months (ΔMuscle, BAR% -5 vs. 3.4 and ΔIMAT, BAR% -0.62 vs. 0.60 for groups with ΔFEV1 ≤ 100 mL vs > 100 mL). Altered body composition, especially increased muscle volume, is associated with functional improvements in BLVR-treated patients.

Early Metabolic Response by PET Predicts Sensitivity to Next-Line Targeted Therapy in EGFR-Mutated Lung Cancer with Unknown Mechanism of Acquired Resistance

Targeted therapy with epidermal growth factor receptor (EGFR) tyrosine kinase inhibitors (TKIs) has established the precision oncology paradigm in lung cancer. Most patients with EGFR-mutated lung cancer respond but eventually acquire resistance. Methods: Patients exhibiting the EGFR p.T790M resistance biomarker benefit from sequenced targeted therapy with osimertinib. We hypothesized that metabolic response as detected by 18F-FDG PET after short-course osimertinib identifies additional patients susceptible to sequenced therapy. Results: Fourteen patients with EGFR-mutated lung cancer and resistance to first- or second-generation EGFR TKI testing negatively for EGFR p.T790M were enrolled in a phase II study. Five patients (36%) achieved a metabolic 18F-FDG PET response and continued osimertinib. In those, the median duration of treatment was not reached (95% CI, 24 mo to not estimable), median progression-free survival was 18.7 mo (95% CI, 14.6 mo to not estimable), and median overall survival was 41.5 mo. Conclusion: Connecting theranostic osimertinib treatment with early metabolic response assessment by PET enables early identification of patients with unknown mechanisms of TKI resistance who derive dramatic clinical benefit from sequenced osimertinib. This defines a novel paradigm for personalization of targeted therapies in patients with lung cancer dependent on a tractable driver oncogene.

Addressing the Contrast Media Recognition Challenge: A Fully Automated Machine Learning Approach for Predicting Contrast Phases in CT Imaging

OBJECTIVES

Accurately acquiring and assigning different contrast-enhanced phases in computed tomography (CT) is relevant for clinicians and for artificial intelligence orchestration to select the most appropriate series for analysis. However, this information is commonly extracted from the CT metadata, which is often wrong. This study aimed at developing an automatic pipeline for classifying intravenous (IV) contrast phases and additionally for identifying contrast media in the gastrointestinal tract (GIT).

MATERIALS AND METHODS

This retrospective study used 1200 CT scans collected at the investigating institution between January 4, 2016 and September 12, 2022, and 240 CT scans from multiple centers from The Cancer Imaging Archive for external validation. The open-source segmentation algorithm TotalSegmentator was used to identify regions of interest (pulmonary artery, aorta, stomach, portal/splenic vein, liver, portal vein/hepatic veins, inferior vena cava, duodenum, small bowel, colon, left/right kidney, urinary bladder), and machine learning classifiers were trained with 5-fold cross-validation to classify IV contrast phases (noncontrast, pulmonary arterial, arterial, venous, and urographic) and GIT contrast enhancement. The performance of the ensembles was evaluated using the receiver operating characteristic area under the curve (AUC) and 95% confidence intervals (CIs).

RESULTS

For the IV phase classification task, the following AUC scores were obtained for the internal test set: 99.59% [95% CI, 99.58-99.63] for the noncontrast phase, 99.50% [95% CI, 99.49-99.52] for the pulmonary-arterial phase, 99.13% [95% CI, 99.10-99.15] for the arterial phase, 99.8% [95% CI, 99.79-99.81] for the venous phase, and 99.7% [95% CI, 99.68-99.7] for the urographic phase. For the external dataset, a mean AUC of 97.33% [95% CI, 97.27-97.35] and 97.38% [95% CI, 97.34-97.41] was achieved for all contrast phases for the first and second annotators, respectively. Contrast media in the GIT could be identified with an AUC of 99.90% [95% CI, 99.89-99.9] in the internal dataset, whereas in the external dataset, an AUC of 99.73% [95% CI, 99.71-99.73] and 99.31% [95% CI, 99.27-99.33] was achieved with the first and second annotator, respectively.

CONCLUSIONS

The integration of open-source segmentation networks and classifiers effectively classified contrast phases and identified GIT contrast enhancement using anatomical landmarks.

Radiomics Analysis of Multiparametric PET/MRI for N- and M-Staging in Patients with Primary Cervical Cancer

PURPOSE

 The aim of this study was to investigate the potential of multiparametric 18F-FDG PET/MR imaging as a platform for radiomics analysis and machine learning algorithms based on primary cervical cancers to predict N- and M-stage in patients.

MATERIALS AND METHODS

 A total of 30 patients with histopathological confirmation of primary and untreated cervical cancer were prospectively enrolled for a multiparametric 18F-FDG PET/MR examination, comprising a dedicated protocol for imaging of the female pelvis. The primary tumor in the uterine cervix was manually segmented on post-contrast T1-weighted images. Quantitative features were extracted from the segmented tumors using the Radiomic Image Processing Toolbox for the R software environment for statistical computing and graphics. 45 different image features were calculated from non-enhanced as well as post-contrast T1-weighted TSE images, T2-weighted TSE images, the ADC map, the parametric Ktrans, Kep, Ve and iAUC maps and PET images, respectively. Statistical analysis and modeling was performed using Python 3.5 and the scikit-learn software machine learning library for the Python programming language.

RESULTS

 Prediction of M-stage was superior when compared to N-stage. Prediction of M-stage using SVM with SVM-RFE as feature selection obtained the highest performance providing sensitivity of 91 % and specificity of 92 %. Using receiver operating characteristic (ROC) analysis of the pooled predictions, the area under the curve (AUC) was 0.97. Prediction of N-stage using RBF-SVM with MIFS as feature selection reached sensitivity of 83 %, specificity of 67 % and an AUC of 0.82.

CONCLUSION

 M- and N-stage can be predicted based on isolated radiomics analyses of the primary tumor in cervical cancers, thus serving as a template for noninvasive tumor phenotyping and patient stratification using high-dimensional feature vectors extracted from multiparametric PET/MRI data.

KEY POINTS

  · Radiomics analysis based on multiparametric PET/MRI enables prediction of the metastatic status of cervical cancers. · Prediction of M-stage is superior to N-stage. · Multiparametric PET/MRI displays a valuable platform for radiomics analyses .

AI as a New Frontier in Contrast Media Research: Bridging the Gap Between Contrast Media Reduction, the Contrast-Free Question and New Application Discoveries

ABSTRACT

Artificial intelligence (AI) techniques are currently harnessed to revolutionize the domain of medical imaging. This review investigates 3 major AI-driven approaches for contrast agent management: new frontiers in contrast agent dose reduction, the contrast-free question, and new applications. By examining recent studies that use AI as a new frontier in contrast media research, we synthesize the current state of the field and provide a comprehensive understanding of the potential and limitations of AI in this context. In doing so, we show the dose limits of reducing the amount of contrast agents and demonstrate why it might not be possible to completely eliminate contrast agents in the future. In addition, we highlight potential new applications to further increase the radiologist's sensitivity at normal doses. At the same time, this review shows which network architectures provide promising approaches and reveals possible artifacts of a paired image-to-image conversion. Furthermore, current US Food and Drug Administration regulatory guidelines regarding AI/machine learning-enabled medical devices are highlighted.

Closing the loop for AI-ready radiology

BACKGROUND

 In recent years, AI has made significant advancements in medical diagnosis and prognosis. However, the incorporation of AI into clinical practice is still challenging and under-appreciated. We aim to demonstrate a possible vertical integration approach to close the loop for AI-ready radiology.

METHOD

 This study highlights the importance of two-way communication for AI-assisted radiology. As a key part of the methodology, it demonstrates the integration of AI systems into clinical practice with structured reports and AI visualization, giving more insight into the AI system. By integrating cooperative lifelong learning into the AI system, we ensure the long-term effectiveness of the AI system, while keeping the radiologist in the loop.  RESULTS:  We demonstrate the use of lifelong learning for AI systems by incorporating AI visualization and structured reports. We evaluate Memory Aware-Synapses and Rehearsal approach and find that both approaches work in practice. Furthermore, we see the advantage of lifelong learning algorithms that do not require the storing or maintaining of samples from previous datasets.

CONCLUSION

 In conclusion, incorporating AI into the clinical routine of radiology requires a two-way communication approach and seamless integration of the AI system, which we achieve with structured reports and visualization of the insight gained by the model. Closing the loop for radiology leads to successful integration, enabling lifelong learning for the AI system, which is crucial for sustainable long-term performance.

KEY POINTS

  · The integration of AI systems into the clinical routine with structured reports and AI visualization.. · Two-way communication between AI and radiologists is necessary to enable AI that keeps the radiologist in the loop.. · Closing the loop enables lifelong learning, which is crucial for long-term, high-performing AI in radiology..

The importance of educational tools and a new software solution for visualizing and quantifying report correction in radiology training

A novel software, DiffTool, was developed in-house to keep track of changes made by board-certified radiologists to preliminary reports created by residents and evaluate its impact on radiological hands-on training. Before (t0) and after (t2-4) the deployment of the software, 18 residents (median age: 29 years; 33% female) completed a standardized questionnaire on professional training. At t2-4 the participants were also requested to respond to three additional questions to evaluate the software. Responses were recorded via a six-point Likert scale ranging from 1 ("strongly agree") to 6 ("strongly disagree"). Prior to the release of the software, 39% (7/18) of the residents strongly agreed with the statement that they manually tracked changes made by board-certified radiologists to each of their radiological reports while 61% were less inclined to agree with that statement. At t2-4, 61% (11/18) stated that they used DiffTool to track differences. Furthermore, we observed an increase from 33% (6/18) to 44% (8/18) of residents who agreed to the statement "I profit from every corrected report". The DiffTool was well accepted among residents with a regular user base of 72% (13/18), while 78% (14/18) considered it a relevant improvement to their training. The results of this study demonstrate the importance of providing a time-efficient way to analyze changes made to preliminary reports as an additive for professional training.

Information extraction from weakly structured radiological reports with natural language queries

OBJECTIVES

Provide physicians and researchers an efficient way to extract information from weakly structured radiology reports with natural language processing (NLP) machine learning models.

METHODS

We evaluate seven different German bidirectional encoder representations from transformers (BERT) models on a dataset of 857,783 unlabeled radiology reports and an annotated reading comprehension dataset in the format of SQuAD 2.0 based on 1223 additional reports.

RESULTS

Continued pre-training of a BERT model on the radiology dataset and a medical online encyclopedia resulted in the most accurate model with an F1-score of 83.97% and an exact match score of 71.63% for answerable questions and 96.01% accuracy in detecting unanswerable questions. Fine-tuning a non-medical model without further pre-training led to the lowest-performing model. The final model proved stable against variation in the formulations of questions and in dealing with questions on topics excluded from the training set.

CONCLUSIONS

General domain BERT models further pre-trained on radiological data achieve high accuracy in answering questions on radiology reports. We propose to integrate our approach into the workflow of medical practitioners and researchers to extract information from radiology reports.

CLINICAL RELEVANCE STATEMENT

By reducing the need for manual searches of radiology reports, radiologists' resources are freed up, which indirectly benefits patients.

KEY POINTS

• BERT models pre-trained on general domain datasets and radiology reports achieve high accuracy (83.97% F1-score) on question-answering for radiology reports. • The best performing model achieves an F1-score of 83.97% for answerable questions and 96.01% accuracy for questions without an answer. • Additional radiology-specific pretraining of all investigated BERT models improves their performance.

k-strip: A novel segmentation algorithm in k-space for the application of skull stripping

BACKGROUND AND OBJECTIVE

We present a novel deep learning-based skull stripping algorithm for magnetic resonance imaging (MRI) that works directly in the information rich complex valued k-space.

METHODS

Using four datasets from different institutions with a total of around 200,000 MRI slices, we show that our network can perform skull-stripping on the raw data of MRIs while preserving the phase information which no other skull stripping algorithm is able to work with. For two of the datasets, skull stripping performed by HD-BET (Brain Extraction Tool) in the image domain is used as the ground truth, whereas the third and fourth dataset comes with per-hand annotated brain segmentations.

RESULTS

All four datasets were very similar to the ground truth (DICE scores of 92 %-99 % and Hausdorff distances of under 5.5 pixel). Results on slices above the eye-region reach DICE scores of up to 99 %, whereas the accuracy drops in regions around the eyes and below, with partially blurred output. The output of k-Strip often has smoothed edges at the demarcation to the skull. Binary masks are created with an appropriate threshold.

CONCLUSION

With this proof-of-concept study, we were able to show the feasibility of working in the k-space frequency domain, preserving phase information, with consistent results. Besides preserving valuable information for further diagnostics, this approach makes an immediate anonymization of patient data possible, already before being transformed into the image domain. Future research should be dedicated to discovering additional ways the k-space can be used for innovative image analysis and further workflows.

Prognostic value of deep learning-derived body composition in advanced pancreatic cancer-a retrospective multicenter study

BACKGROUND

Despite the prognostic relevance of cachexia in pancreatic cancer, individual body composition has not been routinely integrated into treatment planning. In this multicenter study, we investigated the prognostic value of sarcopenia and myosteatosis automatically extracted from routine computed tomography (CT) scans of patients with advanced pancreatic ductal adenocarcinoma (PDAC).

PATIENTS AND METHODS

We retrospectively analyzed clinical imaging data of 601 patients from three German cancer centers. We applied a deep learning approach to assess sarcopenia by the abdominal muscle-to-bone ratio (MBR) and myosteatosis by the ratio of abdominal inter- and intramuscular fat to muscle volume. In the pooled cohort, univariable and multivariable analyses were carried out to analyze the association between body composition markers and overall survival (OS). We analyzed the relationship between body composition markers and laboratory values during the first year of therapy in a subgroup using linear regression analysis adjusted for age, sex, and American Joint Committee on Cancer (AJCC) stage.

RESULTS

Deep learning-derived MBR [hazard ratio (HR) 0.60, 95% confidence interval (CI) 0.47-0.77, P < 0.005] and myosteatosis (HR 3.73, 95% CI 1.66-8.39, P < 0.005) were significantly associated with OS in univariable analysis. In multivariable analysis, MBR (P = 0.019) and myosteatosis (P = 0.02) were associated with OS independent of age, sex, and AJCC stage. In a subgroup, MBR and myosteatosis were associated with albumin and C-reactive protein levels after initiation of therapy. Additionally, MBR was also associated with hemoglobin and total protein levels.

CONCLUSIONS

Our work demonstrates that deep learning can be applied across cancer centers to automatically assess sarcopenia and myosteatosis from routine CT scans. We highlight the prognostic role of our proposed markers and show a strong relationship with protein levels, inflammation, and anemia. In clinical practice, automated body composition analysis holds the potential to further personalize cancer treatment.

Fully automated MR-based virtual biopsy of primary CNS lymphomas

Background

Primary central nervous system lymphomas (PCNSL) pose a challenge as they may mimic gliomas on magnetic resonance imaging (MRI) imaging, compelling precise differentiation for appropriate treatment. This study focuses on developing an automated MRI-based workflow to distinguish between PCNSL and gliomas.

Methods

MRI examinations of 240 therapy-naive patients (141 males and 99 females, mean age: 55.16 years) with cerebral gliomas and PCNSLs (216 gliomas and 24 PCNSLs), each comprising a non-contrast T1-weighted, fluid-attenuated inversion recovery (FLAIR), and contrast-enhanced T1-weighted sequence were included in the study. HD-GLIO, a pre-trained segmentation network, was used to generate segmentations automatically. To validate the segmentation efficiency, 237 manual segmentations were prepared (213 gliomas and 24 PCNSLs). Subsequently, radiomics features were extracted following feature selection and training of an XGBoost algorithm for classification.

Results

The segmentation models for gliomas and PCNSLs achieved a mean Sørensen-Dice coefficient of 0.82 and 0.80 for whole tumors, respectively. Three classification models were developed in this study to differentiate gliomas from PCNSLs. The first model differentiated PCNSLs from gliomas, with an area under the curve (AUC) of 0.99 (F1-score: 0.75). The second model discriminated between high-grade gliomas and PCNSLs with an AUC of 0.91 (F1-score: 0.6), and the third model differentiated between low-grade gliomas and PCNSLs with an AUC of 0.95 (F1-score: 0.89).

Conclusions

This study serves as a pilot investigation presenting an automated virtual biopsy workflow that distinguishes PCNSLs from cerebral gliomas. Prior to clinical use, it is necessary to validate the results in a prospective multicenter setting with a larger number of PCNSL patients.

Optimizing platelet transfusion through a personalized deep learning risk assessment system for demand management

ABSTRACT

Platelet demand management (PDM) is a resource-consuming task for physicians and transfusion managers of large hospitals. Inpatient numbers and institutional standards play significant roles in PDM. However, reliance on these factors alone commonly results in platelet shortages. Using data from multiple sources, we developed, validated, tested, and implemented a patient-specific approach to support PDM that uses a deep learning-based risk score to forecast platelet transfusions for each hospitalized patient in the next 24 hours. The models were developed using retrospective electronic health record data of 34 809 patients treated between 2017 and 2022. Static and time-dependent features included demographics, diagnoses, procedures, blood counts, past transfusions, hematotoxic medications, and hospitalization duration. Using an expanding window approach, we created a training and live-prediction pipeline with a 30-day input and 24-hour forecast. Hyperparameter tuning determined the best validation area under the precision-recall curve (AUC-PR) score for long short-term memory deep learning models, which were then tested on independent data sets from the same hospital. The model tailored for hematology and oncology patients exhibited the best performance (AUC-PR, 0.84; area under the receiver operating characteristic curve [ROC-AUC], 0.98), followed by a multispecialty model covering all other patients (AUC-PR, 0.73). The model specific to cardiothoracic surgery had the lowest performance (AUC-PR, 0.42), likely because of unexpected intrasurgery bleedings. To our knowledge, this is the first deep learning-based platelet transfusion predictor enabling individualized 24-hour risk assessments at high AUC-PR. Implemented as a decision-support system, deep-learning forecasts might improve patient care by detecting platelet demand earlier and preventing critical transfusion shortages.

A Systematic Comparison of Task Adaptation Techniques for Digital Histopathology

Due to an insufficient amount of image annotation, artificial intelligence in computational histopathology usually relies on fine-tuning pre-trained neural networks. While vanilla fine-tuning has shown to be effective, research on computer vision has recently proposed improved algorithms, promising better accuracy. While initial studies have demonstrated the benefits of these algorithms for medical AI, in particular for radiology, there is no empirical evidence for improved accuracy in histopathology. Therefore, based on the ConvNeXt architecture, our study performs a systematic comparison of nine task adaptation techniques, namely, DELTA, L2-SP, MARS-PGM, Bi-Tuning, BSS, MultiTune, SpotTune, Co-Tuning, and vanilla fine-tuning, on five histopathological classification tasks using eight datasets. The results are based on external testing and statistical validation and reveal a multifaceted picture: some techniques are better suited for histopathology than others, but depending on the classification task, a significant relative improvement in accuracy was observed for five advanced task adaptation techniques over the control method, i.e., vanilla fine-tuning (e.g., Co-Tuning: P(≫) = 0.942, d = 2.623). Furthermore, we studied the classification accuracy for three of the nine methods with respect to the training set size (e.g., Co-Tuning: P(≫) = 0.951, γ = 0.748). Overall, our results show that the performance of advanced task adaptation techniques in histopathology is affected by influencing factors such as the specific classification task or the size of the training dataset.

Falling off the screening grid-Predictors for postponed utilization of psycho-oncological support in cancer patients and its implications for distress assessment and management

OBJECTIVE

Distress assessment of cancer patients is considered state-of-the-art. In addition to distress scores, individual care needs are an important factor for the initiation of psycho-oncological interventions. In a mono-centric, observational study, we aimed for characterization of patients indicating a subjective need but declining to utilize support services immediately to facilitate implementation of adapted screenings.

METHODS

This study analyzed retrospective data from routine distress screening and associated data from hospital records. Descriptive, variance and regression analyses were used to assess characteristics of postponed support utilization in patients with mixed cancer diagnoses in different treatment settings.

RESULTS

Of the total sample (N = 1863), 13% indicated a subjective need but postponed support utilization. This subgroup presented as being as burdened by symptoms of depression (p < 0.001), anxiety (p < 0.001) and distress (p < 0.001) as subjectively distressed patients with intent to directly utilize support. Time periods since diagnosis were shorter (p = 0.007) and patients were more often inpatients (p = 0.045).

CONCLUSIONS

Despite high heterogeneity among the subgroups, this study identified distress-related factors and time since diagnosis as possible predictors for postponed utilization of psycho-oncological interventions. Results suggest the necessity for time-individualized support which may improve utilization by distressed patients.

Preoperative Computed Tomography Volumetry and Graft Weight Estimation of Left Lateral Segment in Pediatric Living Donor Liver Transplant

OBJECTIVES

Liver volumetry based on a computed tomography scan is widely used to estimate liver volume before any liver resection, especially before living donorliver donation. The 1-to-1 conversion rule for liver volume to liver weight has been widely adopted; however, debate continues regarding this approach. Therefore, we analyzed the relationship between the left-lateral lobe liver graft volume and actual graft weight.

MATERIALS AND METHODS

This study retrospectively included consecutive donors who underwent left lateral hepatectomy for pediatric living donor liver transplant from December 2008 to September 2020. All donors were healthy adults who met the evaluation criteria for pediatric living donor liver transplant and underwent a preoperative contrast-enhanced computed tomography scan. Manual segmentation of the leftlateral liverlobe for graft volume estimation and intraoperative measurement of an actual graft weight were performed. The relationship between estimated graft volume and actual graft weight was analyzed.

RESULTS

Ninety-four living liver donors were included in the study. The mean actual graft weight was ~283.4 ± 68.5 g, and the mean graft volume was 244.9 ± 63.86 mL. A strong correlation was shown between graft volume and actual graft weight (r = 0.804; P < .001). Bland-Altman analysis revealed an interobserver agreement of 38.0 ± 97.25, and intraclass correlation coefficient showed almost perfect agreement(r = 0.840; P < .001). The conversion formula for calculating graft weight based on computed tomography volumetry was determined based on regression analysis: 0.88 × graft volume + 41.63.

CONCLUSIONS

The estimation of left liver graft weight using only the 1-to-1 rule is subject to measurable variability in calculated graft weights and tends to underestimate the true graft weight. Instead, a different, improved conversion formula should be used to calculate graft weight to more accurately determine donor graft weight-to-recipient body weightratio and reduce the risk of underestimation of liver graft weightin the donor selection process before pediatric living donor liver transplant.

Deep learning in computational dermatopathology of melanoma: A technical systematic literature review

Deep learning (DL) has become one of the major approaches in computational dermatopathology, evidenced by a significant increase in this topic in the current literature. We aim to provide a structured and comprehensive overview of peer-reviewed publications on DL applied to dermatopathology focused on melanoma. In comparison to well-published DL methods on non-medical images (e.g., classification on ImageNet), this field of application comprises a specific set of challenges, such as staining artifacts, large gigapixel images, and various magnification levels. Thus, we are particularly interested in the pathology-specific technical state-of-the-art. We also aim to summarize the best performances achieved thus far with respect to accuracy, along with an overview of self-reported limitations. Accordingly, we conducted a systematic literature review of peer-reviewed journal and conference articles published between 2012 and 2022 in the databases ACM Digital Library, Embase, IEEE Xplore, PubMed, and Scopus, expanded by forward and backward searches to identify 495 potentially eligible studies. After screening for relevance and quality, a total of 54 studies were included. We qualitatively summarized and analyzed these studies from technical, problem-oriented, and task-oriented perspectives. Our findings suggest that the technical aspects of DL for histopathology in melanoma can be further improved. The DL methodology was adopted later in this field, and still lacks the wider adoption of DL methods already shown to be effective for other applications. We also discuss upcoming trends toward ImageNet-based feature extraction and larger models. While DL has achieved human-competitive accuracy in routine pathological tasks, its performance on advanced tasks is still inferior to wet-lab testing (for example). Finally, we discuss the challenges impeding the translation of DL methods to clinical practice and provide insight into future research directions.

Identification of radiomic biomarkers in a set of four skeletal muscle groups on Dixon MRI of the NAKO MR study

In this work, we propose a processing pipeline for the extraction and identification of meaningful radiomics biomarkers in skeletal muscle tissue as displayed using Dixon-weighted MRI. Diverse and robust radiomics features can be identified that may be of aid in the accurate quantification e.g. varying degrees of sarcopenia in respective muscles of large cohorts. As such, the approach comprises the texture feature extraction from raw data based on well established approaches, such as a nnU-Net neural network and the Pyradiomics toolbox, a subsequent selection according to adequate conditions for the muscle tissue of the general population, and an importance-based ranking to further narrow the amount of meaningful features with respect to auxiliary targets. The performance was investigated with respect to the included auxiliary targets, namely age, body mass index (BMI), and fat fraction (FF). Four skeletal muscles with different fiber architecture were included: the mm. glutaei, m. psoas, as well as the extensors and adductors of the thigh. The selection allowed for a reduction from 1015 available texture features to 65 for age, 53 for BMI, and 36 for FF from the available fat/water contrast images considering all muscles jointly. Further, the dependence of the importance rankings calculated for the auxiliary targets on validation sets (in a cross-validation scheme) was investigated by boxplots. In addition, significant differences between subgroups of respective auxiliary targets as well as between both sexes were shown to be present within the ten lowest ranked features by means of Kruskal-Wallis H-tests and Mann-Whitney U-tests. The prediction performance for the selected features and the ranking scheme were verified on validation sets by a random forest based multi-class classification, with strong area under the curve (AUC) values of the receiver operator characteristic (ROC) of 73.03 ± 0.70 % and 73.63 ± 0.70 % for the water and fat images in age, 80.68 ± 0.30 % and 88.03 ± 0.89 % in BMI, as well as 98.36 ± 0.03 % and 98.52 ± 0.09 % in FF.

Predicting Individual Patient Platelet Demand in a Large Tertiary Care Hospital Using Machine Learning

Introduction

An increasing shortage of donor blood is expected, considering the demographic change in Germany. Due to the short shelf life and varying daily fluctuations in consumption, the storage of platelet concentrates (PCs) becomes challenging. This emphasizes the need for reliable prediction of needed PCs for the blood bank inventories. Therefore, the objective of this study was to evaluate multimodal data from multiple source systems within a hospital to predict the number of platelet transfusions in 3 days on a per-patient level.

Methods

Data were collected from 25,190 (42% female and 58% male) patients between 2017 and 2021. For each patient, the number of received PCs, platelet count blood tests, drugs causing thrombocytopenia, acute platelet diseases, procedures, age, gender, and the period of a patient's hospital stay were collected. Two models were trained on samples using a sliding window of 7 days as input and a day 3 target. The model predicts whether a patient will be transfused 3 days in the future. The model was trained with an excessive hyperparameter search using patient-level repeated 5-fold cross-validation to optimize the average macro F2-score.

Results

The trained models were tested on 5,022 unique patients. The best-performing model has a specificity of 0.99, a sensitivity of 0.37, an area under the precision-recall curve score of 0.45, an MCC score of 0.43, and an F1-score of 0.43. However, the model does not generalize well for cases when the need for a platelet transfusion is recognized.

Conclusion

A patient AI-based platelet forecast could improve logistics management and reduce blood product waste. In this study, we build the first model to predict patient individual platelet demand. To the best of our knowledge, we are the first to introduce this approach. Our model predicts the need for platelet units for 3 days in the future. While sensitivity underperforms, specificity performs reliably. The model may be of clinical use as a pretest for potential patients needing a platelet transfusion within the next 3 days. As sensitivity needs to be improved, further studies should introduce deep learning and wider patient characterization to the methodological multimodal, multisource data approach. Furthermore, a hospital-wide consumption of PCs could be derived from individual predictions.

FHIR-PYrate: a data science friendly Python package to query FHIR servers

BACKGROUND

We present FHIR-PYrate, a Python package to handle the full clinical data collection and extraction process. The software is to be plugged into a modern hospital domain, where electronic patient records are used to handle the entire patient's history. Most research institutes follow the same procedures to build study cohorts, but mainly in a non-standardized and repetitive way. As a result, researchers spend time writing boilerplate code, which could be used for more challenging tasks.

METHODS

The package can improve and simplify existing processes in the clinical research environment. It collects all needed functionalities into a straightforward interface that can be used to query a FHIR server, download imaging studies and filter clinical documents. The full capacity of the search mechanism of the FHIR REST API is available to the user, leading to a uniform querying process for all resources, thus simplifying the customization of each use case. Additionally, valuable features like parallelization and filtering are included to make it more performant.

RESULTS

As an exemplary practical application, the package can be used to analyze the prognostic significance of routine CT imaging and clinical data in breast cancer with tumor metastases in the lungs. In this example, the initial patient cohort is first collected using ICD-10 codes. For these patients, the survival information is also gathered. Some additional clinical data is retrieved, and CT scans of the thorax are downloaded. Finally, the survival analysis can be computed using a deep learning model with the CT scans, the TNM staging and positivity of relevant markers as input. This process may vary depending on the FHIR server and available clinical data, and can be customized to cover even more use cases.

CONCLUSIONS

FHIR-PYrate opens up the possibility to quickly and easily retrieve FHIR data, download image data, and search medical documents for keywords within a Python package. With the demonstrated functionality, FHIR-PYrate opens an easy way to assemble research collectives automatically.

Contrast Agent Dose Reduction in MRI Utilizing a Generative Adversarial Network in an Exploratory Animal Study

OBJECTIVES

The aim of this study is to use virtual contrast enhancement to reduce the amount of hepatobiliary gadolinium-based contrast agent in magnetic resonance imaging with generative adversarial networks (GANs) in a large animal model.

METHODS

With 20 healthy Göttingen minipigs, a total of 120 magnetic resonance imaging examinations were performed on 6 different occasions, 50% with reduced (low-dose; 0.005 mmol/kg, gadoxetate) and 50% standard dose (normal-dose; 0.025 mmol/kg). These included arterial, portal venous, venous, and hepatobiliary contrast phases (20 minutes, 30 minutes). Because of incomplete examinations, one animal had to be excluded. Randomly, 3 of 19 animals were selected and withheld for validation (18 examinations). Subsequently, a GAN was trained for image-to-image conversion from low-dose to normal-dose (virtual normal-dose) with the remaining 16 animals (96 examinations). For validation, vascular and parenchymal contrast-to-noise ratio (CNR) was calculated using region of interest measurements of the abdominal aorta, inferior vena cava, portal vein, hepatic parenchyma, and autochthonous back muscles. In parallel, a visual Turing test was performed by presenting the normal-dose and virtual normal-dose data to 3 consultant radiologists, blinded for the type of examination. They had to decide whether they would consider both data sets as consistent in findings and which images were from the normal-dose study.

RESULTS

The pooled dynamic phase vascular and parenchymal CNR increased significantly from low-dose to virtual normal-dose (pooled vascular: P < 0.0001, pooled parenchymal: P = 0.0002) and was found to be not significantly different between virtual normal-dose and normal-dose examinations (vascular CNR [mean ± SD]: low-dose 17.6 ± 6.0, virtual normal-dose 41.8 ± 9.7, and normal-dose 48.4 ± 12.2; parenchymal CNR [mean ± SD]: low-dose 20.2 ± 5.9, virtual normal-dose 28.3 ± 6.9, and normal-dose 29.5 ± 7.2). The pooled parenchymal CNR of the hepatobiliary contrast phases revealed a significant increase from the low-dose (22.8 ± 6.2) to the virtual normal-dose (33.2 ± 6.1; P < 0.0001) and normal-dose sequence (37.0 ± 9.1; P < 0.0001). In addition, there was no significant difference between the virtual normal-dose and normal-dose sequence. In the visual Turing test, on the median, the consultant radiologist reported that the sequences of the normal-dose and virtual normal-dose are consistent in findings in 100% of the examinations. Moreover, the consultants were able to identify the normal-dose series as such in a median 54.5% of the cases.

CONCLUSIONS

In this feasibility study in healthy Göttingen minipigs, it could be shown that GAN-based virtual contrast enhancement can be used to recreate the image impression of normal-dose imaging in terms of CNR and subjective image similarity in both dynamic and hepatobiliary contrast phases from low-dose data with an 80% reduction in gadolinium-based contrast agent dose. Before clinical implementation, further studies with pathologies are needed to validate whether pathologies are correctly represented by the network.

Explainable AI in medical imaging: An overview for clinical practitioners – Saliency-based XAI approaches

Since recent achievements of Artificial Intelligence (AI) have proven significant success and promising results throughout many fields of application during the last decade, AI has also become an essential part of medical research. The improving data availability, coupled with advances in high-performance computing and innovative algorithms, has increased AI's potential in various aspects. Because AI rapidly reshapes research and promotes the development of personalized clinical care, alongside its implementation arises an urgent need for a deep understanding of its inner workings, especially in high-stake domains. However, such systems can be highly complex and opaque, limiting the possibility of an immediate understanding of the system's decisions. Regarding the medical field, a high impact is attributed to these decisions as physicians and patients can only fully trust AI systems when reasonably communicating the origin of their results, simultaneously enabling the identification of errors and biases. Explainable AI (XAI), becoming an increasingly important field of research in recent years, promotes the formulation of explainability methods and provides a rationale allowing users to comprehend the results generated by AI systems. In this paper, we investigate the application of XAI in medical imaging, addressing a broad audience, especially healthcare professionals. The content focuses on definitions and taxonomies, standard methods and approaches, advantages, limitations, and examples representing the current state of research regarding XAI in medical imaging. This paper focuses on saliency-based XAI methods, where the explanation can be provided directly on the input data (image) and which naturally are of special importance in medical imaging.

Explainable AI in medical imaging: An overview for clinical practitioners – Beyond saliency-based XAI approaches

Driven by recent advances in Artificial Intelligence (AI) and Computer Vision (CV), the implementation of AI systems in the medical domain increased correspondingly. This is especially true for the domain of medical imaging, in which the incorporation of AI aids several imaging-based tasks such as classification, segmentation, and registration. Moreover, AI reshapes medical research and contributes to the development of personalized clinical care. Consequently, alongside its extended implementation arises the need for an extensive understanding of AI systems and their inner workings, potentials, and limitations which the field of eXplainable AI (XAI) aims at. Because medical imaging is mainly associated with visual tasks, most explainability approaches incorporate saliency-based XAI methods. In contrast to that, in this article we would like to investigate the full potential of XAI methods in the field of medical imaging by specifically focusing on XAI techniques not relying on saliency, and providing diversified examples. We dedicate our investigation to a broad audience, but particularly healthcare professionals. Moreover, this work aims at establishing a common ground for cross-disciplinary understanding and exchange across disciplines between Deep Learning (DL) builders and healthcare professionals, which is why we aimed for a non-technical overview. Presented XAI methods are divided by a method's output representation into the following categories: Case-based explanations, textual explanations, and auxiliary explanations.

Individualized scan protocols for CT angiography: an animal study for contrast media or radiation dose optimization

BACKGROUND

We investigated about optimization of contrast media (CM) dose or radiation dose in thoracoabdominal computed tomography angiography (CTA) by automated tube voltage selection (ATVS) system configuration and CM protocol adaption.

METHODS

In six minipigs, CTA-optimized protocols were evaluated regarding objective (contrast-to-noise ratio, CNR) and subjective (6 criteria assessed by Likert scale) image quality. Scan parameters were automatically adapted by the ATVS system operating at 90-kV semi-mode and configured for standard, CM saving, or radiation dose saving (image task, quality settings). Injection protocols (dose, flow rate) were adapted manually. This approach was tested for normal and simulated obese conditions.

RESULTS

Radiation exposure (volume-weighted CT dose index) for normal (obese) conditions was 2.4 ± 0.7 (5.0 ± 0.7) mGy (standard), 4.3 ± 1.1 (9.0 ± 1.3) mGy (CM reduced), and 1.7 ± 0.5 (3.5 ± 0.5) mGy (radiation reduced). The respective CM doses for normal (obese) settings were 210 (240) mgI/kg, 155 (177) mgI/kg, and 252 (288) mgI/kg. No significant differences in CNR (normal; obese) were observed between standard (17.8 ± 3.0; 19.2 ± 4.0), CM-reduced (18.2 ± 3.3; 20.5 ± 4.9), and radiation-saving CTAs (16.0 ± 3.4; 18.4 ± 4.1). Subjective analysis showed similar values for optimized and standard CTAs. Only the parameter diagnostic acceptability was significantly lower for radiation-saving CTA compared to the standard CTA.

CONCLUSIONS

The CM dose (-26%) or radiation dose (-30%) for thoracoabdominal CTA can be reduced while maintaining objective and subjective image quality, demonstrating the feasibility of the personalization of CTA scan protocols.

KEY POINTS

• Computed tomography angiography protocols could be adapted to individual patient requirements using an automated tube voltage selection system combined with adjusted contrast media injection. • Using an adapted automated tube voltage selection system, a contrast media dose reduction (-26%) or radiation dose reduction (-30%) could be possible.

Adipopenia in body composition analysis: a promising imaging biomarker and potential predictive factor for patients undergoing transjugular intrahepatic portosystemic shunt placement

OBJECTIVE

Body tissue composition plays a crucial role in the multisystemic processes of advanced liver disease and has been shown to be influenced by transjugular intrahepatic portosystemic shunt (TIPS). A differentiated analysis of the various tissue compartments has not been performed until now. The purpose of this study was to evaluate the value of imaging biomarkers derived from automated body composition analysis (BCA) to predict clinical and functional outcome.

METHODS

A retrospective analysis of 56 patients undergoing TIPS procedure between 2013 and 2021 was performed. BCA on the base of pre-interventional CT examination was used to determine quantitative data as well as ratios of bone, muscle and fat masses. Furthermore, a BCA-derived sarcopenia marker was investigated. Regarding potential correlations between BCA imaging biomarkers and the occurrence of hepatic encephalopathy (HE) as well as 1-year survival, an exploratory analysis was conducted.

RESULTS

No BCA imaging biomarker was associated with the occurrence of HE after TIPS placement. However, there were significant differences in alive and deceased patients regarding the BCA-derived sarcopenia marker (alive: 1.60, deceased: 1.83, p = 0.046), ratios of intra- and intermuscular fat/skeletal volume (alive: 0.53, deceased: 0.31, p = 0.015) and intra- and intermuscular fat/muscle volume (alive: 0.21, deceased: 0.14, p = 0.031).

CONCLUSION

A lower amount of intra- and intermuscular adipose tissue might have protective effects regarding liver derived complications and survival.

ADVANCES IN KNOWLEDGE

Precise characterization of body tissue components with automated BCA might provide prognostic information in patients with advanced liver disease undergoing TIPS procedure.

AI co-pilot: content-based image retrieval for the reading of rare diseases in chest CT

The aim of the study was to evaluate the impact of the newly developed Similar patient search (SPS) Web Service, which supports reading complex lung diseases in computed tomography (CT), on the diagnostic accuracy of residents. SPS is an image-based search engine for pre-diagnosed cases along with related clinical reference content ( https://eref.thieme.de ). The reference database was constructed using 13,658 annotated regions of interest (ROIs) from 621 patients, comprising 69 lung diseases. For validation, 50 CT scans were evaluated by five radiology residents without SPS, and three months later with SPS. The residents could give a maximum of three diagnoses per case. A maximum of 3 points was achieved if the correct diagnosis without any additional diagnoses was provided. The residents achieved an average score of 17.6 ± 5.0 points without SPS. By using SPS, the residents increased their score by 81.8% to 32.0 ± 9.5 points. The improvement of the score per case was highly significant (p = 0.0001). The residents required an average of 205.9 ± 350.6 s per case (21.9% increase) when SPS was used. However, in the second half of the cases, after the residents became more familiar with SPS, this increase dropped to 7%. Residents' average score in reading complex chest CT scans improved by 81.8% when the AI-driven SPS with integrated clinical reference content was used. The increase in time per case due to the use of the SPS was minimal.

Deep learning-based assessment of body composition and liver tumour burden for survival modelling in advanced colorectal cancer

BACKGROUND

Personalized therapy planning remains a significant challenge in advanced colorectal cancer care, despite extensive research on prognostic and predictive markers. A strong correlation of sarcopenia or overall body composition and survival has been described. Here, we explore whether automated assessment of body composition and liver metastases from standard of care CT images can add to clinical parameters in personalized survival risk prognostication.

METHODS

We retrospectively analysed clinical imaging data from 85 patients (50.6% female, mean age 58.9 SD 12.2 years) with colorectal cancer and synchronous liver metastases. Pretrained deep learning models were used to assess body composition and liver metastasis geometry from abdominal CT images before the initiation of systemic treatment. Abdominal muscle-to-bone ratio (MBR) was calculated by dividing abdominal muscle volume by abdominal bone volume. MBR was compared with body mass index (BMI), abdominal muscle volume, and abdominal muscle volume divided by height squared. Differences in overall survival based on body composition and liver metastasis parameters were compared using Kaplan-Meier survival curves. Results were correlated with clinical and biomarker data to develop a machine learning model for survival risk prognostication.

RESULTS

The MBR, unlike abdominal muscle volume or BMI, was significantly associated with overall survival (HR 0.39, 95% CI: 0.19-0.80, P = 0.009). The MBR (P = 0.022), liver metastasis surface area (P = 0.01) and primary tumour sidedness (P = 0.007) were independently associated with overall survival in multivariate analysis. Body composition parameters did not correlate with KRAS mutational status or primary tumour sidedness. A prediction model based on MBR, liver metastasis surface area and primary tumour sidedness achieved a concordance index of 0.69.

CONCLUSIONS

Automated segmentation enables to extract prognostic parameters from routine imaging data for personalized survival modelling in advanced colorectal cancer patients.

Preoperative PET-SUVmax and volume based PET parameters of the primary tumor fail to predict nodal upstaging in early-stage lung cancer

OBJECTIVES

Accurate nodal staging is of utmost importance in patients with lung cancer. FDG-PET/CT imaging is now part of the routine staging. Despite thorough preoperative staging nodal upstaging still occurs in early-stage lung cancer. However, the predictive value of preoperative PET metrics of the primary tumor on nodal upstaging remains to be unexplored. Our aim was to assess the association of these preoperative PET-parameters with nodal upstaging in histologically confirmed lung adenocarcinoma and squamous cell carcinoma.

METHODS

From January 2016 to November 2018, 500 patients with pT1-T2/cN0 lung cancer received an anatomical resection with curative intent. 171 patients with adenocarcinoma and squamous cell carcinoma and available PET-CTs were retrospectively included. We analyzed the the association of nodal upstaging with preoperative PET-SUVmax and metabolic PET metrics including total lesion glycolysis (TLG) and metabolic tumor volume (MTV) with different defined thresholds.

RESULTS

High values of preoperative PET-SUVmax of the primary tumor were associated with squamous cell carcinoma (p < 0.0001) and with larger tumors (p < 0.0001). Increased preoperative C-reactive protein levels (<1mg/dL) correlated significantly with high preoperative PET-SUVmax values (p < 0.0001). No significant relationship between PET-SUVmax and lactate dehydrogenase activity (p = 0.6818), white blood cell count (p = 0.7681), gender (p = 0.1115) or age (p = 0.9284) was observed. Nodal upstaging rate was 14.0 % with 8.8 % N1 and 5.3 % N2 upstaging. Tumor size (p = 0.0468) and number of removed lymph nodes (p = 0.0461) were significant predictors of nodal upstaging but no significant association was found with histology or PET parameters. Of note, increased MTV - regardless of the threshold - tended to associate with nodal upstaging.

CONCLUSION

Early-stage lung cancer patients with squamous histology and T2 tumors presented increased preoperative PET-SUVmax values. Nevertheless, beyond tumor size and number of removed lymph nodes neither SUVmax nor metabolic PET parameters MTV and TLG were significant predictors of nodal upstaging.

Fully automated preoperative liver volumetry incorporating the anatomical location of the central hepatic vein

The precise preoperative calculation of functional liver volumes is essential prior major liver resections, as well as for the evaluation of a suitable donor for living donor liver transplantation. The aim of this study was to develop a fully automated, reproducible, and quantitative 3D volumetry of the liver from standard CT examinations of the abdomen as part of routine clinical imaging. Therefore, an in-house dataset of 100 venous phase CT examinations for training and 30 venous phase ex-house CT examinations with a slice thickness of 5 mm for testing and validating were fully annotated with right and left liver lobe. Multi-Resolution U-Net 3D neural networks were employed for segmenting these liver regions. The Sørensen-Dice coefficient was greater than 0.9726 ± 0.0058, 0.9639 ± 0.0088, and 0.9223 ± 0.0187 and a mean volume difference of 32.12 ± 19.40 ml, 22.68 ± 21.67 ml, and 9.44 ± 27.08 ml compared to the standard of reference (SoR) liver, right lobe, and left lobe annotation was achieved. Our results show that fully automated 3D volumetry of the liver on routine CT imaging can provide reproducible, quantitative, fast and accurate results without needing any examiner in the preoperative work-up for hepatobiliary surgery and especially for living donor liver transplantation.

Contrast Media Reduction in Computed Tomography With Deep Learning Using a Generative Adversarial Network in an Experimental Animal Study

OBJECTIVE

This feasibility study aimed to use optimized virtual contrast enhancement through generative adversarial networks (GAN) to reduce the dose of iodine-based contrast medium (CM) during abdominal computed tomography (CT) in a large animal model.

METHODS

Multiphasic abdominal low-kilovolt CTs (90 kV) with low (low CM, 105 mgl/kg) and normal contrast media doses (normal CM, 350 mgl/kg) were performed with 20 healthy Göttingen minipigs on 3 separate occasions for a total of 120 examinations. These included an early arterial, late arterial, portal venous, and venous contrast phase. One animal had to be excluded because of incomplete examinations. Three of the 19 animals were randomly selected and withheld for validation (18 studies). Subsequently, the GAN was trained for image-to-image conversion from low CM to normal CM (virtual CM) with the remaining 16 animals (96 examinations). For validation, region of interest measurements were performed in the abdominal aorta, inferior vena cava, portal vein, liver parenchyma, and autochthonous back muscles, and the contrast-to-noise ratio (CNR) was calculated. In addition, the normal CM and virtual CM data were presented in a visual Turing test to 3 radiology consultants. On the one hand, they had to decide which images were derived from the normal CM examination. On the other hand, they had to evaluate whether both images are pathological consistent.

RESULTS

Average vascular CNR (low CM 6.9 ± 7.0 vs virtual CM 28.7 ± 23.8, P < 0.0001) and parenchymal (low CM 1.5 ± 0.7 vs virtual CM 3.8 ± 2.0, P < 0.0001) CNR increased significantly by GAN-based contrast enhancement in all contrast phases and was not significantly different from normal CM examinations (vascular: virtual CM 28.7 ± 23.8 vs normal CM 34.2 ± 28.8; parenchymal: virtual CM 3.8 ± 2.0 vs normal CM 3.7 ± 2.6). During the visual Turing testing, the radiology consultants reported that images from normal CM and virtual CM were pathologically consistent in median in 96.5% of the examinations. Furthermore, it was possible for the examiners to identify the normal CM data as such in median in 91% of the cases.

CONCLUSIONS

In this feasibility study, it could be demonstrated in an experimental setting with healthy Göttingen minipigs that the amount of CM for abdominal CT can be reduced by approximately 70% by GAN-based contrast enhancement with satisfactory image quality.

CT-derived body composition analysis could possibly replace DXA and BIA to monitor NET-patients

Patients with neuroendocrine tumors of gastro-entero-pancreatic origin (GEP-NET) experience changes in fat and muscle composition. Dual-energy X-ray absorptiometry (DXA) and bioelectrical impedance analysis (BIA) are currently used to analyze body composition. Changes thereof could indicate cancer progression or response to treatment. This study examines the correlation between CT-based (computed tomography) body composition analysis (BCA) and DXA or BIA measurement. 74 GEP-NET-patients received whole-body [68Ga]-DOTATOC-PET/CT, BIA, and DXA-scans. BCA was performed based on the non-contrast-enhanced, 5 mm, whole-body-CT images. BCA from CT shows a strong correlation between body fat ratio with DXA (r = 0.95, ρC = 0.83) and BIA (r = 0.92, ρC = 0.76) and between skeletal muscle ratio with BIA: r = 0.81, ρC = 0.49. The deep learning-network achieves highly accurate results (mean Sørensen-Dice-score 0.93). Using BCA on routine Positron emission tomography/CT-scans to monitor patients’ body composition in the diagnostic workflow can reduce additional exams whilst substantially amplifying measurement in slower progressing cancers such as GEP-NET.

Artificial intelligence guided enhancement of digital PET: scans as fast as CT?

Purpose

Both digital positron emission tomography (PET) detector technologies and artificial intelligence based image post-reconstruction methods allow to reduce the PET acquisition time while maintaining diagnostic quality. The aim of this study was to acquire ultra-low-count fluorodeoxyglucose (FDG) ExtremePET images on a digital PET/computed tomography (CT) scanner at an acquisition time comparable to a CT scan and to generate synthetic full-dose PET images using an artificial neural network.

Methods

This is a prospective, single-arm, single-center phase I/II imaging study. A total of 587 patients were included. For each patient, a standard and an ultra-low-count FDG PET/CT scan (whole-body acquisition time about 30 s) were acquired. A modified pix2pixHD deep-learning network was trained employing 387 data sets as training and 200 as test cohort. Three models (PET-only and PET/CT with or without group convolution) were compared. Detectability and quantification were evaluated.

Results

The PET/CT input model with group convolution performed best regarding lesion signal recovery and was selected for detailed evaluation. Synthetic PET images were of high visual image quality; mean absolute lesion SUV_max (maximum standardized uptake value) difference was 1.5. Patient-based sensitivity and specificity for lesion detection were 79% and 100%, respectively. Not-detected lesions were of lower tracer uptake and lesion volume. In a matched-pair comparison, patient-based (lesion-based) detection rate was 89% (78%) for PERCIST (PET response criteria in solid tumors)-measurable and 36% (22%) for non PERCIST-measurable lesions.

Conclusion

Lesion detectability and lesion quantification were promising in the context of extremely fast acquisition times. Possible application scenarios might include re-staging of late-stage cancer patients, in whom assessment of total tumor burden can be of higher relevance than detailed evaluation of small and low-uptake lesions.

Supplementary Information

The online version contains supplementary material available at 10.1007/s00259-022-05901-x.

Validating Automatic Concept-Based Explanations for AI-Based Digital Histopathology

Digital histopathology poses several challenges such as label noise, class imbalance, limited availability of labelled data, and several latent biases to deep learning, negatively influencing transparency, reproducibility, and classification performance. In particular, biases are well known to cause poor generalization. Proposed tools from explainable artificial intelligence (XAI), bias detection, and bias discovery suffer from technical challenges, complexity, unintuitive usage, inherent biases, or a semantic gap. A promising XAI method, not studied in the context of digital histopathology is automated concept-based explanation (ACE). It automatically extracts visual concepts from image data. Our objective is to evaluate ACE's technical validity following design science principals and to compare it to Guided Gradient-weighted Class Activation Mapping (Grad-CAM), a conventional pixel-wise explanation method. To that extent, we created and studied five convolutional neural networks (CNNs) in four different skin cancer settings. Our results demonstrate that ACE is a valid tool for gaining insights into the decision process of histopathological CNNs that can go beyond explanations from the control method. ACE validly visualized a class sampling ratio bias, measurement bias, sampling bias, and class-correlated bias. Furthermore, the complementary use with Guided Grad-CAM offers several benefits. Finally, we propose practical solutions for several technical challenges. In contradiction to results from the literature, we noticed lower intuitiveness in some dermatopathology scenarios as compared to concept-based explanations on real-world images.

Deep Learning-driven classification of external DICOM studies for PACS archiving

OBJECTIVES

Over the course of their treatment, patients often switch hospitals, requiring staff at the new hospital to import external imaging studies to their local database. In this study, the authors present MOdality Mapping and Orchestration (MOMO), a Deep Learning-based approach to automate this mapping process by combining metadata analysis and a neural network ensemble.

METHODS

A set of 11,934 imaging series with existing anatomical labels was retrieved from the PACS database of the local hospital to train an ensemble of neural networks (DenseNet-161 and ResNet-152), which process radiological images and predict the type of study they belong to. We developed an algorithm that automatically extracts relevant metadata from imaging studies, regardless of their structure, and combines it with the neural network ensemble, forming a powerful classifier. A set of 843 anonymized external studies from 321 hospitals was hand-labeled to assess performance. We tested several variations of this algorithm.

RESULTS

MOMO achieves 92.71% accuracy and 2.63% minor errors (at 99.29% predictive power) on the external study classification task, outperforming both a commercial product (82.86% accuracy, 1.36% minor errors, 96.20% predictive power) and a pure neural network ensemble (72.69% accuracy, 10.3% minor errors, 99.05% predictive power) performing the same task. We find that the highest performance is achieved by an algorithm that combines all information into one vote-based classifier.

CONCLUSION

Deep Learning combined with metadata matching is a promising and flexible approach for the automated classification of external DICOM studies for PACS archiving.

KEY POINTS

• The algorithm can successfully identify 76 medical study types across seven modalities (CT, X-ray angiography, radiographs, MRI, PET (+CT/MRI), ultrasound, and mammograms). • The algorithm outperforms a commercial product performing the same task by a significant margin (> 9% accuracy gain). • The performance of the algorithm increases through the application of Deep Learning techniques.

18F-flutemetamol positron emission tomography in cardiac amyloidosis

PURPOSE

Bone-tracer scintigraphy has an established role in diagnosis of cardiac amyloidosis (CA) as it detects transthyretin amyloidosis (ATTR). Positron emission tomography (PET) with amyloid tracers has shown high sensitivity for detection of both ATTR and light-chain (AL) CA. We aimed to investigate the accuracy of 18F-flutemetamol in CA.

METHODS

We enrolled patients with CA or non-amyloid heart failure (NA-HF), who underwent cardiac 18F-flutemetamol PET/MRI or PET/CT. Myocardial and blood pool standardized tracer uptake values (SUV) were estimated. Late gadolinium enhancement (LGE) and T1 mapping/ extracellular volume (ECV) estimation were performed.

RESULTS

We included 17 patients (12 with CA, 5 with NA-HF). PET/MRI was conducted in 13 patients, while PET/CT was conducted in 4. LGE was detected in 8 of 9 CA patients. Global relaxation time and ECV were higher in CA (1448 vs. 1326, P = 0.02 and 58.9 vs. 33.7%, P = 0.006, respectively). Positive PET studies were demonstrated in 2 of 12 patients with CA (AL and ATTR). Maximal and mean SUV did not differ between groups (2.21 vs. 1.69, P = 0.18 and 1.73 vs. 1.30, P = 0.13).

CONCLUSION

Although protein-independent binding is supported by our results, the diagnostic yield of PET was low. We demonstrate here for the first time the low sensitivity of PET for CA.

Fully Automated MR Based Virtual Biopsy of Cerebral Gliomas

OBJECTIVE

The aim of this study was to investigate the diagnostic accuracy of a radiomics analysis based on a fully automated segmentation and a simplified and robust MR imaging protocol to provide a comprehensive analysis of the genetic profile and grading of cerebral gliomas for everyday clinical use.

METHODS

MRI examinations of 217 therapy-naïve patients with cerebral gliomas, each comprising a non-contrast T1-weighted, FLAIR and contrast-enhanced T1-weighted sequence, were included in the study. In addition, clinical and laboratory parameters were incorporated into the analysis. The BraTS 2019 pretrained DeepMedic network was used for automated segmentation. The segmentations generated by DeepMedic were evaluated with 200 manual segmentations with a DICE score of 0.8082 ± 0.1321. Subsequently, the radiomics signatures were utilized to predict the genetic profile of ATRX, IDH1/2, MGMT and 1p19q co-deletion, as well as differentiating low-grade glioma from high-grade glioma.

RESULTS

The network provided an AUC (validation/test) for the differentiation between low-grade gliomas vs. high-grade gliomas of 0.981 ± 0.015/0.885 ± 0.02. The best results were achieved for the prediction of the ATRX expression loss with AUCs of 0.979 ± 0.028/0.923 ± 0.045, followed by 0.929 ± 0.042/0.861 ± 0.023 for the prediction of IDH1/2. The prediction of 1p19q and MGMT achieved moderate results, with AUCs of 0.999 ± 0.005/0.711 ± 0.128 for 1p19q and 0.854 ± 0.046/0.742 ± 0.050 for MGMT.

CONCLUSION

This fully automated approach utilizing simplified MR protocols to predict the genetic profile and grading of cerebral gliomas provides an easy and efficient method for non-invasive tumor decoding.

Joint Imaging Platform for Federated Clinical Data Analytics

PURPOSE

Image analysis is one of the most promising applications of artificial intelligence (AI) in health care, potentially improving prediction, diagnosis, and treatment of diseases. Although scientific advances in this area critically depend on the accessibility of large-volume and high-quality data, sharing data between institutions faces various ethical and legal constraints as well as organizational and technical obstacles.

METHODS

The Joint Imaging Platform (JIP) of the German Cancer Consortium (DKTK) addresses these issues by providing federated data analysis technology in a secure and compliant way. Using the JIP, medical image data remain in the originator institutions, but analysis and AI algorithms are shared and jointly used. Common standards and interfaces to local systems ensure permanent data sovereignty of participating institutions.

RESULTS

The JIP is established in the radiology and nuclear medicine departments of 10 university hospitals in Germany (DKTK partner sites). In multiple complementary use cases, we show that the platform fulfills all relevant requirements to serve as a foundation for multicenter medical imaging trials and research on large cohorts, including the harmonization and integration of data, interactive analysis, automatic analysis, federated machine learning, and extensibility and maintenance processes, which are elementary for the sustainability of such a platform.

CONCLUSION

The results demonstrate the feasibility of using the JIP as a federated data analytics platform in heterogeneous clinical information technology and software landscapes, solving an important bottleneck for the application of AI to large-scale clinical imaging data.

Contrast agent dose reduction in computed tomography with deep learning using a conditional generative adversarial network

OBJECTIVES

To reduce the dose of intravenous iodine-based contrast media (ICM) in CT through virtual contrast-enhanced images using generative adversarial networks.

METHODS

Dual-energy CTs in the arterial phase of 85 patients were randomly split into an 80/20 train/test collective. Four different generative adversarial networks (GANs) based on image pairs, which comprised one image with virtually reduced ICM and the original full ICM CT slice, were trained, testing two input formats (2D and 2.5D) and two reduced ICM dose levels (-50% and -80%). The amount of intravenous ICM was reduced by creating virtual non-contrast series using dual-energy and adding the corresponding percentage of the iodine map. The evaluation was based on different scores (L1 loss, SSIM, PSNR, FID), which evaluate the image quality and similarity. Additionally, a visual Turing test (VTT) with three radiologists was used to assess the similarity and pathological consistency.

RESULTS

The -80% models reach an SSIM of > 98%, PSNR of > 48, L1 of between 7.5 and 8, and an FID of between 1.6 and 1.7. In comparison, the -50% models reach a SSIM of > 99%, PSNR of > 51, L1 of between 6.0 and 6.1, and an FID between 0.8 and 0.95. For the crucial question of pathological consistency, only the 50% ICM reduction networks achieved 100% consistency, which is required for clinical use.

CONCLUSIONS

The required amount of ICM for CT can be reduced by 50% while maintaining image quality and diagnostic accuracy using GANs. Further phantom studies and animal experiments are required to confirm these initial results.

KEY POINTS

• The amount of contrast media required for CT can be reduced by 50% using generative adversarial networks. • Not only the image quality but especially the pathological consistency must be evaluated to assess safety. • A too pronounced contrast media reduction could influence the pathological consistency in our collective at 80%.

Sociodemographic data and APOE-ε4 augmentation for MRI-based detection of amnestic mild cognitive impairment using deep learning systems

Detection and diagnosis of early and subclinical stages of Alzheimer's Disease (AD) play an essential role in the implementation of intervention and prevention strategies. Neuroimaging techniques predominantly provide insight into anatomic structure changes associated with AD. Deep learning methods have been extensively applied towards creating and evaluating models capable of differentiating between cognitively unimpaired, patients with Mild Cognitive Impairment (MCI) and AD dementia. Several published approaches apply information fusion techniques, providing ways of combining several input sources in the medical domain, which contributes to knowledge of broader and enriched quality. The aim of this paper is to fuse sociodemographic data such as age, marital status, education and gender, and genetic data (presence of an apolipoprotein E (APOE)-ε4 allele) with Magnetic Resonance Imaging (MRI) scans. This enables enriched multi-modal features, that adequately represent the MRI scan visually and is adopted for creating and modeling classification systems capable of detecting amnestic MCI (aMCI). To fully utilize the potential of deep convolutional neural networks, two extra color layers denoting contrast intensified and blurred image adaptations are virtually augmented to each MRI scan, completing the Red-Green-Blue (RGB) color channels. Deep convolutional activation features (DeCAF) are extracted from the average pooling layer of the deep learning system Inception_v3. These features from the fused MRI scans are used as visual representation for the Long Short-Term Memory (LSTM) based Recurrent Neural Network (RNN) classification model. The proposed approach is evaluated on a sub-study containing 120 participants (aMCI = 61 and cognitively unimpaired = 59) of the Heinz Nixdorf Recall (HNR) Study with a baseline model accuracy of 76%. Further evaluation was conducted on the ADNI Phase 1 dataset with 624 participants (aMCI = 397 and cognitively unimpaired = 227) with a baseline model accuracy of 66.27%. Experimental results show that the proposed approach achieves 90% accuracy and 0.90 F1-Score at classification of aMCI vs. cognitively unimpaired participants on the HNR Study dataset, and 77% accuracy and 0.83 F1-Score on the ADNI dataset.

Is there a connection between immunohistochemical markers and grading of lung cancer with apparent diffusion coefficient (ADC) and standardised uptake values (SUV) of hybrid 18F-FDG-PET/MRI?

INTRODUCTION

To correlate tumour grading and prognostic immunohistochemical markers of lung cancer with simultaneously acquired standardised uptake values (SUV) and apparent diffusion coefficient (ADC) derived from hybrid PET/MRI.

METHODS

In this retrospective study, 55 consecutive patients (mean age 62.5 ± 9.2 years) with therapy-naïve, histologically proven lung cancer were included. All patients underwent whole-body PET/MRI using 18F-flourdeoxyglucose (18F-FDG) as a radiotracer. Diffusion-weighted imaging of the chest (DWI, b-values: 0, 500, 1000 s/mm² ) was performed simultaneously with PET acquisition. Histopathological tumour grading was available in 43/55 patients. In 15/55 patients, immunohistochemical markers, that is, phospho-AKT Ser473 (pAKTS473), phosphorylated extracellular signal-regulated kinase (pERK), phosphatase and tensin homolog (PTEN), and human epidermal growth factor receptor 2 (erbB2) were available.

RESULTS

The average SUVmax, SUVmean, ADCmin and ADCmean in lung cancer primaries were 12.6 ± 5.9, 7.7 ± 4.6, 569.9 ± 96.1 s/mm² and 825.8 ± 93.2 s/mm² , respectively. We found a significant inverse correlation between the ADCmin and SUVmax (r = -0.58, P < 0.001) as well as between the ADCmin and SUVmean (r = -0.44, P < 0.001). Tumour grading showed a significant positive correlation with SUVmax and SUVmean (R = 0.34 and R = 0.31, both P < 0.05) and a significant inverse correlation with ADCmin and ADCmean (r = -0.30 and r = -0.40, both P < 0.05). In addition, erbB2 showed a significant inverse correlation with SUVmax and SUVmean (r = -0.50 and r = -0.49, both P < 0.05). The other immunohistochemical markers did not show any significant correlation.

CONCLUSION

18F-FDG-PET/MRI showed weak to moderate correlations between SUV, ADC, tumour grading and erbB2-expression of lung cancer. Hence, 18F-FDG-PET/MRI may, to some extent, offer complementary information to the histopathology of lung cancer, for the evaluation of tumour aggressiveness and treatment response.