Classification of Pancreatic Cysts in Computed Tomography Images Using a Random Forest and Convolutional Neural Network Ensemble

Role of artificial intelligence in the characterization of indeterminate pancreatic head mass and its usefulness in preoperative diagnosis

Artificial intelligence (AI) has been used in various fields of day-to-day life and its role in medicine is immense. Understanding of oncology has been improved with the introduction of AI which helps in diagnosis, treatment planning, management, prognosis, and follow-up. It also helps to identify high-risk groups who can be subjected to timely screening for early detection of malignant conditions. It is more important in pancreatic cancer as it is one of the major causes of cancer-related deaths worldwide and there are no specific early features (clinical and radiological) for diagnosis. With improvement in imaging modalities (computed tomography, magnetic resonance imaging, endoscopic ultrasound), most often clinicians were being challenged with lesions that were difficult to diagnose with human competence. AI has been used in various other branches of medicine to differentiate such indeterminate lesions including the thyroid gland, breast, lungs, liver, adrenal gland, kidney, etc. In the case of pancreatic cancer, the role of AI has been explored and is still ongoing. This review article will focus on how AI can be used to diagnose pancreatic cancer early or differentiate it from benign pancreatic lesions, therefore, management can be planned at an earlier stage.

Preoperative differentiation of pancreatic cystic neoplasm subtypes on computed tomography radiomics

Background

Serous cystic neoplasm (SCN), mucinous cystic neoplasm (MCN), and intraductal papillary mucinous neoplasm (IPMN) comprise a large proportion of pancreatic cystic neoplasms (PCNs). Patients with MCN and IPMN require surgery due to the potential of malignant transformation, whereas those with SCN require periodic surveillance. However, the differential diagnosis of patients with PCNs before treatment remains a great challenge for all surgeons. Therefore, the establishment of a reliable diagnostic tool is urgently required for the improvement of precision diagnostics.

Methods

Between February 2015 and December 2020, 143 consecutive patients with PCNs who were confirmed by postoperative pathology were retrospectively included in the study cohort, then randomized into development and test cohorts at a ratio of 7:3. The predictors of preoperative clinical-radiologic parameters were evaluated by univariate and multivariable logistic regression analyses. A total of 1,218 radiomics features were computationally extracted from the enhanced computed tomography (CT) scans of the tumor region, and a radiomics signature was established by the random forest algorithm. In the development cohort, multi- and binary-class radiomics models integrating preoperative variables and radiomics features were constructed to distinguish between the 3 types of PCNs. The receiver operating characteristic (ROC) curve and the area under the curve (AUC) were used to evaluate the predictive efficiency of the model. An independent internal test cohort was applied to validate the classification models.

Results

All preoperative prediction models were built by integrating the radiomics signature with 13 diagnosis-related radiomics features and 3 important clinical-radiologic parameters: age, sex, and tumor diameter. The multiclass prediction model presented an overall accuracy of 0.804 in the development cohort and 0.707 in the test cohort. The binary-class prediction models displayed higher overall accuracies of 0.853, 0.866, and 0.928 in the development dataset and 0.750, 0.839, and 0.889 in the test dataset. In the test cohort, the binary-class radiomics models showed better predictive performances {AUC =0.914 [95% confidence interval (CI): 0.786 to 1.000], 0.863 (95% CI: 0.714 to 0.941), and 0.926 (95% CI: 0.824 to 1.000)} than the multiclass radiomics model [AUC =0.850 (95% CI: 0.696 to 1.000)], with a large net benefit in the decision curve analysis (DCA). The radiomics-based nomogram provided the correct predicted probability for the diagnosis of PCNs.

Conclusions

The proposed radiomics models with clinical-radiologic parameters and radiomics features help to predict the accurate diagnosis among PCNs to advance personalized medicine.

Differential diagnosis of pancreatic cystic neoplasms through a radiomics-assisted system

Pancreatic cystic neoplasms (PCNs) are a group of heterogeneous diseases with distinct prognosis. Existing differential diagnosis methods require invasive biopsy or prolonged monitoring. We sought to develop an inexpensive, non-invasive differential diagnosis system for PCNs based on radiomics features and clinical characteristics for a higher total PCN screening rate. We retrospectively analyzed computed tomography images and clinical data from 129 patients with PCN, including 47 patients with intraductal papillary mucinous neoplasms (IPMNs), 49 patients with serous cystadenomas (SCNs), and 33 patients with mucinous cystic neoplasms (MCNs). Six clinical characteristics and 944 radiomics features were tested, and nine features were finally selected for model construction using DXScore algorithm. A five-fold cross-validation algorithm and a test group were applied to verify the results. In the five-fold cross-validation section, the AUC value of our model was 0.8687, and the total accuracy rate was 74.23%, wherein the accuracy rates of IPMNs, SCNs, and MCNs were 74.26%, 78.37%, and 68.00%, respectively. In the test group, the AUC value was 0.8462 and the total accuracy rate was 73.61%. In conclusion, our research constructed an end-to-end powerful PCN differential diagnosis system based on radiomics method, which could assist decision-making in clinical practice.

Classification of pancreatic cystic neoplasms using radiomic feature analysis is equivalent to an experienced academic radiologist: a step toward computer-augmented diagnostics for radiologists

PURPOSE

A wide array of benign and malignant lesions of the pancreas can be cystic and these cystic lesions can have overlapping imaging appearances. The purpose of this study is to compare the diagnostic accuracy of a radiomics-based pancreatic cyst classifier to an experienced academic radiologist.

METHODS

In this IRB-approved retrospective single-institution study, patients with surgically resected pancreatic cysts who underwent preoperative abdominal CT from 2003 to 2016 were identified. Pancreatic cyst(s) and background pancreas were manually segmented, and 488 radiomics features were extracted. Random forest classification based on radiomics features, age, and gender was evaluated with fourfold cross-validation. An academic radiologist blinded to the final pathologic diagnosis reviewed each case and provided the most likely diagnosis.

RESULTS

214 patients were included (64 intraductal papillary mucinous neoplasms, 33 mucinous cystic neoplasms, 60 serous cystadenomas, 24 solid pseudopapillary neoplasms, and 33 cystic neuroendocrine tumors). The radiomics-based machine learning approach showed AUC of 0.940 in pancreatic cyst classification, compared with AUC of 0.895 for the radiologist.

CONCLUSION

Radiomics-based machine learning achieved equivalent performance as an experienced academic radiologist in the classification of pancreatic cysts. The high diagnostic accuracy can potentially maximize the efficiency of healthcare utilization by maximizing detection of high-risk lesions.

Artificial intelligence and machine learning in cancer imaging

An increasing array of tools is being developed using artificial intelligence (AI) and machine learning (ML) for cancer imaging. The development of an optimal tool requires multidisciplinary engagement to ensure that the appropriate use case is met, as well as to undertake robust development and testing prior to its adoption into healthcare systems. This multidisciplinary review highlights key developments in the field. We discuss the challenges and opportunities of AI and ML in cancer imaging; considerations for the development of algorithms into tools that can be widely used and disseminated; and the development of the ecosystem needed to promote growth of AI and ML in cancer imaging.

Online calculator for predicting the risk of malignancy in patients with pancreatic cystic neoplasms: A multicenter, retrospective study

BACKGROUND

Efficient and practical methods for predicting the risk of malignancy in patients with pancreatic cystic neoplasms (PCNs) are lacking.

AIM

To establish a nomogram-based online calculator for predicting the risk of malignancy in patients with PCNs.

METHODS

In this study, the clinicopathological data of target patients in three medical centers were analyzed. The independent sample t-test, Mann–Whitney U test or chi-squared test were used as appropriate for statistical analysis. After univariable and multivariable logistic regression analysis, five independent factors were screened and incorporated to develop a calculator for predicting the risk of malignancy. Finally, the concordance index (C-index), calibration, area under the curve, decision curve analysis and clinical impact curves were used to evaluate the performance of the calculator.

RESULTS

Enhanced mural nodules [odds ratio (OR): 4.314; 95% confidence interval (CI): 1.618–11.503, P = 0.003], tumor diameter ≥ 40 mm (OR: 3.514; 95%CI: 1.138–10.849, P = 0.029), main pancreatic duct dilatation (OR: 3.267; 95%CI: 1.230–8.678, P = 0.018), preoperative neutrophil-to-lymphocyte ratio ≥ 2.288 (OR: 2.702; 95%CI: 1.008–7.244, P = 0.048], and preoperative serum CA19-9 concentration ≥ 34 U/mL (OR: 3.267; 95%CI: 1.274–13.007, P = 0.018) were independent risk factors for a high risk of malignancy in patients with PCNs. In the training cohort, the nomogram achieved a C-index of 0.824 for predicting the risk of malignancy. The predictive ability of the model was then validated in an external cohort (C-index: 0.893). Compared with the risk factors identified in the relevant guidelines, the current model showed better predictive performance and clinical utility.

CONCLUSION

The calculator demonstrates optimal predictive performance for identifying the risk of malignancy, potentially yielding a personalized method for patient selection and decision-making in clinical practice.

One 3D VOI-based deep learning radiomics strategy, clinical model and radiologists for predicting lymph node metastases in pancreatic ductal adenocarcinoma based on multiphasic contrast-enhanced computer tomography

Purpose

We designed to construct one 3D VOI-based deep learning radiomics strategy for identifying lymph node metastases (LNM) in pancreatic ductal adenocarcinoma on the basis of multiphasic contrast-enhanced computer tomography and to assist clinical decision-making.

Methods

This retrospective research enrolled 139 PDAC patients undergoing pre-operative arterial phase and venous phase scanning examination between 2015 and 2021. A primary group (training group and validation group) and an independent test group were divided. The DLR strategy included three sections. (1) Residual network three dimensional-18 (Resnet 3D-18) architecture was constructed for deep learning feature extraction. (2) Least absolute shrinkage and selection operator model was used for feature selection. (3) Fully connected network served as the classifier. The DLR strategy was applied for constructing different 3D CNN models using 5-fold cross-validation. Radiomics scores (Rad score) were calculated for distinguishing the statistical difference between negative and positive lymph nodes. A clinical model was constructed by combining significantly different clinical variables using univariate and multivariable logistic regression. The manifestation of two radiologists was detected for comparing with computer-developed models. Receiver operating characteristic curves, the area under the curve, accuracy, precision, recall, and F1 score were used for evaluating model performance.

Results

A total of 45, 49, and 59 deep learning features were selected via LASSO model. No matter in which 3D CNN model, Rad score demonstrated the deep learning features were significantly different between non-LNM and LNM groups. The AP+VP DLR model yielded the best performance in predicting status of lymph node in PDAC with an AUC of 0.995 (95% CI:0.989-1.000) in training group; an AUC of 0.940 (95% CI:0.910-0.971) in validation group; and an AUC of 0.949 (95% CI:0.914-0.984) in test group. The clinical model enrolled the histological grade, CA19-9 level and CT-reported tumor size. The AP+VP DLR model outperformed AP DLR model, VP DLR model, clinical model, and two radiologists.

Conclusions

The AP+VP DLR model based on Resnet 3D-18 demonstrated excellent ability for identifying LNM in PDAC, which could act as a non-invasive and accurate guide for clinical therapeutic strategies. This 3D CNN model combined with 3D tumor segmentation technology is labor-saving, promising, and effective.

Update on quantitative radiomics of pancreatic tumors

Radiomics is a newer approach for analyzing radiological images obtained from conventional imaging modalities such as computed tomography, magnetic resonance imaging, endoscopic ultrasonography, and positron emission tomography. Radiomics involves extracting quantitative data from the images and assessing them to identify diagnostic or prognostic features such as tumor grade, resectability, tumor response to neoadjuvant therapy, and survival. The purpose of this review is to discuss the basic principles of radiomics and provide an overview of the current clinical applications of radiomics in the field of pancreatic tumors.

Research trends of artificial intelligence in pancreatic cancer: a bibliometric analysis

Purpose

We evaluated the related research on artificial intelligence (AI) in pancreatic cancer (PC) through bibliometrics analysis and explored the research hotspots and current status from 1997 to 2021.

Methods

Publications related to AI in PC were retrieved from the Web of Science Core Collection (WoSCC) during 1997-2021. Bibliometrix package of R software 4.0.3 and VOSviewer were used to bibliometrics analysis.

Results

A total of 587 publications in this field were retrieved from WoSCC database. After 2018, the number of publications grew rapidly. The United States and Johns Hopkins University were the most influential country and institution, respectively. A total of 2805 keywords were investigated, 81 of which appeared more than 10 times. Co-occurrence analysis categorized these keywords into five types of clusters: (1) AI in biology of PC, (2) AI in pathology and radiology of PC, (3) AI in the therapy of PC, (4) AI in risk assessment of PC and (5) AI in endoscopic ultrasonography (EUS) of PC. Trend topics and thematic maps show that keywords " diagnosis ", “survival”, “classification”, and “management” are the research hotspots in this field.

Conclusion

The research related to AI in pancreatic cancer is still in the initial stage. Currently, AI is widely studied in biology, diagnosis, treatment, risk assessment, and EUS of pancreatic cancer. This bibliometrics study provided an insight into AI in PC research and helped researchers identify new research orientations.

Application of Artificial Intelligence in the Management of Pancreatic Cystic Lesions

The rate of incidentally detected pancreatic cystic lesions (PCLs) has increased over the past decade and was recently reported at 8%. These lesions pose a unique challenge, as each subtype of PCL carries a different risk of malignant transformation, ranging from 0% (pancreatic pseudocyst) to 34–68% (main duct intraductal papillary mucinous neoplasm). It is imperative to correctly risk-stratify the malignant potential of these lesions in order to provide the correct care course for the patient, ranging from monitoring to surgical intervention. Even with the multiplicity of guidelines (i.e., the American Gastroenterology Association guidelines and Fukuoka/International Consensus guidelines) and multitude of diagnostic information, risk stratification of PCLs falls short. Studies have reported that 25–64% of patients undergoing PCL resection have pancreatic cysts with no malignant potential, and up to 78% of mucin-producing cysts resected harbor no malignant potential on pathological evaluation. Clinicians are now incorporating artificial intelligence technology to aid in the management of these difficult lesions. This review article focuses on advancements in artificial intelligence within digital pathomics, radiomics, and genomics as they apply to the diagnosis and risk stratification of PCLs.

A metric learning-based method using graph neural network for pancreatic cystic neoplasm classification from CTs

PURPOSE

Pancreatic cystic neoplasms (PCNs) are relatively rare neoplasms and difficult to be classified preoperatively. Ordinary deep learning methods have great potential to provide support for doctors in PCNs classification but require a quantity of labeled samples and exact segmentation of neoplasm. The proposed metric learning-based method using graph neural network aims to overcome the limitations brought by small and imbalanced dataset and get fast and accurate PCNs classification result from computed tomography (CT) images.

METHODS

The proposed framework applies graph neural network (GNN). GNNs perform well in fusing information and modeling relational data and get better results on dataset with small size. Based on metric learning strategy, model learns distance from the data. The similarity-based algorithm enhances the classification performance, and more characteristic information is found. We use a convolutional neural network (CNN) to extract features from given images. Then GNN is used to find the similarity between each two feature vectors and complete the classification. Several subtasks consisting of randomly selected images are established to improve generalization of the model. The experiments are carried out on the dataset provided by Huashan Hospital. The dataset is labeled by postoperative pathological analysis and contains ROI information calibrated by experts. We set two tasks based on the dataset: benign or malignant diagnosis of PCNs and classification of specific types.

RESULTS

Our model shows good performance on the 2 tasks with accuracies of 88.926% and 74.497%. The comparison of different methods' F1 scores in the benign or malignant diagnosis shows the proposed GNN-based method effectively reduces the negative impact brought by imbalanced dataset, which is also verified by the macro-average comparison in the 4-class classification task.

CONCLUSIONS

Compared with existing models, the proposed GNN-based model shows better performance in terms of imbalanced dataset with small size while reducing labeling cost. The result provides a possibility for its application into the computer aided diagnosis of PCNs. This article is protected by copyright. All rights reserved.

Using Quantitative Imaging for Personalized Medicine in Pancreatic Cancer: A Review of Radiomics and Deep Learning Applications

Simple Summary

With a five-year survival rate of only 3% for the majority of patients, pancreatic cancer is a global healthcare challenge. Radiomics and deep learning, two novel quantitative imaging methods that treat medical images as minable data instead of just pictures, have shown promise in advancing personalized management of pancreatic cancer through diagnosing precursor diseases, early detection, accurate diagnosis, and treatment personalization. Radiomics and deep learning methods aim to collect hidden information in medical images that is missed by conventional radiology practices through expanding the data search and comparing information across different patients. Both methods have been studied and applied in pancreatic cancer. In this review, we focus on the current progress of these two methods in pancreatic cancer and provide a comprehensive narrative review on the topic. With better regulation, enhanced workflow, and larger prospective patient datasets, radiomics and deep learning methods could show real hope in the battle against pancreatic cancer through personalized precision medicine.

Abstract

As the most lethal major cancer, pancreatic cancer is a global healthcare challenge. Personalized medicine utilizing cutting-edge multi-omics data holds potential for major breakthroughs in tackling this critical problem. Radiomics and deep learning, two trendy quantitative imaging methods that take advantage of data science and modern medical imaging, have shown increasing promise in advancing the precision management of pancreatic cancer via diagnosing of precursor diseases, early detection, accurate diagnosis, and treatment personalization and optimization. Radiomics employs manually-crafted features, while deep learning applies computer-generated automatic features. These two methods aim to mine hidden information in medical images that is missed by conventional radiology and gain insights by systematically comparing the quantitative image information across different patients in order to characterize unique imaging phenotypes. Both methods have been studied and applied in various pancreatic cancer clinical applications. In this review, we begin with an introduction to the clinical problems and the technology. After providing technical overviews of the two methods, this review focuses on the current progress of clinical applications in precancerous lesion diagnosis, pancreatic cancer detection and diagnosis, prognosis prediction, treatment stratification, and radiogenomics. The limitations of current studies and methods are discussed, along with future directions. With better standardization and optimization of the workflow from image acquisition to analysis and with larger and especially prospective high-quality datasets, radiomics and deep learning methods could show real hope in the battle against pancreatic cancer through big data-based high-precision personalization.

Cystic Neoplasms of the Pancreas: Differential Diagnosis and Radiology Correlation

Although the probability of pancreatic cystic neoplasms (PCNs) being detected is raising year by year, their differential diagnosis and individualized treatment are still a challenge in clinical work. PCNs are tumors containing cystic components with different biological behaviors, and their clinical manifestations, epidemiology, imaging features, and malignant risks are different. Some are benign [e.g., serous cystic neoplasms (SCNs)], with a barely possible that turning into malignant, while others display a low or higher malignant risk [e.g., solid pseudopapillary neoplasms (SPNs), intraductal papillary mucinous neoplasms (IPMNs), and mucinous cystic neoplasms (MCNs)]. PCN management should concentrate on preventing the progression of malignant tumors while preventing complications caused by unnecessary surgical intervention. Clinically, various advanced imaging equipment are usually combined to obtain a more reliable preoperative diagnosis. The challenge for clinicians and radiologists is how to accurately diagnose PCNs before surgery so that corresponding surgical methods and follow-up strategies can be developed or not, as appropriate. The objective of this review is to sum up the clinical features, imaging findings and management of the most common PCNs according to the classic literature and latest guidelines.

3D Virtual Pancreatography

We present 3D virtual pancreatography (VP), a novel visualization procedure and application for non-invasive diagnosis and classification of pancreatic lesions, the precursors of pancreatic cancer. Currently, non-invasive screening of patients is performed through visual inspection of 2D axis-aligned CT images, though the relevant features are often not clearly visible nor automatically detected. VP is an end-to-end visual diagnosis system that includes: A machine learning based automatic segmentation of the pancreatic gland and the lesions, a semi-automatic approach to extract the primary pancreatic duct, a machine learning based automatic classification of lesions into four prominent types, and specialized 3D and 2D exploratory visualizations of the pancreas, lesions and surrounding anatomy. We combine volume rendering with pancreas- and lesion-centric visualizations and measurements for effective diagnosis. We designed VP through close collaboration and feedback from expert radiologists, and evaluated it on multiple real-world CT datasets with various pancreatic lesions and case studies examined by the expert radiologists.

Recent advances in the diagnostic evaluation of pancreatic cystic lesions

Pancreatic cystic lesions (PCLs) are becoming more prevalent due to more frequent abdominal imaging and the increasing age of the general population. It has become crucial to identify these PCLs and subsequently risk stratify them to guide management. Given the high morbidity associated with pancreatic surgery, only those PCLs at high risk for malignancy should undergo such treatment. However, current diagnostic testing is suboptimal at accurately diagnosing and risk stratifying PCLs. Therefore, research has focused on developing new techniques for differentiating mucinous from non-mucinous PCLs and identifying high risk lesions for malignancy. Cross sectional imaging radiomics can potentially improve the predictive accuracy of primary risk stratification of PCLs at the time of detection to guide invasive testing. While cyst fluid glucose has reemerged as a potential biomarker, cyst fluid molecular markers have improved accuracy for identifying specific types of PCLs. Endoscopic ultrasound guided approaches such as confocal laser endomicroscopy and through the needle microforceps biopsy have shown a good correlation with histopathological findings and are evolving techniques for identifying and risk stratifying PCLs. While most of these recent diagnostics are only practiced at selective tertiary care centers, they hold a promise that management of PCLs will only get better in the future.

Design of Optimal Deep Learning-Based Pancreatic Tumor and Nontumor Classification Model Using Computed Tomography Scans

Pancreatic tumor is a lethal kind of tumor and its prediction is really poor in the current scenario. Automated pancreatic tumor classification using computer-aided diagnosis (CAD) model is necessary to track, predict, and classify the existence of pancreatic tumors. Artificial intelligence (AI) can offer extensive diagnostic expertise and accurate interventional image interpretation. With this motivation, this study designs an optimal deep learning based pancreatic tumor and nontumor classification (ODL-PTNTC) model using CT images. The goal of the ODL-PTNTC technique is to detect and classify the existence of pancreatic tumors and nontumor. The proposed ODL-PTNTC technique includes adaptive window filtering (AWF) technique to remove noise existing in it. In addition, sailfish optimizer based Kapur's Thresholding (SFO-KT) technique is employed for image segmentation process. Moreover, feature extraction using Capsule Network (CapsNet) is derived to generate a set of feature vectors. Furthermore, Political Optimizer (PO) with Cascade Forward Neural Network (CFNN) is employed for classification purposes. In order to validate the enhanced performance of the ODL-PTNTC technique, a series of simulations take place and the results are investigated under several aspects. A comprehensive comparative results analysis stated the promising performance of the ODL-PTNTC technique over the recent approaches.

Pancreatic Serous Cystic Neoplasms and Mucinous Cystic Neoplasms: Differential Diagnosis by Combining Imaging Features and Enhanced CT Texture Analysis

Objective

To establish a diagnostic model by combining imaging features with enhanced CT texture analysis to differentiate pancreatic serous cystadenomas (SCNs) from pancreatic mucinous cystadenomas (MCNs).

Materials and Methods

Fifty-seven and 43 patients with pathology-confirmed SCNs and MCNs, respectively, from one center were analyzed and divided into a training cohort (n = 72) and an internal validation cohort (n = 28). An external validation cohort (n = 28) from another center was allocated. Demographic and radiological information were collected. The least absolute shrinkage and selection operator (LASSO) and recursive feature elimination linear support vector machine (RFE_LinearSVC) were implemented to select significant features. Multivariable logistic regression algorithms were conducted for model construction. Receiver operating characteristic (ROC) curves for the models were evaluated, and their prediction efficiency was quantified by the area under the curve (AUC), 95% confidence interval (95% CI), sensitivity and specificity.

Results

Following multivariable logistic regression analysis, the AUC was 0.932 and 0.887, the sensitivity was 87.5% and 90%, and the specificity was 82.4% and 84.6% with the training and validation cohorts, respectively, for the model combining radiological features and CT texture features. For the model based on radiological features alone, the AUC was 0.84 and 0.91, the sensitivity was 75% and 66.7%, and the specificity was 82.4% and 77% with the training and validation cohorts, respectively.

Conclusion

This study showed that a logistic model combining radiological features and CT texture features is more effective in distinguishing SCNs from MCNs of the pancreas than a model based on radiological features alone.

COVID-view: Diagnosis of COVID-19 using Chest CT

Significant work has been done towards deep learning (DL) models for automatic lung and lesion segmentation and classification of COVID-19 on chest CT data. However, comprehensive visualization systems focused on supporting the dual visual+DL diagnosis of COVID-19 are non-existent. We present COVID-view, a visualization application specially tailored for radiologists to diagnose COVID-19 from chest CT data. The system incorporates a complete pipeline of automatic lungs segmentation, localization/isolation of lung abnormalities, followed by visualization, visual and DL analysis, and measurement/quantification tools. Our system combines the traditional 2D workflow of radiologists with newer 2D and 3D visualization techniques with DL support for a more comprehensive diagnosis. COVID-view incorporates a novel DL model for classifying the patients into positive/negative COVID-19 cases, which acts as a reading aid for the radiologist using COVID-view and provides the attention heatmap as an explainable DL for the model output. We designed and evaluated COVID-view through suggestions, close feedback and conducting case studies of real-world patient data by expert radiologists who have substantial experience diagnosing chest CT scans for COVID-19, pulmonary embolism, and other forms of lung infections. We present requirements and task analysis for the diagnosis of COVID-19 that motivate our design choices and results in a practical system which is capable of handling real-world patient cases.

Artificial intelligence in gastroenterology: A state-of-the-art review

The development of artificial intelligence (AI) has increased dramatically in the last 20 years, with clinical applications progressively being explored for most of the medical specialties. The field of gastroenterology and hepatology, substantially reliant on vast amounts of imaging studies, is not an exception. The clinical applications of AI systems in this field include the identification of premalignant or malignant lesions (e.g., identification of dysplasia or esophageal adenocarcinoma in Barrett’s esophagus, pancreatic malignancies), detection of lesions (e.g., polyp identification and classification, small-bowel bleeding lesion on capsule endoscopy, pancreatic cystic lesions), development of objective scoring systems for risk stratification, predicting disease prognosis or treatment response [e.g., determining survival in patients post-resection of hepatocellular carcinoma), determining which patients with inflammatory bowel disease (IBD) will benefit from biologic therapy], or evaluation of metrics such as bowel preparation score or quality of endoscopic examination. The objective of this comprehensive review is to analyze the available AI-related studies pertaining to the entirety of the gastrointestinal tract, including the upper, middle and lower tracts; IBD; the hepatobiliary system; and the pancreas, discussing the findings and clinical applications, as well as outlining the current limitations and future directions in this field.

Improving Image Resolution of Whole-Heart Coronary MRA Using Convolutional Neural Network

Whole-heart coronary magnetic resonance angiography (WHCMRA) permits the noninvasive assessment of coronary artery disease without radiation exposure. However, the image resolution of WHCMRA is limited. Recently, convolutional neural networks (CNNs) have obtained increased interest as a method for improving the resolution of medical images. The purpose of this study is to improve the resolution of WHCMRA images using a CNN. Free-breathing WHCMRA images with 512 × 512 pixels (pixel size = 0.65 mm) were acquired in 80 patients with known or suspected coronary artery disease using a 1.5 T magnetic resonance (MR) system with 32 channel coils. A CNN model was optimized by evaluating CNNs with different structures. The proposed CNN model was trained based on the relationship of signal patterns between low-resolution patches (small regions) and the corresponding high-resolution patches using a training dataset collected from 40 patients. Images with 512 × 512 pixels were restored from 256 × 256 down-sampled WHCMRA images (pixel size = 1.3 mm) with three different approaches: the proposed CNN, bicubic interpolation (BCI), and the previously reported super-resolution CNN (SRCNN). High-resolution WHCMRA images obtained using the proposed CNN model were significantly better than those of BCI and SRCNN in terms of root mean squared error, peak signal to noise ratio, and structure similarity index measure with respect to the original WHCMRA images. The proposed CNN approach can provide high-resolution WHCMRA images with better accuracy than BCI and SRCNN. The high-resolution WHCMRA obtained using the proposed CNN model will be useful for identifying coronary artery disease.

Preoperative differentiation of serous cystic neoplasms from mucin-producing pancreatic cystic neoplasms using a CT-based radiomics nomogram

PURPOSE

To develop and validate a CT-based radiomics nomogram in preoperative differential diagnosis of SCNs from mucin-producing PCNs.

MATERIAL AND METHODS

A total of 89 patients consisting of 31 SCNs, 30 IPMNs, and 28 MCNs who underwent preoperative CT were analyzed. A total of 710 radiomics features were extracted from each case. Patients were divided into training (n = 63) and validation cohorts (n = 26) with a ratio of 7:3. Least absolute shrinkage and selection operator (LASSO) method and logistic regression analysis were used for feature selection and model construction. A nomogram was created from a comprehensive model consisting of clinical features and the fusion radiomics signature. A decision curve analysis was used for clinical decisions.

RESULTS

The radiomics features extracted from CT could assist with the differentiation of SCNs from mucin-producing PCNs in both the training and validation cohorts. The signature of the combination of the plain, late arterial, and venous phases had the largest areas under the curve (AUCs) of 0.960 (95% CI 0.910-1) in the training cohort and 0.817 (95% CI 0.651-0.983) in the validation cohort with good calibration. The value and efficacy of the nomogram was verified using decision curve analysis.

CONCLUSION

A comprehensive nomogram incorporating clinical features and fusion radiomics signature can differentiate SCNs from mucin-producing PCNs.

Random forest-based prediction of stroke outcome

We research into the clinical, biochemical and neuroimaging factors associated with the outcome of stroke patients to generate a predictive model using machine learning techniques for prediction of mortality and morbidity 3-months after admission. The dataset consisted of patients with ischemic stroke (IS) and non-traumatic intracerebral hemorrhage (ICH) admitted to Stroke Unit of a European Tertiary Hospital prospectively registered. We identified the main variables for machine learning Random Forest (RF), generating a predictive model that can estimate patient mortality/morbidity according to the following groups: (1) IS + ICH, (2) IS, and (3) ICH. A total of 6022 patients were included: 4922 (mean age 71.9 ± 13.8 years) with IS and 1100 (mean age 73.3 ± 13.1 years) with ICH. NIHSS at 24, 48 h and axillary temperature at admission were the most important variables to consider for evolution of patients at 3-months. IS + ICH group was the most stable for mortality prediction [0.904 ± 0.025 of area under the receiver operating characteristics curve (AUC)]. IS group presented similar results, although variability between experiments was slightly higher (0.909 ± 0.032 of AUC). ICH group was the one in which RF had more problems to make adequate predictions (0.9837 vs. 0.7104 of AUC). There were no major differences between IS and IS + ICH groups according to morbidity prediction (0.738 and 0.755 of AUC) but, after checking normality with a Shapiro Wilk test with the null hypothesis that the data follow a normal distribution, it was rejected with W = 0.93546 (p-value < 2.2e-16). Conditions required for a parametric test do not hold, and we performed a paired Wilcoxon Test assuming the null hypothesis that all the groups have the same performance. The null hypothesis was rejected with a value < 2.2e-16, so there are statistical differences between IS and ICH groups. In conclusion, machine learning algorithms RF can be effectively used in stroke patients for long-term outcome prediction of mortality and morbidity.

Visual Analytics of a Computer-Aided Diagnosis System for Pancreatic Lesions

Machine learning is a powerful and effective tool for medical image analysis to perform computer-aided diagnosis (CAD). Having great potential in improving the accuracy of a diagnosis, CAD systems are often analyzed in terms of the final accuracy, leading to a limited understanding of the internal decision process, impossibility to gain insights, and ultimately to skepticism from clinicians. We present a visual analytics approach to uncover the decision-making process of a CAD system for classifying pancreatic cystic lesions. This CAD algorithm consists of two distinct components: random forest (RF), which classifies a set of predefined features, including demographic features, and a convolutional neural network (CNN), which analyzes radiological (imaging) features of the lesions. We study the class probabilities generated by the RF and the semantical meaning of the features learned by the CNN. We also use an eye tracker to better understand which radiological features are particularly useful for a radiologist to make a diagnosis and to quantitatively compare with the features that lead the CNN to its final classification decision. Additionally, we evaluate the effects and benefits of supplying the CAD system with a case-based visual aid in a second-reader setting.

Artificial intelligence: a critical review of current applications in pancreatic imaging

The applications of artificial intelligence (AI), including machine learning and deep learning, in the field of pancreatic disease imaging are rapidly expanding. AI can be used for the detection of pancreatic ductal adenocarcinoma and other pancreatic tumors but also for pancreatic lesion characterization. In this review, the basic of radiomics, recent developments and current results of AI in the field of pancreatic tumors are presented. Limitations and future perspectives of AI are discussed.

Artificial intelligence for the management of pancreatic diseases

Novel artificial intelligence techniques are emerging in all fields of healthcare, including gastroenterology. The aim of this review is to give an overview of artificial intelligence applications in the management of pancreatic diseases. We performed a systematic literature search in PubMed and Medline up to May 2020 to identify relevant articles. Our results showed that the development of machine-learning based applications is rapidly evolving in the management of pancreatic diseases, guiding precision medicine in clinical, endoscopic and radiologic settings. Before implementation into clinical practice, further research should focus on the external validation of novel techniques, clarifying the accuracy and robustness of these models.

Pancreas image mining: a systematic review of radiomics

OBJECTIVES

To systematically review published studies on the use of radiomics of the pancreas.

METHODS

The search was conducted in the MEDLINE database. Human studies that investigated the applications of radiomics in diseases of the pancreas were included. The radiomics quality score was calculated for each included study.

RESULTS

A total of 72 studies encompassing 8863 participants were included. Of them, 66 investigated focal pancreatic lesions (pancreatic cancer, precancerous lesions, or benign lesions); 4, pancreatitis; and 2, diabetes mellitus. The principal applications of radiomics were differential diagnosis between various types of focal pancreatic lesions (n = 19), classification of pancreatic diseases (n = 23), and prediction of prognosis or treatment response (n = 30). Second-order texture features were most useful for the purpose of differential diagnosis of diseases of the pancreas (with 100% of studies investigating them found a statistically significant feature), whereas filtered image features were most useful for the purpose of classification of diseases of the pancreas and prediction of diseases of the pancreas (with 100% of studies investigating them found a statistically significant feature). The median radiomics quality score of the included studies was 28%, with the interquartile range of 22% to 36%. The radiomics quality score was significantly correlated with the number of extracted radiomics features (r = 0.52, p < 0.001) and the study sample size (r = 0.34, p = 0.003).

CONCLUSIONS

Radiomics of the pancreas holds promise as a quantitative imaging biomarker of both focal pancreatic lesions and diffuse changes of the pancreas. The usefulness of radiomics features may vary depending on the purpose of their application. Standardisation of image acquisition protocols and image pre-processing is warranted prior to considering the use of radiomics of the pancreas in routine clinical practice.

KEY POINTS

• Methodologically sound studies on radiomics of the pancreas are characterised by a large sample size and a large number of extracted features. • Optimisation of the radiomics pipeline will increase the clinical utility of mineable pancreas imaging data. • Radiomics of the pancreas is a promising personalised medicine tool in diseases of the pancreas.

Radiomics for the Diagnosis and Differentiation of Pancreatic Cystic Lesions

Radiomics, also known as quantitative imaging or texture analysis, involves extracting a large number of features traditionally unmeasured in conventional radiological cross-sectional images and converting them into mathematical models. This review describes this approach and its use in the evaluation of pancreatic cystic lesions (PCLs). This discipline has the potential of more accurately assessing, classifying, risk stratifying, and guiding the management of PCLs. Existing studies have provided important insight into the role of radiomics in managing PCLs. Although these studies are limited by the use of retrospective design, single center data, and small sample sizes, radiomic features in combination with clinical data appear to be superior to the current standard of care in differentiating cyst type and in identifying mucinous PCLs with high-grade dysplasia. Combining radiomic features with other novel endoscopic diagnostics, including cyst fluid molecular analysis and confocal endomicroscopy, can potentially optimize the predictive accuracy of these models. There is a need for multicenter prospective studies to elucidate the role of radiomics in the management of PCLs.

Differential Diagnosis for Pancreatic Cysts in CT Scans Using Densely-Connected Convolutional Networks

The lethal nature of pancreatic ductal adenocarcinoma (PDAC) calls for early differential diagnosis of pancreatic cysts, which are identified in up to 16% of normal subjects, and some of them may develop into PDAC. Pancreatic cysts have a large variation in size and shape, and the precise segmentation of them remains rather challenging, which restricts the computer-aided interpretation of CT images acquired for differential diagnosis. We propose a computer-aided framework for early differential diagnosis of pancreatic cysts without pre-segmenting the lesions using densely-connected convolutional networks (Dense-Net). The Dense-Net learns high-level features from whole abnormal pancreas and builds mappings between medical imaging appearance to different pathological types of pancreatic cysts. To enhance the clinical applicability, we integrate saliency maps in the framework to assist the physicians to understand the decision of the deep learning method. The test on a cohort of 206 patients with 4 pathologically confirmed subtypes of pancreatic cysts has achieved an overall accuracy of 72.8%, which is significantly higher than the baseline accuracy of 48.1%. The superior performance on this challenging dataset strongly supports the clinical potential of our developed method.

Elevating pancreatic cystic lesion stratification: Current and future pancreatic cancer biomarker(s)

Pancreatic ductal adenocarcinoma (PDAC) is an incredibly deadly disease with a 5-year survival rate of 9%. The presence of pancreatic cystic lesions (PCLs) confers an increased likelihood of future pancreatic cancer in patients placing them in a high-risk category. Discerning concurrent malignancy and risk of future PCL progression to cancer must be carefully and accurately determined to improve survival outcomes and avoid unnecessary morbidity of pancreatic resection. Unfortunately, current image-based guidelines are inadequate to distinguish benign from malignant lesions. There continues to be a need for accurate molecular and imaging biomarker(s) capable of identifying malignant PCLs and predicting the malignant potential of PCLs to enable risk stratification and effective intervention management. This review provides an update on the current status of biomarkers from pancreatic cystic fluid, pancreatic juice, and seromic molecular analyses and discusses the potential of radiomics for differentiating PCLs harboring cancer from those that do not.

Radiomics in stratification of pancreatic cystic lesions: Machine learning in action

Pancreatic cystic lesions (PCLs) are well-known precursors of pancreatic cancer. Their diagnosis can be challenging as their behavior varies from benign to malignant disease. Precise and timely management of malignant pancreatic cysts might prevent transformation to pancreatic cancer. However, the current consensus guidelines, which rely on standard imaging features to predict cyst malignancy potential, are conflicting and unclear. This has led to an increased interest in radiomics, a high-throughput extraction of comprehensible data from standard of care images. Radiomics can be used as a diagnostic and prognostic tool in personalized medicine. It utilizes quantitative image analysis to extract features in conjunction with machine learning and artificial intelligence (AI) methods like support vector machines, random forest, and convolutional neural network for feature selection and classification. Selected features can then serve as imaging biomarkers to predict high-risk PCLs. Radiomics studies conducted heretofore on PCLs have shown promising results. This cost-effective approach would help us to differentiate benign PCLs from malignant ones and potentially guide clinical decision-making leading to better utilization of healthcare resources. In this review, we discuss the process of radiomics, its myriad applications such as diagnosis, prognosis, and prediction of therapy response. We also discuss the outcomes of studies involving radiomic analysis of PCLs and pancreatic cancer, and challenges associated with this novel field along with possible solutions. Although these studies highlight the potential benefit of radiomics in the prevention and optimal treatment of pancreatic cancer, further studies are warranted before incorporating radiomics into the clinical decision support system.

Computer-Aided Diagnosis of Pancreas Serous Cystic Neoplasms: A Radiomics Method on Preoperative MDCT Images

OBJECTIVE

Our aim was to propose a preoperative computer-aided diagnosis scheme to differentiate pancreatic serous cystic neoplasms from other pancreatic cystic neoplasms, providing supportive opinions for clinicians and avoiding overtreatment.

MATERIALS AND METHODS

In this retrospective study, 260 patients with pancreatic cystic neoplasm were included. Each patient underwent a multidetector row computed tomography scan and pancreatic resection. In all, 200 patients constituted a cross-validation cohort, and 60 patients formed an independent validation cohort. Demographic information, clinical information, and multidetector row computed tomography images were obtained from Picture Archiving and Communication Systems. The peripheral margin of each neoplasm was manually outlined by experienced radiologists. A radiomics system containing 24 guideline-based features and 385 radiomics high-throughput features was designed. After the feature extraction, least absolute shrinkage selection operator regression was used to select the most important features. A support vector machine classifier with 5-fold cross-validation was applied to build the diagnostic model. The independent validation cohort was used to validate the performance.

RESULTS

Only 31 of 102 serous cystic neoplasm cases in this study were recognized correctly by clinicians before the surgery. Twenty-two features were selected from the radiomics system after 100 bootstrapping repetitions of the least absolute shrinkage selection operator regression. The diagnostic scheme performed accurately and robustly, showing the area under the receiver operating characteristic curve = 0.767, sensitivity = 0.686, and specificity = 0.709. In the independent validation cohort, we acquired similar results with receiver operating characteristic curve = 0.837, sensitivity = 0.667, and specificity = 0.818.

CONCLUSION

The proposed radiomics-based computer-aided diagnosis scheme could increase preoperative diagnostic accuracy and assist clinicians in making accurate management decisions.