Deep learning-based automated detection of pulmonary embolism on CT pulmonary angiograms: No significant effects on report communication times and patient turnaround in the emergency department nine months after technical implementation

Efficient management of pulmonary embolism diagnosis using a two-step interconnected machine learning model based on electronic health records data

Pulmonary Embolism (PE) is a life-threatening clinical disease with no specific clinical symptoms and Computed Tomography Angiography (CTA) is used for diagnosis. Clinical decision support scoring systems like Wells and rGeneva based on PE risk factors have been developed to estimate the pre-test probability but are underused, leading to continuous overuse of CTA imaging. This diagnostic study aimed to propose a novel approach for efficient management of PE diagnosis using a two-step interconnected machine learning framework directly by analyzing patients' Electronic Health Records data. First, we performed feature importance analysis according to the result of LightGBM superiority for PE prediction, then four state-of-the-art machine learning methods were applied for PE prediction based on the feature importance results, enabling swift and accurate pre-test diagnosis. Throughout the study patients' data from different departments were collected from Sina educational hospital, affiliated with the Tehran University of medical sciences in Iran. Generally, the Ridge classification method obtained the best performance with an F1 score of 0.96. Extensive experimental findings showed the effectiveness and simplicity of this diagnostic process of PE in comparison with the existing scoring systems. The main strength of this approach centered on PE disease management procedures, which would reduce avoidable invasive CTA imaging and be applied as a primary prognosis of PE, hence assisting the healthcare system, clinicians, and patients by reducing costs and promoting treatment quality and patient satisfaction.

Modern imaging of acute pulmonary embolism

The first-choice imaging test for visualization of thromboemboli in the pulmonary vasculature in patients with suspected acute pulmonary embolism (PE) is multidetector computed tomography pulmonary angiography (CTPA) - a readily available and widely used imaging technique. Through technological advancements over the past years, alternative imaging techniques for the diagnosis of PE have become available, whilst others are still under investigation. In particular, the evolution of artificial intelligence (AI) is expected to enable further innovation in diagnostic management of PE. In this narrative review, current CTPA techniques and the emerging technology photon-counting CT (PCCT), as well as other modern imaging techniques of acute PE are discussed, including CTPA with iodine maps based on subtraction or dual-energy acquisition, single-photon emission CT (SPECT), magnetic resonance angiography (MRA), and magnetic resonance direct thrombus imaging (MRDTI). Furthermore, potential applications of AI are discussed.

Added value of an artificial intelligence algorithm in reducing the number of missed incidental acute pulmonary embolism in routine portal venous phase chest CT

OBJECTIVES

The purpose of this study was to evaluate the incremental value of artificial intelligence (AI) compared to the diagnostic accuracy of radiologists alone in detecting incidental acute pulmonary embolism (PE) on routine portal venous contrast-enhanced chest computed tomography (CT).

METHODS

CTs of 3089 consecutive patients referred to the radiology department for a routine contrast-enhanced chest CT between 27-5-2020 and 31-12-2020, were retrospectively analysed by a CE-certified and FDA-approved AI algorithm. The diagnostic performance of the AI was compared to the initial report. To determine the reference standard, discordant findings were independently evaluated by two readers. In case of disagreement, another experienced cardiothoracic radiologist with knowledge of the initial report and the AI output adjudicated.

RESULTS

The prevalence of acute incidental PE in the reference standard was 2.2% (67 of 3089 patients). In 25 cases, AI detected initially unreported PE. This included three cases concerning central/lobar PE. Sensitivity of the AI algorithm was significantly higher than the outcome of the initial report (respectively 95.5% vs. 62.7%, p < 0.001), whereas specificity was very high for both (respectively 99.6% vs 99.9%, p = 0.012). The AI algorithm only showed a slightly higher amount of false-positive findings (11 vs. 2), resulting in a significantly lower PPV (85.3% vs. 95.5%, p = 0.047).

CONCLUSION

The AI algorithm showed high diagnostic accuracy in diagnosing incidental PE, detecting an additional 25 cases of initially unreported PE, accounting for 37.3% of all positive cases.

CLINICAL RELEVANCE STATEMENT

Radiologist support from AI algorithms in daily practice can prevent missed incidental acute PE on routine chest CT, without a high burden of false-positive cases.

KEY POINTS

• Incidental pulmonary embolism is often missed by radiologists in non-diagnostic scans with suboptimal contrast opacification within the pulmonary trunk. • An artificial intelligence algorithm showed higher sensitivity detecting incidental pulmonary embolism on routine portal venous chest CT compared to the initial report. • Implementation of artificial intelligence support in routine daily practice will reduce the number of missed incidental pulmonary embolism.

Automated detection and segmentation of pulmonary embolisms on computed tomography pulmonary angiography (CTPA) using deep learning but without manual outlining

We present a novel computer algorithm to automatically detect and segment pulmonary embolisms (PEs) on computed tomography pulmonary angiography (CTPA). This algorithm is based on deep learning but does not require manual outlines of the PE regions. Given a CTPA scan, both intra- and extra-pulmonary arteries were firstly segmented. The arteries were then partitioned into several parts based on size (radius). Adaptive thresholding and constrained morphological operations were used to identify suspicious PE regions within each part. The confidence of a suspicious region to be PE was scored based on its contrast in the arteries. This approach was applied to the publicly available RSNA Pulmonary Embolism CT Dataset (RSNA-PE) to identify three-dimensional (3-D) PE negative and positive image patches, which were used to train a 3-D Recurrent Residual U-Net (R2-Unet) to automatically segment PE. The feasibility of this computer algorithm was validated on an independent test set consisting of 91 CTPA scans acquired from a different medical institute, where the PE regions were manually located and outlined by a thoracic radiologist (>18 years' experience). An R2-Unet model was also trained and validated on the manual outlines using a 5-fold cross-validation method. The CNN model trained on the high-confident PE regions showed a Dice coefficient of 0.676±0.168 and a false positive rate of 1.86 per CT scan, while the CNN model trained on the manual outlines demonstrated a Dice coefficient of 0.647±0.192 and a false positive rate of 4.20 per CT scan. The former model performed significantly better than the latter model (p<0.01). The promising performance of the developed PE detection and segmentation algorithm suggests the feasibility of training a deep learning network without dedicating significant efforts to manual annotations of the PE regions on CTPA scans.

Prospective Evaluation of AI Triage of Pulmonary Emboli on CT Pulmonary Angiograms

Background Artificial intelligence (AI) algorithms have shown high accuracy for detection of pulmonary embolism (PE) on CT pulmonary angiography (CTPA) studies in academic studies. Purpose To determine whether use of an AI triage system to detect PE on CTPA studies improves radiologist performance or examination and report turnaround times in a clinical setting. Materials and Methods This prospective single-center study included adult participants who underwent CTPA for suspected PE in a clinical practice setting. Consecutive CTPA studies were evaluated in two phases, first by radiologists alone (n = 31) (May 2021 to June 2021) and then by radiologists aided by a commercially available AI triage system (n = 37) (September 2021 to December 2021). Sixty-two percent of radiologists (26 of 42 radiologists) interpreted studies in both phases. The reference standard was determined by an independent re-review of studies by thoracic radiologists and was used to calculate performance metrics. Diagnostic accuracy and turnaround times were compared using Pearson χ2 and Wilcoxon rank sum tests. Results Phases 1 and 2 included 503 studies (participant mean age, 54.0 years ± 17.8 [SD]; 275 female, 228 male) and 1023 studies (participant mean age, 55.1 years ± 17.5; 583 female, 440 male), respectively. In phases 1 and 2, 14.5% (73 of 503) and 15.9% (163 of 1023) of CTPA studies were positive for PE (P = .47). Mean wait time for positive PE studies decreased from 21.5 minutes without AI to 11.3 minutes with AI (P < .001). The accuracy and miss rate, respectively, for radiologist detection of any PE on CTPA studies was 97.6% and 12.3% without AI and 98.6% and 6.1% with AI, which was not significantly different (P = .15 and P = .11, respectively). Conclusion The use of an AI triage system to detect any PE on CTPA studies improved wait times but did not improve radiologist accuracy, miss rate, or examination and report turnaround times. © RSNA, 2023 Supplemental material is available for this article. See also the editorial by Murphy and Tee in this issue.

Artificial intelligence-based opportunistic detection of coronary artery stenosis on aortic computed tomography angiography in emergency department patients with acute chest pain

Aims

To evaluate a deep-learning model (DLM) for detecting coronary stenoses in emergency room patients with acute chest pain (ACP) explored with electrocardiogram-gated aortic computed tomography angiography (CTA) to rule out aortic dissection.

Methods and results

This retrospective study included 217 emergency room patients (41% female, mean age 67.2 years) presenting with ACP and evaluated by aortic CTA at our institution. Computed tomography angiography was assessed by two readers, who rated the coronary arteries as 1 (no stenosis), 2 (<50% stenosis), or 3 (≥50% stenosis). Computed tomography angiography was categorized as high quality (HQ), if all three main coronary arteries were analysable and low quality (LQ) otherwise. Curvilinear coronary images were rated by a DLM using the same system. Per-patient and per-vessel analyses were conducted. One hundred and twenty-one patients had HQ and 96 LQ CTA. Sensitivity, specificity, positive predictive value, negative predictive value (NPV), and accuracy of the DLM in patients with high-quality image for detecting ≥50% stenoses were 100, 62, 59, 100, and 75% at the patient level and 98, 79, 57, 99, and 84% at the vessel level, respectively. Sensitivity was lower (79%) for detecting ≥50% stenoses at the vessel level in patients with low-quality image. Diagnostic accuracy was 84% in both groups. All 12 patients with acute coronary syndrome (ACS) and stenoses by invasive coronary angiography (ICA) were rated 3 by the DLM.

Conclusion

A DLM demonstrated high NPV for significant coronary artery stenosis in patients with ACP. All patients with ACS and stenoses by ICA were identified by the DLM.

Artificial intelligence in clinical workflow processes in vascular surgery and beyond

In the past decade, artificial intelligence (AI)-based applications have exploded in health care. In cardiovascular disease, and vascular surgery specifically, AI tools such as machine learning, natural language processing, and deep neural networks have been applied to automatically detect underdiagnosed diseases, such as peripheral artery disease, abdominal aortic aneurysms, and atherosclerotic cardiovascular disease. In addition to disease detection and risk stratification, AI has been used to identify guideline-concordant statin therapy use and reasons for nonuse, which has important implications for population-based cardiovascular disease health. Although many studies highlight the potential applications of AI, few address true clinical workflow implementation of available AI-based tools. Specific examples, such as determination of optimal statin treatment based on individual patient risk factors and enhancement of intraoperative fluoroscopy and ultrasound imaging, demonstrate the potential promise of AI integration into clinical workflow. Many challenges to AI implementation in health care remain, including data interoperability, model bias and generalizability, prospective evaluation, privacy and security, and regulation. Multidisciplinary and multi-institutional collaboration, as well as adopting a framework for integration, will be critical for the successful implementation of AI tools into clinical practice.

Radiologist Worklist Reprioritization Using Artificial Intelligence: Impact on Report Turnaround Times for CTPA Examinations Positive for Acute Pulmonary Embolism

BACKGROUND. In patients with acute pulmonary embolism (PE), timely intervention (e.g., initiation of anticoagulation) is critical for optimizing clinical outcomes. OBJECTIVE. The purpose of this study was to evaluate the effect of artificial intelligence (AI)-based radiologist worklist reprioritization on report turnaround times for pulmonary CTA (CTPA) examinations positive for acute PE. METHODS. This retrospective single-center study included patients who underwent CTPA before (October 1, 2018-March 31, 2019 [pre-AI period]) and after (October 1, 2019-March 31, 2020 [post-AI period]) implementation of an AI tool that reprioritized CTPA examinations to the top of radiologists' reading worklists if acute PE was detected. EMR and dictation system timestamps were used to determine the wait time (time from examination completion to report initiation), read time (time from report initiation to report availability), and report turnaround time (sum of wait and read times) for the examinations. Times for reports positive for PE, with final radiology reports as reference, were compared between periods. RESULTS. The study included 2501 examinations of 2197 patients (1307 women, 890 men; mean age, 57.4 ± 17.0 [SD] years), including 1335 examinations from the pre-AI period and 1166 from the post-AI period. The frequency of acute PE, based on radiology reports, was 15.1% (201/1335) during the pre-AI period and 12.3% (144/1166) during the post-AI period. During the post-AI period, the AI tool reprioritized 12.7% (148/1166) of examinations. For PE-positive examinations, the post-AI period, compared with the pre-AI period, had significantly shorter mean report turnaround time (47.6 vs 59.9 minutes; mean difference, 12.3 minutes [95% CI, 0.6-26.0 minutes]) and mean wait time (21.4 vs 33.4 minutes; mean difference, 12.0 minutes [95% CI, 0.9-25.3 minutes]) but no significant difference in mean read time (26.3 vs 26.5 minutes; mean difference, 0.2 minutes [95% CI, -2.8 to 3.2 minutes]). During regular operational hours, wait time was significantly shorter in the post-AI than in the pre-AI period for routine-priority examinations (15.3 vs 43.7 minutes; mean difference, 28.4 minutes [95% CI, 2.2-64.7 minutes]) but not for stat- or urgent-priority examinations. CONCLUSION. AI-driven worklist reprioritization yielded reductions in report turnaround time and wait time for PE-positive CTPA examinations. CLINICAL IMPACT. By assisting radiologists in providing rapid diagnoses, the AI tool has potential for enabling earlier interventions for acute PE.

Role of artificial intelligence in oncologic emergencies: a narrative review

Oncologic emergencies are a wide spectrum of oncologic conditions caused directly by malignancies or their treatment. Oncologic emergencies may be classified according to the underlying physiopathology in metabolic, hematologic, and structural conditions. In the latter, radiologists have a pivotal role, through an accurate diagnosis useful to provide optimal patient care. Structural conditions may involve the central nervous system, thorax, or abdomen, and emergency radiologists have to know the characteristics imaging findings of each one of them. The number of oncologic emergencies is growing due to the increased incidence of malignancies in the general population and also to the improved survival of these patients thanks to the advances in cancer treatment. Artificial intelligence (AI) could be a solution to assist emergency radiologists with this rapidly increasing workload. To our knowledge, AI applications in the setting of the oncologic emergency are mostly underexplored, probably due to the relatively low number of oncologic emergencies and the difficulty in training algorithms. However, cancer emergencies are defined by the cause and not by a specific pattern of radiological symptoms and signs. Therefore, it can be expected that AI algorithms developed for the detection of these emergencies in the non-oncological field can be transferred to the clinical setting of oncologic emergency. In this review, a craniocaudal approach was followed and central nervous system, thoracic, and abdominal oncologic emergencies have been addressed regarding the AI applications reported in literature. Among the central nervous system emergencies, AI applications have been reported for brain herniation and spinal cord compression. In the thoracic district the addressed emergencies were pulmonary embolism, cardiac tamponade and pneumothorax. Pneumothorax was the most frequently described application for AI, to improve sensibility and to reduce the time-to-diagnosis. Finally, regarding abdominal emergencies, AI applications for abdominal hemorrhage, intestinal obstruction, intestinal perforation, and intestinal intussusception have been described.

Artificial Intelligence Tool for Detection and Worklist Prioritization Reduces Time to Diagnosis of Incidental Pulmonary Embolism at CT

Purpose

To evaluate the diagnostic efficacy of artificial intelligence (AI) software in detecting incidental pulmonary embolism (IPE) at CT and shorten the time to diagnosis with use of radiologist reading worklist prioritization.

Materials and Methods

In this study with historical controls and prospective evaluation, regulatory-cleared AI software was evaluated to prioritize IPE on routine chest CT scans with intravenous contrast agent in adult oncology patients. Diagnostic accuracy metrics were calculated, and temporal end points, including detection and notification times (DNTs), were assessed during three time periods (April 2019 to September 2020): routine workflow without AI, human triage without AI, and worklist prioritization with AI.

Results

In total, 11 736 CT scans in 6447 oncology patients (mean age, 63 years ± 12 [SD]; 3367 men) were included. Prevalence of IPE was 1.3% (51 of 3837 scans), 1.4% (54 of 3920 scans), and 1.0% (38 of 3979 scans) for the respective time periods. The AI software detected 131 true-positive, 12 false-negative, 31 false-positive, and 11 559 true-negative results, achieving 91.6% sensitivity, 99.7% specificity, 99.9% negative predictive value, and 80.9% positive predictive value. During prospective evaluation, AI-based worklist prioritization reduced the median DNT for IPE-positive examinations to 87 minutes (vs routine workflow of 7714 minutes and human triage of 4973 minutes). Radiologists' missed rate of IPE was significantly reduced from 44.8% (47 of 105 scans) without AI to 2.6% (one of 38 scans) when assisted by the AI tool (P < .001).

Conclusion

AI-assisted workflow prioritization of IPE on routine CT scans in oncology patients showed high diagnostic accuracy and significantly shortened the time to diagnosis in a setting with a backlog of examinations.Keywords: CT, Computer Applications, Detection, Diagnosis, Embolism, Thorax, ThrombosisSupplemental material is available for this article.© RSNA, 2023See also the commentary by Elicker in this issue.

Incidental pulmonary embolism in patients with cancer: prevalence, underdiagnosis and evaluation of an AI algorithm for automatic detection of pulmonary embolism

OBJECTIVES

To assess the prevalence of reported and unreported incidental pulmonary embolism (iPE) in patients with cancer, and to evaluate an artificial intelligence (AI) algorithm for automatic detection of iPE.

METHODS

Retrospective cohort study on patients with cancer with an elective CT study including the chest between 2018-07-01 and 2019-06-30. All study reports and images were reviewed to identify reported and unreported iPE and were processed by the AI algorithm.

RESULTS

One thousand sixty-nine patients (1892 studies) were included. Per study, iPE was present in 75 studies (4.0%), of which 16 (21.3%) were reported. Unreported iPE had a significantly lower number of involved vessels compared to reported iPE, with a median of 2 (interquartile range, IQR, 1-4) versus 5 (IQR 3-9.75), p < 0.001. There were no significant differences in age, cancer type, or attenuation of the main pulmonary artery. The AI algorithm correctly identified 68 of 75 iPE, with 3 false positives (sensitivity 90.7%, specificity 99.8%, PPV 95.6%, NPV 99.6%). False negatives occurred in cases with 1-3 involved vessels. Of the unreported iPE, 32/59 (54.2%) were proximal to the subsegmental arteries.

CONCLUSION

In patients with cancer, the prevalence of iPE was 4.0%, of which only 21% were reported. Greater than 50% of unreported iPE were proximal to the subsegmental arteries. The AI algorithm had a very high sensitivity and specificity with only three false positives, with the potential to increase the detection rate of iPE.

KEY POINTS

• In a retrospective single-center study on patients with cancer, unreported iPE were common, with the majority lying proximal to the subsegmental arteries. • The evaluated AI algorithm had a very high sensitivity and specificity, so has the potential to increase the detection rate of iPE.

Artificial Intelligence Techniques to Predict the Airway Disorders Illness: A Systematic Review

Airway disease is a major healthcare issue that causes at least 3 million fatalities every year. It is also considered one of the foremost causes of death all around the globe by 2030. Numerous studies have been undertaken to demonstrate the latest advances in artificial intelligence algorithms to assist in identifying and classifying these diseases. This comprehensive review aims to summarise the state-of-the-art machine and deep learning-based systems for detecting airway disorders, envisage the trends of the recent work in this domain, and analyze the difficulties and potential future paths. This systematic literature review includes the study of one hundred fifty-five articles on airway diseases such as cystic fibrosis, emphysema, lung cancer, Mesothelioma, covid-19, pneumoconiosis, asthma, pulmonary edema, tuberculosis, pulmonary embolism as well as highlights the automated learning techniques to predict them. The study concludes with a discussion and challenges about expanding the efficiency and machine and deep learning-assisted airway disease detection applications.

Trends in the application of deep learning networks in medical image analysis: Evolution between 2012 and 2020

PURPOSE

To evaluate the general rules and future trajectories of deep learning (DL) networks in medical image analysis through bibliometric and hot spot analysis of original articles published between 2012 and 2020.

METHODS

Original articles related to DL and medical imaging were retrieved from the PubMed database. For the analysis, data regarding radiological subspecialties; imaging techniques; DL networks; sample size; study purposes, setting, origins and design; statistical analysis; funding sources; authors; and first authors' affiliation was manually extracted from each article. The Bibliographic Item Co-Occurrence Matrix Builder and VOSviewer were used to identify the research topics of the included articles and illustrate the future trajectories of studies.

RESULTS

The study included 2685 original articles. The number of publications on DL and medical imaging has increased substantially since 2017, accounting for 97.2% of all included articles. We evaluated the rules of the application of 47 DL networks to eight radiological tasks on 11 human organ sites. Neuroradiology, thorax, and abdomen were frequent research subjects, while thyroid was under-represented. Segmentation and classification tasks were the primary purposes. U-Net, ResNet, and VGG were the most frequently used Convolutional neural network-derived networks. GAN-derived networks were widely developed and applied in 2020, and transfer learning was highlighted in the COVID-19 studies. Brain, prostate, and diabetic retinopathy-related studies were mature research topics in the field. Breast- and lung-related studies were in a stage of rapid development.

CONCLUSIONS

This study evaluates the general rules and future trajectories of DL network application in medical image analyses and provides guidance for future studies.

Automated Screening for Abdominal Aortic Aneurysm in CT Scans under Clinical Conditions Using Deep Learning

Abdominal aortic aneurysms (AAA) may remain clinically silent until they enlarge and patients present with a potentially lethal rupture. This necessitates early detection and elective treatment. The goal of this study was to develop an easy-to-train algorithm which is capable of automated AAA screening in CT scans and can be applied to an intra-hospital environment. Three deep convolutional neural networks (ResNet, VGG-16 and AlexNet) were adapted for 3D classification and applied to a dataset consisting of 187 heterogenous CT scans. The 3D ResNet outperformed both other networks. Across the five folds of the first training dataset it achieved an accuracy of 0.856 and an area under the curve (AUC) of 0.926. Subsequently, the algorithms performance was verified on a second data set containing 106 scans, where it ran fully automated and resulted in an accuracy of 0.953 and an AUC of 0.971. A layer-wise relevance propagation (LRP) made the decision process interpretable and showed that the network correctly focused on the aortic lumen. In conclusion, the deep learning-based screening proved to be robust and showed high performance even on a heterogeneous multi-center data set. Integration into hospital workflow and its effect on aneurysm management would be an exciting topic of future research.