Expert-level detection of acute intracranial hemorrhage on head computed tomography using deep learning

The new era of artificial intelligence in neuroradiology: current research and promising tools

Radiology has a number of characteristics that make it an especially suitable medical discipline for early artificial intelligence (AI) adoption. These include having a well-established digital workflow, standardized protocols for image storage, and numerous well-defined interpretive activities. The more than 200 commercial radiologic AI-based products recently approved by the Food and Drug Administration (FDA) to assist radiologists in a number of narrow image-analysis tasks such as image enhancement, workflow triage, and quantification, corroborate this observation. However, in order to leverage AI to boost efficacy and efficiency, and to overcome substantial obstacles to widespread successful clinical use of these products, radiologists should become familiarized with the emerging applications in their particular areas of expertise. In light of this, in this article we survey the existing literature on the application of AI-based techniques in neuroradiology, focusing on conditions such as vascular diseases, epilepsy, and demyelinating and neurodegenerative conditions. We also introduce some of the algorithms behind the applications, briefly discuss a few of the challenges of generalization in the use of AI models in neuroradiology, and skate over the most relevant commercially available solutions adopted in clinical practice. If well designed, AI algorithms have the potential to radically improve radiology, strengthening image analysis, enhancing the value of quantitative imaging techniques, and mitigating diagnostic errors.

Semi-supervised Learning for Generalizable Intracranial Hemorrhage Detection and Segmentation

Purpose To develop and evaluate a semi-supervised learning model for intracranial hemorrhage detection and segmentation on an out-of-distribution head CT evaluation set. Materials and Methods This retrospective study used semi-supervised learning to bootstrap performance. An initial "teacher" deep learning model was trained on 457 pixel-labeled head CT scans collected from one U.S. institution from 2010 to 2017 and used to generate pseudo labels on a separate unlabeled corpus of 25 000 examinations from the Radiological Society of North America and American Society of Neuroradiology. A second "student" model was trained on this combined pixel- and pseudo-labeled dataset. Hyperparameter tuning was performed on a validation set of 93 scans. Testing for both classification (n = 481 examinations) and segmentation (n = 23 examinations, or 529 images) was performed on CQ500, a dataset of 481 scans performed in India, to evaluate out-of-distribution generalizability. The semi-supervised model was compared with a baseline model trained on only labeled data using area under the receiver operating characteristic curve, Dice similarity coefficient, and average precision metrics. Results The semi-supervised model achieved a statistically significant higher examination area under the receiver operating characteristic curve on CQ500 compared with the baseline (0.939 [95% CI: 0.938, 0.940] vs 0.907 [95% CI: 0.906, 0.908]; P = .009). It also achieved a higher Dice similarity coefficient (0.829 [95% CI: 0.825, 0.833] vs 0.809 [95% CI: 0.803, 0.812]; P = .012) and pixel average precision (0.848 [95% CI: 0.843, 0.853]) vs 0.828 [95% CI: 0.817, 0.828]) compared with the baseline. Conclusion The addition of unlabeled data in a semi-supervised learning framework demonstrates stronger generalizability potential for intracranial hemorrhage detection and segmentation compared with a supervised baseline. Keywords: Semi-supervised Learning, Traumatic Brain Injury, CT, Machine Learning Supplemental material is available for this article. Published under a CC BY 4.0 license. See also the commentary by Swimburne in this issue.

Imaging Findings in Acute Traumatic Brain Injury: a National Institute of Neurological Disorders and Stroke Common Data Element-Based Pictorial Review and Analysis of Over 4000 Admission Brain Computed Tomography Scans from the Collaborative European NeuroTrauma Effectiveness Research in Traumatic Brain Injury (CENTER-TBI) Study

In 2010, the National Institute of Neurological Disorders and Stroke (NINDS) created a set of common data elements (CDEs) to help standardize the assessment and reporting of imaging findings in traumatic brain injury (TBI). However, as opposed to other standardized radiology reporting systems, a visual overview and data to support the proposed standardized lexicon are lacking. We used over 4000 admission computed tomography (CT) scans of patients with TBI from the Collaborative European NeuroTrauma Effectiveness Research in Traumatic Brain Injury (CENTER-TBI) study to develop an extensive pictorial overview of the NINDS TBI CDEs, with visual examples and background information on individual pathoanatomical lesion types, up to the level of supplemental and emerging information (e.g., location and estimated volumes). We documented the frequency of lesion occurrence, aiming to quantify the relative importance of different CDEs for characterizing TBI, and performed a critical appraisal of our experience with the intent to inform updating of the CDEs. In addition, we investigated the co-occurrence and clustering of lesion types and the distribution of six CT classification systems. The median age of the 4087 patients in our dataset was 50 years (interquartile range, 29-66; range, 0-96), including 238 patients under 18 years old (5.8%). Traumatic subarachnoid hemorrhage (45.3%), skull fractures (37.4%), contusions (31.3%), and acute subdural hematoma (28.9%) were the most frequently occurring CT findings in acute TBI. The ranking of these lesions was the same in patients with mild TBI (baseline Glasgow Coma Scale [GCS] score 13-15) compared with those with moderate-severe TBI (baseline GCS score 3-12), but the frequency of occurrence was up to three times higher in moderate-severe TBI. In most TBI patients with CT abnormalities, there was co-occurrence and clustering of different lesion types, with significant differences between mild and moderate-severe TBI patients. More specifically, lesion patterns were more complex in moderate-severe TBI patients, with more co-existing lesions and more frequent signs of mass effect. These patients also had higher and more heterogeneous CT score distributions, associated with worse predicted outcomes. The critical appraisal of the NINDS CDEs was highly positive, but revealed that full assessment can be time consuming, that some CDEs had very low frequencies, and identified a few redundancies and ambiguity in some definitions. Whilst primarily developed for research, implementation of CDE templates for use in clinical practice is advocated, but this will require development of an abbreviated version. In conclusion, with this study, we provide an educational resource for clinicians and researchers to help assess, characterize, and report the vast and complex spectrum of imaging findings in patients with TBI. Our data provides a comprehensive overview of the contemporary landscape of TBI imaging pathology in Europe, and the findings can serve as empirical evidence for updating the current NINDS radiologic CDEs to version 3.0.

Global research evolution and frontier analysis of artificial intelligence in brain injury: A bibliometric analysis

Research on artificial intelligence for brain injury is currently a prominent area of scientific research. A significant amount of related literature has been accumulated in this field. This study aims to identify hotspots and clarify research resources by conducting literature metrology visualization analysis, providing valuable ideas and references for related fields. The research object of this paper consists of 3000 articles cited in the core database of Web of Science from 1998 to 2023. These articles are visualized and analyzed using VOSviewer and CiteSpace. The bibliometric analysis reveals a continuous increase in the number of articles published on this topic, particularly since 2016, indicating significant growth. The United States stands out as the leading country in artificial intelligence for brain injury, followed by China, which tends to catch up. The core research institutions are primarily universities in developed countries, but there is a lack of cooperation and communication between research groups. With the development of computer technology, the research in this field has shown strong wave characteristics, experiencing the early stage of applied research based on expert systems, the middle stage of prediction research based on machine learning, and the current phase of diversified research focused on deep learning. Artificial intelligence has innovative development prospects in brain injury, providing a new orientation for the treatment and auxiliary diagnosis in this field.

Predicting hematoma expansion in acute spontaneous intracerebral hemorrhage: integrating clinical factors with a multitask deep learning model for non-contrast head CT

PURPOSE

To predict hematoma growth in intracerebral hemorrhage patients by combining clinical findings with non-contrast CT imaging features analyzed through deep learning.

METHODS

Three models were developed to predict hematoma expansion (HE) in 572 patients. We utilized multi-task learning for both hematoma segmentation and prediction of expansion: the Image-to-HE model processed hematoma slices, extracting features and computing a normalized DL score for HE prediction. The Clinical-to-HE model utilized multivariate logistic regression on clinical variables. The Integrated-to-HE model combined image-derived and clinical data. Significant clinical variables were selected using forward selection in logistic regression. The two models incorporating clinical variables were statistically validated.

RESULTS

For hematoma detection, the diagnostic performance of the developed multi-task model was excellent (AUC, 0.99). For expansion prediction, three models were evaluated for predicting HE. The Image-to-HE model achieved an accuracy of 67.3%, sensitivity of 81.0%, specificity of 64.0%, and an AUC of 0.76. The Clinical-to-HE model registered an accuracy of 74.8%, sensitivity of 81.0%, specificity of 73.3%, and an AUC of 0.81. The Integrated-to-HE model, merging both image and clinical data, excelled with an accuracy of 81.3%, sensitivity of 76.2%, specificity of 82.6%, and an AUC of 0.83. The Integrated-to-HE model, aligning closest to the diagonal line and indicating the highest level of calibration, showcases superior performance in predicting HE outcomes among the three models.

CONCLUSION

The integration of clinical findings with non-contrast CT imaging features analyzed through deep learning showed the potential for improving the prediction of HE in acute spontaneous intracerebral hemorrhage patients.

An Efficient CNN-Based Method for Intracranial Hemorrhage Segmentation from Computerized Tomography Imaging

Intracranial hemorrhage (ICH) resulting from traumatic brain injury is a serious issue, often leading to death or long-term disability if not promptly diagnosed. Currently, doctors primarily use Computerized Tomography (CT) scans to detect and precisely locate a hemorrhage, typically interpreted by radiologists. However, this diagnostic process heavily relies on the expertise of medical professionals. To address potential errors, computer-aided diagnosis systems have been developed. In this study, we propose a new method that enhances the localization and segmentation of ICH lesions in CT scans by using multiple images created through different data augmentation techniques. We integrate residual connections into a U-Net-based segmentation network to improve the training efficiency. Our experiments, based on 82 CT scans from traumatic brain injury patients, validate the effectiveness of our approach, achieving an IOU score of 0.807 ± 0.03 for ICH segmentation using 10-fold cross-validation.

Software with artificial intelligence-derived algorithms for analysing CT brain scans in people with a suspected acute stroke: a systematic review and cost-effectiveness analysis

Background

Artificial intelligence-derived software technologies have been developed that are intended to facilitate the review of computed tomography brain scans in patients with suspected stroke.

Objectives

To evaluate the clinical and cost-effectiveness of using artificial intelligence-derived software to support review of computed tomography brain scans in acute stroke in the National Health Service setting.

Methods

Twenty-five databases were searched to July 2021. The review process included measures to minimise error and bias. Results were summarised by research question, artificial intelligence-derived software technology and study type. The health economic analysis focused on the addition of artificial intelligence-derived software-assisted review of computed tomography angiography brain scans for guiding mechanical thrombectomy treatment decisions for people with an ischaemic stroke. The de novo model (developed in R Shiny, R Foundation for Statistical Computing, Vienna, Austria) consisted of a decision tree (short-term) and a state transition model (long-term) to calculate the mean expected costs and quality-adjusted life-years for people with ischaemic stroke and suspected large-vessel occlusion comparing artificial intelligence-derived software-assisted review to usual care.

Results

A total of 22 studies (30 publications) were included in the review; 18/22 studies concerned artificial intelligence-derived software for the interpretation of computed tomography angiography to detect large-vessel occlusion. No study evaluated an artificial intelligence-derived software technology used as specified in the inclusion criteria for this assessment. For artificial intelligence-derived software technology alone, sensitivity and specificity estimates for proximal anterior circulation large-vessel occlusion were 95.4% (95% confidence interval 92.7% to 97.1%) and 79.4% (95% confidence interval 75.8% to 82.6%) for Rapid (iSchemaView, Menlo Park, CA, USA) computed tomography angiography, 91.2% (95% confidence interval 77.0% to 97.0%) and 85.0 (95% confidence interval 64.0% to 94.8%) for Viz LVO (Viz.ai, Inc., San Fransisco, VA, USA) large-vessel occlusion, 83.8% (95% confidence interval 77.3% to 88.7%) and 95.7% (95% confidence interval 91.0% to 98.0%) for Brainomix (Brainomix Ltd, Oxford, UK) e-computed tomography angiography and 98.1% (95% confidence interval 94.5% to 99.3%) and 98.2% (95% confidence interval 95.5% to 99.3%) for Avicenna CINA (Avicenna AI, La Ciotat, France) large-vessel occlusion, based on one study each. These studies were not considered appropriate to inform cost-effectiveness modelling but formed the basis by which the accuracy of artificial intelligence plus human reader could be elicited by expert opinion. Probabilistic analyses based on the expert elicitation to inform the sensitivity of the diagnostic pathway indicated that the addition of artificial intelligence to detect large-vessel occlusion is potentially more effective (quality-adjusted life-year gain of 0.003), more costly (increased costs of £8.61) and cost-effective for willingness-to-pay thresholds of £3380 per quality-adjusted life-year and higher.

Limitations and conclusions

The available evidence is not suitable to determine the clinical effectiveness of using artificial intelligence-derived software to support the review of computed tomography brain scans in acute stroke. The economic analyses did not provide evidence to prefer the artificial intelligence-derived software strategy over current clinical practice. However, results indicated that if the addition of artificial intelligence-derived software-assisted review for guiding mechanical thrombectomy treatment decisions increased the sensitivity of the diagnostic pathway (i.e. reduced the proportion of undetected large-vessel occlusions), this may be considered cost-effective.

Future work

Large, preferably multicentre, studies are needed (for all artificial intelligence-derived software technologies) that evaluate these technologies as they would be implemented in clinical practice.

Study registration

This study is registered as PROSPERO CRD42021269609.

Funding

This award was funded by the National Institute for Health and Care Research (NIHR) Evidence Synthesis programme (NIHR award ref: NIHR133836) and is published in full in Health Technology Assessment; Vol. 28, No. 11. See the NIHR Funding and Awards website for further award information.

AI in Orthodontics: Revolutionizing Diagnostics and Treatment Planning-A Comprehensive Review

The advent of artificial intelligence (AI) in medicine has transformed various medical specialties, including orthodontics. AI has shown promising results in enhancing the accuracy of diagnoses, treatment planning, and predicting treatment outcomes. Its usage in orthodontic practices worldwide has increased with the availability of various AI applications and tools. This review explores the principles of AI, its applications in orthodontics, and its implementation in clinical practice. A comprehensive literature review was conducted, focusing on AI applications in dental diagnostics, cephalometric evaluation, skeletal age determination, temporomandibular joint (TMJ) evaluation, decision making, and patient telemonitoring. Due to study heterogeneity, no meta-analysis was possible. AI has demonstrated high efficacy in all these areas, but variations in performance and the need for manual supervision suggest caution in clinical settings. The complexity and unpredictability of AI algorithms call for cautious implementation and regular manual validation. Continuous AI learning, proper governance, and addressing privacy and ethical concerns are crucial for successful integration into orthodontic practice.

Strengthening deep-learning models for intracranial hemorrhage detection: strongly annotated computed tomography images and model ensembles

Background and purpose

Multiple attempts at intracranial hemorrhage (ICH) detection using deep-learning techniques have been plagued by clinical failures. We aimed to compare the performance of a deep-learning algorithm for ICH detection trained on strongly and weakly annotated datasets, and to assess whether a weighted ensemble model that integrates separate models trained using datasets with different ICH improves performance.

Methods

We used brain CT scans from the Radiological Society of North America (27,861 CT scans, 3,528 ICHs) and AI-Hub (53,045 CT scans, 7,013 ICHs) for training. DenseNet121, InceptionResNetV2, MobileNetV2, and VGG19 were trained on strongly and weakly annotated datasets and compared using independent external test datasets. We then developed a weighted ensemble model combining separate models trained on all ICH, subdural hemorrhage (SDH), subarachnoid hemorrhage (SAH), and small-lesion ICH cases. The final weighted ensemble model was compared to four well-known deep-learning models. After external testing, six neurologists reviewed 91 ICH cases difficult for AI and humans.

Results

InceptionResNetV2, MobileNetV2, and VGG19 models outperformed when trained on strongly annotated datasets. A weighted ensemble model combining models trained on SDH, SAH, and small-lesion ICH had a higher AUC, compared with a model trained on all ICH cases only. This model outperformed four deep-learning models (AUC [95% C.I.]: Ensemble model, 0.953[0.938-0.965]; InceptionResNetV2, 0.852[0.828-0.873]; DenseNet121, 0.875[0.852-0.895]; VGG19, 0.796[0.770-0.821]; MobileNetV2, 0.650[0.620-0.680]; p < 0.0001). In addition, the case review showed that a better understanding and management of difficult cases may facilitate clinical use of ICH detection algorithms.

Conclusion

We propose a weighted ensemble model for ICH detection, trained on large-scale, strongly annotated CT scans, as no model can capture all aspects of complex tasks.

Diagnostic test accuracy of machine learning algorithms for the detection intracranial hemorrhage: a systematic review and meta-analysis study

BACKGROUND

This systematic review and meta-analysis were conducted to objectively evaluate the evidence of machine learning (ML) in the patient diagnosis of Intracranial Hemorrhage (ICH) on computed tomography (CT) scans.

METHODS

Until May 2023, systematic searches were conducted in ISI Web of Science, PubMed, Scopus, Cochrane Library, IEEE Xplore Digital Library, CINAHL, Science Direct, PROSPERO, and EMBASE for studies that evaluated the diagnostic precision of ML model-assisted ICH detection. Patients with and without ICH as the target condition who were receiving CT-Scan were eligible for the research, which used ML algorithms based on radiologists' reports as the gold reference standard. For meta-analysis, pooled sensitivities, specificities, and a summary receiver operating characteristics curve (SROC) were used.

RESULTS

At last, after screening the title, abstract, and full paper, twenty-six retrospective and three prospective, and two retrospective/prospective studies were included. The overall (Diagnostic Test Accuracy) DTA of retrospective studies with a pooled sensitivity was 0.917 (95% CI 0.88-0.943, I2 = 99%). The pooled specificity was 0.945 (95% CI 0.918-0.964, I2 = 100%). The pooled diagnostic odds ratio (DOR) was 219.47 (95% CI 104.78-459.66, I2 = 100%). These results were significant for the specificity of the different network architecture models (p-value = 0.0289). However, the results for sensitivity (p-value = 0.6417) and DOR (p-value = 0.2187) were not significant. The ResNet algorithm has higher pooled specificity than other algorithms with 0.935 (95% CI 0.854-0.973, I2 = 93%).

CONCLUSION

This meta-analysis on DTA of ML algorithms for detecting ICH by assessing non-contrast CT-Scans shows the ML has an acceptable performance in diagnosing ICH. Using ResNet in ICH detection remains promising prediction was improved via training in an Architecture Learning Network (ALN).

Intracranial hemorrhages in patients with COVID-19: a systematic review of the literature, regarding six cases in an Amazonian population

BACKGROUND

 Coronavirus disease 2019 (COVID-19) has emerged as a public health emergency worldwide, predominantly affecting the respiratory tract. However, evidence supports the involvement of extrapulmonary sites, including reports of intracranial hemorrhages.

OBJECTIVE

 To describe six original cases and review the literature on intracranial hemorrhages in patients diagnosed with COVID-19 by molecular methods.

METHODS

 A systematic literature review was performed on MEDLINE, PubMed, and NCBI electronic databases to identify eligible studies. Of the total 1,624 articles retrieved, only 53 articles met the inclusion criteria.

RESULTS

 The overall incidence of intracranial hemorrhage in patients hospitalized for COVID-19 was 0.26%. In this patient group, the mean age was 60 years, and the majority were male (68%) with initial respiratory symptoms (73%) and some comorbidity. Before the diagnosis of hemorrhage, 43% of patients were using anticoagulants, 47.3% at therapeutic doses. The intraparenchymal (50%) was the most affected compartment, followed by the subarachnoid (34%), intraventricular (11%), and subdural (7%). There was a predominance of lobar over non-lobar topographies. Multifocal or multicompartmental hemorrhages were described in 25% of cases. Overall mortality in the cohort studies was 44%, while around 55% of patients were discharged from hospital.

CONCLUSION

 Despite the unusual association, the combination of these two diseases is associated with high rates of mortality and morbidity, as well as more severe clinicoradiological presentations. Further studies are needed to provide robust evidence on the exact pathophysiology behind the occurrence of intracranial hemorrhages after COVID-19 infection.

Temporal Uncertainty Localization to Enable Human-in-the-Loop Analysis of Dynamic Contrast-Enhanced Cardiac MRI Datasets

Dynamic contrast-enhanced (DCE) cardiac magnetic resonance imaging (CMRI) is a widely used modality for diagnosing myocardial blood flow (perfusion) abnormalities. During a typical free-breathing DCE-CMRI scan, close to 300 time-resolved images of myocardial perfusion are acquired at various contrast "wash in/out" phases. Manual segmentation of myocardial contours in each time-frame of a DCE image series can be tedious and time-consuming, particularly when non-rigid motion correction has failed or is unavailable. While deep neural networks (DNNs) have shown promise for analyzing DCE-CMRI datasets, a "dynamic quality control" (dQC) technique for reliably detecting failed segmentations is lacking. Here we propose a new space-time uncertainty metric as a dQC tool for DNN-based segmentation of free-breathing DCE-CMRI datasets by validating the proposed metric on an external dataset and establishing a human-in-the-loop framework to improve the segmentation results. In the proposed approach, we referred the top 10% most uncertain segmentations as detected by our dQC tool to the human expert for refinement. This approach resulted in a significant increase in the Dice score (p < 0.001) and a notable decrease in the number of images with failed segmentation (16.2% to 11.3%) whereas the alternative approach of randomly selecting the same number of segmentations for human referral did not achieve any significant improvement. Our results suggest that the proposed dQC framework has the potential to accurately identify poor-quality segmentations and may enable efficient DNN-based analysis of DCE-CMRI in a human-in-the-loop pipeline for clinical interpretation and reporting of dynamic CMRI datasets.

Automated intracranial hemorrhage detection in traumatic brain injury using 3D CNN

Objectives

Intracranial hemorrhage (ICH) is a prevalent and potentially fatal consequence of traumatic brain injury (TBI). Timely identification of ICH is crucial to ensure timely intervention and to optimize better patient outcomes. However, the current methods for diagnosing ICH from head computed tomography (CT) scans require skilled personnel (Radiologists and/or Neurosurgeons) who may be unavailable in all centers, especially in rural areas. The aim of this study is to develop a neurotrauma screening tool for identifying ICH from head CT scans of TBI patients.

Materials and Methods

We prospectively collected head CT scans from the Department of Neurosurgery, All India Institute of Medical Sciences, New Delhi. Approximately 738 consecutive head CT scans from patients enrolled in the department were collected for this study spanning a duration of 9 months, that is, January 2020 to September 2020. The metadata collected along with the head CT scans consisted of demographic and clinical details and the radiologist's report which was used as the gold standard. A deep learning-based 3D convolutional neural network (CNN) model was trained on the dataset. The pre-processing, hyperparameters, and augmentation were common for training the 3D CNN model whereas the training modules were set differently. The model was trained along with the save best model option and was monitored by validation metrics. The Institute Ethics Committee permission was taken before starting the study.

Results

We developed a 3D CNN model for automatically detecting the ICH from head CT scans. The screening tool was tested in 20 cases and trained on 200 head CT scans, with 99 normal head CT and 101 CT scans with some type of ICH. The final model performed with 90% sensitivity, 70% specificity, and 80% accuracy.

Conclusion

Our study reveals that the automated screening tool exhibits a commendable level of accuracy and sensitivity in detecting ICH from the head CT scans. The results indicate that the 3D CNN approach has a potential for further exploring the TBI-related pathologies.

Application of deep learning models for detection of subdural hematoma: a systematic review and meta-analysis

BACKGROUND

This study aimed to investigate the application of deep learning (DL) models for the detection of subdural hematoma (SDH).

METHODS

We conducted a comprehensive search using relevant keywords. Articles extracted were original studies in which sensitivity and/or specificity were reported. Two different approaches of frequentist and Bayesian inference were applied. For quality and risk of bias assessment we used Quality Assessment of Diagnostic Accuracy Studies-2 (QUADAS-2).

RESULTS

We analyzed 22 articles that included 1,997,749 patients. In the first step, the frequentist method showed a pooled sensitivity of 88.8% (95% confidence interval (CI): 83.9% to 92.4%) and a specificity of 97.2% (95% CI 94.6% to 98.6%). In the second step, using Bayesian methods including 11 studies that reported sensitivity and specificity, a sensitivity rate of 86.8% (95% CI: 77.6% to 92.9%) at a specificity level of 86.9% (95% CI: 60.9% to 97.2%) was achieved. The risk of bias assessment was not remarkable using QUADAS-2.

CONCLUSION

DL models might be an appropriate tool for detecting SDHs with a reasonably high sensitivity and specificity.

Deep learning-assisted identification and quantification of aneurysmal subarachnoid hemorrhage in non-contrast CT scans: Development and external validation of Hybrid 2D/3D UNet

Accurate stroke assessment and consequent favorable clinical outcomes rely on the early identification and quantification of aneurysmal subarachnoid hemorrhage (aSAH) in non-contrast computed tomography (NCCT) images. However, hemorrhagic lesions can be complex and difficult to distinguish manually. To solve these problems, here we propose a novel Hybrid 2D/3D UNet deep-learning framework for automatic aSAH identification and quantification in NCCT images. We evaluated 1824 consecutive patients admitted with aSAH to four hospitals in China between June 2018 and May 2022. Accuracy and precision, Dice scores and intersection over union (IoU), and interclass correlation coefficients (ICC) were calculated to assess model performance, segmentation performance, and correlations between automatic and manual segmentation, respectively. A total of 1355 patients with aSAH were enrolled: 931, 101, 179, and 144 in four datasets, of whom 326 were scanned with Siemens, 640 with Philips, and 389 with GE Medical Systems scanners. Our proposed deep-learning method accurately identified (accuracies 0.993-0.999) and segmented (Dice scores 0.550-0.897) hemorrhage in both the internal and external datasets, even combinations of hemorrhage subtypes. We further developed a convenient AI-assisted platform based on our algorithm to assist clinical workflows, whose performance was comparable to manual measurements by experienced neurosurgeons (ICCs 0.815-0.957) but with greater efficiency and reduced cost. While this tool has not yet been prospectively tested in clinical practice, our innovative hybrid network algorithm and platform can accurately identify and quantify aSAH, paving the way for fast and cheap NCCT interpretation and a reliable AI-based approach to expedite clinical decision-making for aSAH patients.

A systematic review on intracranial aneurysm and hemorrhage detection using machine learning and deep learning techniques

The risk of discovering an intracranial aneurysm during the initial screening and follow-up screening are reported as around 11%, and 7% respectively (Zuurbie et al., 2023) to these mass effects, unruptured aneurysms frequently generate symptoms, however, the real hazard occurs when an aneurysm ruptures and results in a cerebral hemorrhage known as a subarachnoid hemorrhage. The objective is to study the multiple kinds of hemorrhage and aneurysm detection problems and develop machine and deep learning models to recognise them. Due to its early stage, subarachnoid hemorrhage, the most typical symptom after aneurysm rupture, is an important medical condition. It frequently results in severe neurological emergencies or even death. Although most aneurysms are asymptomatic and won't burst, because of their unpredictable growth, even small aneurysms are susceptible. A timely diagnosis is essential to prevent early mortality because a large percentage of hemorrhage cases present can be fatal. Physiological/imaging markers and the degree of the subarachnoid hemorrhage can be used as indicators for potential early treatments in hemorrhage. The hemodynamic pathomechanisms and microcellular environment should remain a priority for academics and medical professionals. There is still disagreement about how and when to care for aneurysms that have not ruptured despite studies reporting on the risk of rupture and outcomes. We are optimistic that with the progress in our understanding of the pathophysiology of hemorrhages and aneurysms and the advancement of artificial intelligence has made it feasible to conduct analyses with a high degree of precision, effectiveness and reliability.

Role of artificial intelligence and machine learning in the diagnosis of cerebrovascular disease

Introduction

Cerebrovascular diseases are known to cause significant morbidity and mortality to the general population. In patients with cerebrovascular disease, prompt clinical evaluation and radiographic interpretation are both essential in optimizing clinical management and in triaging patients for critical and potentially life-saving neurosurgical interventions. With recent advancements in the domains of artificial intelligence (AI) and machine learning (ML), many AI and ML algorithms have been developed to further optimize the diagnosis and subsequent management of cerebrovascular disease. Despite such advances, further studies are needed to substantively evaluate both the diagnostic accuracy and feasibility of these techniques for their application in clinical practice. This review aims to analyze the current use of AI and MI algorithms in the diagnosis of, and clinical decision making for cerebrovascular disease, and to discuss both the feasibility and future applications of utilizing such algorithms.

Methods

We review the use of AI and ML algorithms to assist clinicians in the diagnosis and management of ischemic stroke, hemorrhagic stroke, intracranial aneurysms, and arteriovenous malformations (AVMs). After identifying the most widely used algorithms, we provide a detailed analysis of the accuracy and effectiveness of these algorithms in practice.

Results

The incorporation of AI and ML algorithms for cerebrovascular patients has demonstrated improvements in time to detection of intracranial pathologies such as intracerebral hemorrhage (ICH) and infarcts. For ischemic and hemorrhagic strokes, commercial AI software platforms such as RapidAI and Viz.AI have bene implemented into routine clinical practice at many stroke centers to expedite the detection of infarcts and ICH, respectively. Such algorithms and neural networks have also been analyzed for use in prognostication for such cerebrovascular pathologies. These include predicting outcomes for ischemic stroke patients, hematoma expansion, risk of aneurysm rupture, bleeding of AVMs, and in predicting outcomes following interventions such as risk of occlusion for various endovascular devices. Preliminary analyses have yielded promising sensitivities when AI and ML are used in concert with imaging modalities and a multidisciplinary team of health care providers.

Conclusion

The implementation of AI and ML algorithms to supplement clinical practice has conferred a high degree of accuracy, efficiency, and expedited detection in the clinical and radiographic evaluation and management of ischemic and hemorrhagic strokes, AVMs, and aneurysms. Such algorithms have been explored for further purposes of prognostication for these conditions, with promising preliminary results. Further studies should evaluate the longitudinal implementation of such techniques into hospital networks and residency programs to supplement clinical practice, and the extent to which these techniques improve patient care and clinical outcomes in the long-term.

Real world validation of an AI-based CT hemorrhage detection tool

Introduction

Intracranial hemorrhage (ICH) is a potentially life-threatening medical event that requires expedited diagnosis with computed tomography (CT). Automated medical imaging triaging tools can rapidly bring scans containing critical abnormalities, such as ICH, to the attention of radiologists and clinicians. Here, we retrospectively investigated the real-world performance of VeriScout™, an artificial intelligence-based CT hemorrhage detection and triage tool.

Methods

Ground truth for the presence or absence of ICH was iteratively determined by expert consensus in an unselected dataset of 527 consecutively acquired non-contrast head CT scans, which were sub-grouped according to the presence of artefact, post-operative features and referral source. The performance of VeriScout™ was compared with the ground truths for all groups.

Results

VeriScout™ detected hemorrhage with a sensitivity of 0.92 (CI 0.84-0.96) and a specificity of 0.96 (CI 0.94-0.98) in the global dataset, exceeding the sensitivity of general radiologists (0.88) with only a minor relative decrement in specificity (0.98). Crucially, the AI tool detected 13/14 cases of subarachnoid hemorrhage, a potentially fatal condition that is often missed in emergency department settings. There was no decrement in the performance of VeriScout™ in scans containing artefact or postoperative change. Using an integrated informatics platform, VeriScout™ was deployed into the existing radiology workflow. Detected hemorrhage cases were flagged in the hospital radiology information system (RIS) and relevant, annotated, preview images made available in the picture archiving and communications system (PACS) within 10 min.

Conclusion

AI-based radiology worklist prioritization for critical abnormalities, such as ICH, may enhance patient care without adding to radiologist or clinician burden.

Clinical Utility of Near-Infrared Device in Detecting Traumatic Intracranial Hemorrhage: A Pilot Study Toward Application as an Emergent Diagnostic Modality in a Low-Resource Setting

Limited computed tomography (CT) availability in low- and middle-income countries frequently impedes life-saving neurosurgical decompression for traumatic brain injury. A reliable, accessible, cost-effective solution is necessary to detect and localize bleeds. We report the largest study to date using a near-infrared device (NIRD) to detect traumatic intracranial bleeds. Patients with confirmed or suspected head trauma who received a head CT scan were included. Within 30 min of the initial head CT scan, a blinded examiner scanned each patient's cranium with a NIRD, interrogating bilaterally the frontal, parietal, temporal, and occipital quadrants Sensitivity, specificity, accuracy, and precision were investigated. We recruited 500 consecutive patients; 104 patients had intracranial bleeding. For all patients with CT-proven bleeds, irrespective of size, initial NIRD scans localized the bleed to the appropriate quadrant with a sensitivity of 86% and specificity of 96% compared with CT. For extra-axial bleeds >3.5mL, sensitivity and specificity were 94% and 96%, respectively. For longitudinal serial rescans with the NIRD, sensitivity was 89% (< 4 days from injury: sensitivity: 99%), and specificity was 96%. For all patients who required craniectomy or craniotomy, the device demonstrated 100% sensitivity. NIRD is highly sensitive, specific, and reproducible over time in diagnosing intracranial bleeds. NIRD may inform neurosurgical decision making in settings where CT scanning is unavailable or impractical.

Label-efficient deep semantic segmentation of intracranial hemorrhages in CT-scans

Intracranial hemorrhage (ICH) is a common finding in traumatic brain injury (TBI) and computed tomography (CT) is considered the gold standard for diagnosis. Automated detection of ICH provides clinical value in diagnostics and in the ability to feed robust quantification measures into future prediction models. Several studies have explored ICH detection and segmentation but the research process is somewhat hindered due to a lack of open large and labeled datasets, making validation and comparison almost impossible. The complexity of the task is further challenged by the heterogeneity of ICH patterns, requiring a large number of labeled data to train robust and reliable models. Consequently, due to the labeling cost, there is a need for label-efficient algorithms that can exploit easily available unlabeled or weakly-labeled data. Our aims for this study were to evaluate whether transfer learning can improve ICH segmentation performance and to compare a variety of transfer learning approaches that harness unlabeled and weakly-labeled data. Three self-supervised and three weakly-supervised transfer learning approaches were explored. To be used in our comparisons, we also manually labeled a dataset of 51 CT scans. We demonstrate that transfer learning improves ICH segmentation performance on both datasets. Unlike most studies on ICH segmentation our work relies exclusively on publicly available datasets, allowing for easy comparison of performances in future studies. To further promote comparison between studies, we also present a new public dataset of ICH-labeled CT scans, Seq-CQ500.

Intracranial hemorrhage detection in 3D computed tomography images using a bi-directional long short-term memory network-based modified genetic algorithm

Introduction

Intracranial hemorrhage detection in 3D Computed Tomography (CT) brain images has gained more attention in the research community. The major issue to deal with the 3D CT brain images is scarce and hard to obtain the labelled data with better recognition results.

Methods

To overcome the aforementioned problem, a new model has been implemented in this research manuscript. After acquiring the images from the Radiological Society of North America (RSNA) 2019 database, the region of interest (RoI) was segmented by employing Otsu's thresholding method. Then, feature extraction was performed utilizing Tamura features: directionality, contrast, coarseness, and Gradient Local Ternary Pattern (GLTP) descriptors to extract vectors from the segmented RoI regions. The extracted vectors were dimensionally reduced by proposing a modified genetic algorithm, where the infinite feature selection technique was incorporated with the conventional genetic algorithm to further reduce the redundancy within the regularized vectors. The selected optimal vectors were finally fed to the Bi-directional Long Short Term Memory (Bi-LSTM) network to classify intracranial hemorrhage sub-types, such as subdural, intraparenchymal, subarachnoid, epidural, and intraventricular.

Results

The experimental investigation demonstrated that the Bi-LSTM based modified genetic algorithm obtained 99.40% sensitivity, 99.80% accuracy, and 99.48% specificity, which are higher compared to the existing machine learning models: Naïve Bayes, Random Forest, Support Vector Machine (SVM), Recurrent Neural Network (RNN), and Long Short-Term Memory (LSTM) network.

Artificial Intelligence and Glaucoma: Going Back to Basics

There has been a recent surge in the number of publications centered on the use of artificial intelligence (AI) to diagnose various systemic diseases. The Food and Drug Administration has approved several algorithms for use in clinical practice. In ophthalmology, most advances in AI relate to diabetic retinopathy, which is a disease process with agreed upon diagnostic and classification criteria. However, this is not the case for glaucoma, which is a relatively complex disease without agreed-upon diagnostic criteria. Moreover, currently available public datasets that focus on glaucoma have inconstant label quality, further complicating attempts at training AI algorithms efficiently. In this perspective paper, we discuss specific details related to developing AI models for glaucoma and suggest potential steps to overcome current limitations.

Artificial Intelligence and Machine Learning in the Diagnosis and Management of Stroke: A Narrative Review of United States Food and Drug Administration-Approved Technologies

Stroke is an emergency in which delays in treatment can lead to significant loss of neurological function and be fatal. Technologies that increase the speed and accuracy of stroke diagnosis or assist in post-stroke rehabilitation can improve patient outcomes. No resource exists that comprehensively assesses artificial intelligence/machine learning (AI/ML)-enabled technologies indicated for the management of ischemic and hemorrhagic stroke. We queried a United States Food and Drug Administration (FDA) database, along with PubMed and private company websites, to identify the recent literature assessing the clinical performance of FDA-approved AI/ML-enabled technologies. The FDA has approved 22 AI/ML-enabled technologies that triage brain imaging for more immediate diagnosis or promote post-stroke neurological/functional recovery. Technologies that assist with diagnosis predominantly use convolutional neural networks to identify abnormal brain images (e.g., CT perfusion). These technologies perform comparably to neuroradiologists, improve clinical workflows (e.g., time from scan acquisition to reading), and improve patient outcomes (e.g., days spent in the neurological ICU). Two devices are indicated for post-stroke rehabilitation by leveraging neuromodulation techniques. Multiple FDA-approved technologies exist that can help clinicians better diagnose and manage stroke. This review summarizes the most up-to-date literature regarding the functionality, performance, and utility of these technologies so clinicians can make informed decisions when using them in practice.

Research advances in imaging markers for predicting hematoma expansion in intracerebral hemorrhage: a narrative review

Introduction

Stroke is a major global health concern and is ranked as the second leading cause of death worldwide, with the third highest incidence of disability. Intracerebral hemorrhage (ICH) is a devastating form of stroke that is responsible for a significant proportion of stroke-related morbidity and mortality worldwide. Hematoma expansion (HE), which occurs in up to one-third of ICH patients, is a strong predictor of poor prognosis and can be potentially preventable if high-risk patients are identified early. In this review, we provide a comprehensive summary of previous research in this area and highlight the potential use of imaging markers for future research studies.

Recent advances

Imaging markers have been developed in recent years to aid in the early detection of HE and guide clinical decision-making. These markers have been found to be effective in predicting HE in ICH patients and include specific manifestations on Computed Tomography (CT) and CT Angiography (CTA), such as the spot sign, leakage sign, spot-tail sign, island sign, satellite sign, iodine sign, blend sign, swirl sign, black hole sign, and hypodensities. The use of imaging markers holds great promise for improving the management and outcomes of ICH patients.

Conclusion

The management of ICH presents a significant challenge, and identifying high-risk patients for HE is crucial to improving outcomes. The use of imaging markers for HE prediction can aid in the rapid identification of such patients and may serve as potential targets for anti-HE therapies in the acute phase of ICH. Therefore, further research is needed to establish the reliability and validity of these markers in identifying high-risk patients and guiding appropriate treatment decisions.

Evaluation of techniques to improve a deep learning algorithm for the automatic detection of intracranial haemorrhage on CT head imaging

BACKGROUND

Deep learning (DL) algorithms are playing an increasing role in automatic medical image analysis.

PURPOSE

To evaluate the performance of a DL model for the automatic detection of intracranial haemorrhage and its subtypes on non-contrast CT (NCCT) head studies and to compare the effects of various preprocessing and model design implementations.

METHODS

The DL algorithm was trained and externally validated on open-source, multi-centre retrospective data containing radiologist-annotated NCCT head studies. The training dataset was sourced from four research institutions across Canada, the USA and Brazil. The test dataset was sourced from a research centre in India. A convolutional neural network (CNN) was used, with its performance compared against similar models with additional implementations: (1) a recurrent neural network (RNN) attached to the CNN, (2) preprocessed CT image-windowed inputs and (3) preprocessed CT image-concatenated inputs. The area under the receiver operating characteristic curve (AUC-ROC) and microaveraged precision (mAP) score were used to evaluate and compare model performances.

RESULTS

The training and test datasets contained 21,744 and 491 NCCT head studies, respectively, with 8,882 (40.8%) and 205 (41.8%) positive for intracranial haemorrhage. Implementation of preprocessing techniques and the CNN-RNN framework increased mAP from 0.77 to 0.93 and increased AUC-ROC [95% confidence intervals] from 0.854 [0.816-0.889] to 0.966 [0.951-0.980] (p-value = 3.91 × 10^-12).

CONCLUSIONS

The deep learning model accurately detected intracranial haemorrhage and improved in performance following specific implementation techniques, demonstrating clinical potential as a decision support tool and an automated system to improve radiologist workflow efficiency.

KEY POINTS

• The deep learning model detected intracranial haemorrhages on computed tomography with high accuracy. • Image preprocessing, such as windowing, plays a large role in improving deep learning model performance. • Implementations which enable an analysis of interslice dependencies can improve deep learning model performance. • Visual saliency maps can facilitate explainable artificial intelligence systems. • Deep learning within a triage system may expedite earlier intracranial haemorrhage detection.

Deep learning based automatic detection algorithm for acute intracranial haemorrhage: a pivotal randomized clinical trial

Acute intracranial haemorrhage (AIH) is a potentially life-threatening emergency that requires prompt and accurate assessment and management. This study aims to develop and validate an artificial intelligence (AI) algorithm for diagnosing AIH using brain-computed tomography (CT) images. A retrospective, multi-reader, pivotal, crossover, randomised study was performed to validate the performance of an AI algorithm was trained using 104,666 slices from 3010 patients. Brain CT images (12,663 slices from 296 patients) were evaluated by nine reviewers belonging to one of the three subgroups (non-radiologist physicians, n = 3; board-certified radiologists, n = 3; and neuroradiologists, n = 3) with and without the aid of our AI algorithm. Sensitivity, specificity, and accuracy were compared between AI-unassisted and AI-assisted interpretations using the chi-square test. Brain CT interpretation with AI assistance results in significantly higher diagnostic accuracy than that without AI assistance (0.9703 vs. 0.9471, p < 0.0001, patient-wise). Among the three subgroups of reviewers, non-radiologist physicians demonstrate the greatest improvement in diagnostic accuracy for brain CT interpretation with AI assistance compared to that without AI assistance. For board-certified radiologists, the diagnostic accuracy for brain CT interpretation is significantly higher with AI assistance than without AI assistance. For neuroradiologists, although brain CT interpretation with AI assistance results in a trend for higher diagnostic accuracy compared to that without AI assistance, the difference does not reach statistical significance. For the detection of AIH, brain CT interpretation with AI assistance results in better diagnostic performance than that without AI assistance, with the most significant improvement observed for non-radiologist physicians.

Deep Learning-Based Computed Tomography Image Standardization to Improve Generalizability of Deep Learning-Based Hepatic Segmentation

OBJECTIVE

We aimed to investigate whether image standardization using deep learning-based computed tomography (CT) image conversion would improve the performance of deep learning-based automated hepatic segmentation across various reconstruction methods.

MATERIALS AND METHODS

We collected contrast-enhanced dual-energy CT of the abdomen that was obtained using various reconstruction methods, including filtered back projection, iterative reconstruction, optimum contrast, and monoenergetic images with 40, 60, and 80 keV. A deep learning based image conversion algorithm was developed to standardize the CT images using 142 CT examinations (128 for training and 14 for tuning). A separate set of 43 CT examinations from 42 patients (mean age, 10.1 years) was used as the test data. A commercial software program (MEDIP PRO v2.0.0.0, MEDICALIP Co. Ltd.) based on 2D U-NET was used to create liver segmentation masks with liver volume. The original 80 keV images were used as the ground truth. We used the paired t-test to compare the segmentation performance in the Dice similarity coefficient (DSC) and difference ratio of the liver volume relative to the ground truth volume before and after image standardization. The concordance correlation coefficient (CCC) was used to assess the agreement between the segmented liver volume and ground-truth volume.

RESULTS

The original CT images showed variable and poor segmentation performances. The standardized images achieved significantly higher DSCs for liver segmentation than the original images (DSC [original, 5.40%-91.27%] vs. [standardized, 93.16%-96.74%], all P < 0.001). The difference ratio of liver volume also decreased significantly after image conversion (original, 9.84%-91.37% vs. standardized, 1.99%-4.41%). In all protocols, CCCs improved after image conversion (original, -0.006-0.964 vs. standardized, 0.990-0.998).

CONCLUSION

Deep learning-based CT image standardization can improve the performance of automated hepatic segmentation using CT images reconstructed using various methods. Deep learning-based CT image conversion may have the potential to improve the generalizability of the segmentation network.

Prior knowledge-based precise diagnosis of blend sign from head computed tomography

Introduction

Automated diagnosis of intracranial hemorrhage on head computed tomography (CT) plays a decisive role in clinical management. This paper presents a prior knowledge-based precise diagnosis of blend sign network from head CT scans.

Method

We employ the object detection task as an auxiliary task in addition to the classification task, which could incorporate the hemorrhage location as prior knowledge into the detection framework. The auxiliary task could help the model pay more attention to the regions with hemorrhage, which is beneficial for distinguishing the blend sign. Furthermore, we propose a self-knowledge distillation strategy to deal with inaccuracy annotations.

Results

In the experiment, we retrospectively collected 1749 anonymous non-contrast head CT scans from the First Affiliated Hospital of China Medical University. The dataset contains three categories: no intracranial hemorrhage (non-ICH), normal intracranial hemorrhage (normal ICH), and blend sign. The experimental results demonstrate that our method performs better than other methods.

Discussion

Our method has the potential to assist less-experienced head CT interpreters, reduce radiologists' workload, and improve efficiency in natural clinical settings.

Intracranial Hemorrhage Detection Using Parallel Deep Convolutional Models and Boosting Mechanism

Intracranial hemorrhage (ICH) can lead to death or disability, which requires immediate action from radiologists. Due to the heavy workload, less experienced staff, and the complexity of subtle hemorrhages, a more intelligent and automated system is necessary to detect ICH. In literature, many artificial-intelligence-based methods are proposed. However, they are less accurate for ICH detection and subtype classification. Therefore, in this paper, we present a new methodology to improve the detection and subtype classification of ICH based on two parallel paths and a boosting technique. The first path employs the architecture of ResNet101-V2 to extract potential features from windowed slices, whereas Inception-V4 captures significant spatial information in the second path. Afterwards, the detection and subtype classification of ICH is performed by the light gradient boosting machine (LGBM) using the outputs of ResNet101-V2 and Inception-V4. Thus, the combined solution, known as ResNet101-V2, Inception-V4, and LGBM (Res-Inc-LGBM), is trained and tested over the brain computed tomography (CT) scans of CQ500 and Radiological Society of North America (RSNA) datasets. The experimental results state that the proposed solution efficiently obtains 97.7% accuracy, 96.5% sensitivity, and 97.4% F1 score using the RSNA dataset. Moreover, the proposed Res-Inc-LGBM outperforms the standard benchmarks for the detection and subtype classification of ICH regarding the accuracy, sensitivity, and F1 score. The results prove the significance of the proposed solution for its real-time application.

Deep Learning Applied to Intracranial Hemorrhage Detection

Intracranial hemorrhage is a serious medical problem that requires rapid and often intensive medical care. Identifying the location and type of any hemorrhage present is a critical step in the treatment of the patient. Detection of, and diagnosis of, a hemorrhage that requires an urgent procedure is a difficult and time-consuming process for human experts. In this paper, we propose methods based on EfficientDet's deep-learning technology that can be applied to the diagnosis of hemorrhages at a patient level and which could, thus, become a decision-support system. Our proposal is two-fold. On the one hand, the proposed technique classifies slices of computed tomography scans for the presence of hemorrhage or its lack of, and evaluates whether the patient is positive in terms of hemorrhage, and achieving, in this regard, 92.7% accuracy and 0.978 ROC AUC. On the other hand, our methodology provides visual explanations of the chosen classification using the Grad-CAM methodology.

Detection of Critical Spinal Epidural Lesions on CT Using Machine Learning

BACKGROUND

Critical spinal epidural pathologies can cause paralysis or death if untreated. Although magnetic resonance imaging is the preferred modality for visualizing these pathologies, computed tomography (CT) occurs far more commonly than magnetic resonance imaging in the clinical setting.

OBJECTIVE

A machine learning model was developed to screen for critical epidural lesions on CT images at a large-scale teleradiology practice. This model has utility for both worklist prioritization of emergent studies and identifying missed findings.

MATERIALS AND METHODS

There were 153 studies with epidural lesions available for training. These lesions were segmented and used to train a machine learning model. A test data set was also created using previously missed epidural lesions. The trained model was then integrated into a teleradiology workflow for 90 days. Studies were sent to secondary manual review if the model detected an epidural lesion but none was mentioned in the clinical report.

RESULTS

The model correctly identified 50.0% of epidural lesions in the test data set with 99.0% specificity. For prospective data, the model correctly prioritized 66.7% of the 18 epidural lesions diagnosed on the initial read with 98.9% specificity. There were 2.0 studies flagged for potential missed findings per day, and 17 missed epidural lesions were found during a 90-day time period. These results suggest almost half of critical spinal epidural lesions visible on CT imaging are being missed on initial diagnosis.

CONCLUSION

A machine learning model for identifying spinal epidural hematomas and abscesses on CT can be implemented in a clinical workflow.

[Machine learning technologies in CT-based diagnostics and classification of intracranial hemorrhages]

This review discusses pooled experience of creation, implementation and effectiveness of machine learning technologies in CT-based diagnosis of intracranial hemorrhages. The authors analyzed 21 original articles between 2015 and 2022 using the following keywords: «intracranial hemorrhage», «machine learning», «deep learning», «artificial intelligence». The review contains general data on basic concepts of machine learning and also considers in more detail such aspects as technical characteristics of data sets used for creation of AI algorithms for certain type of clinical task, their possible impact on effectiveness and clinical experience.

Predicting prognosis of primary pontine hemorrhage using CT image and deep learning

Prognosis of primary pontine hemorrhage (PPH) is important for treatment planning and patient management. However, only few clinical factors were reported to have prognostic value to PPH. Here, we propose a deep learning (DL) model that mines high-dimensional prognostic information from computed tomography (CT) images and combines clinical factors for predicting individualized prognosis of PPH. We proposed a multi-task DL model to learn high-dimensional CT features of hematoma and perihematomal areas for predicting the risk of 30-day mortality, 90-day mortality and 90-day functional outcome of PPH simultaneously. We further explored the combination of the DL model and clinical factors by building a combined model. All the models were trained in a training cohort (n = 219) and tested in an independent testing cohort (n = 35). The DL model achieved area under the curve (AUC) of 0.886, 0.886, and 0.759 in predicting 30-day mortality, 90-day mortality and 90-day functional outcome of PPH in the independent testing cohort, which improved over the previously reported new PPH score and the clinical model. When combining the DL model and clinical factors, the combined model achieved improved performance (AUC = 0.920, 0.941, and 0.894), indicating that DL model mines CT information that complements clinical factors. Through DL visualization technique, we found that the internal structure of hematoma and its expansion to perihematomal regions are important for predicting the prognosis of PPH. This DL model provides an easy-to-use way for predicting individualized prognosis of PPH by mining high-dimensional information from CT images, and showed improvement over clinical factors and present methods.

Traumatic brain injury: progress and challenges in prevention, clinical care, and research

Traumatic brain injury (TBI) has the highest incidence of all common neurological disorders, and poses a substantial public health burden. TBI is increasingly documented not only as an acute condition but also as a chronic disease with long-term consequences, including an increased risk of late-onset neurodegeneration. The first Lancet Neurology Commission on TBI, published in 2017, called for a concerted effort to tackle the global health problem posed by TBI. Since then, funding agencies have supported research both in high-income countries (HICs) and in low-income and middle-income countries (LMICs). In November 2020, the World Health Assembly, the decision-making body of WHO, passed resolution WHA73.10 for global actions on epilepsy and other neurological disorders, and WHO launched the Decade for Action on Road Safety plan in 2021. New knowledge has been generated by large observational studies, including those conducted under the umbrella of the International Traumatic Brain Injury Research (InTBIR) initiative, established as a collaboration of funding agencies in 2011. InTBIR has also provided a huge stimulus to collaborative research in TBI and has facilitated participation of global partners. The return on investment has been high, but many needs of patients with TBI remain unaddressed. This update to the 2017 Commission presents advances and discusses persisting and new challenges in prevention, clinical care, and research.

In LMICs, the occurrence of TBI is driven by road traffic incidents, often involving vulnerable road users such as motorcyclists and pedestrians. In HICs, most TBI is caused by falls, particularly in older people (aged ≥65 years), who often have comorbidities. Risk factors such as frailty and alcohol misuse provide opportunities for targeted prevention actions. Little evidence exists to inform treatment of older patients, who have been commonly excluded from past clinical trials—consequently, appropriate evidence is urgently required. Although increasing age is associated with worse outcomes from TBI, age should not dictate limitations in therapy. However, patients injured by low-energy falls (who are mostly older people) are about 50% less likely to receive critical care or emergency interventions, compared with those injured by high-energy mechanisms, such as road traffic incidents.

Mild TBI, defined as a Glasgow Coma sum score of 13–15, comprises most of the TBI cases (over 90%) presenting to hospital. Around 50% of adult patients with mild TBI presenting to hospital do not recover to pre-TBI levels of health by 6 months after their injury. Fewer than 10% of patients discharged after presenting to an emergency department for TBI in Europe currently receive follow-up. Structured follow-up after mild TBI should be considered good practice, and urgent research is needed to identify which patients with mild TBI are at risk for incomplete recovery.

The selection of patients for CT is an important triage decision in mild TBI since it allows early identification of lesions that can trigger hospital admission or life-saving surgery. Current decision making for deciding on CT is inefficient, with 90–95% of scanned patients showing no intracranial injury but being subjected to radiation risks. InTBIR studies have shown that measurement of blood-based biomarkers adds value to previously proposed clinical decision rules, holding the potential to improve efficiency while reducing radiation exposure. Increased concentrations of biomarkers in the blood of patients with a normal presentation CT scan suggest structural brain damage, which is seen on MR scanning in up to 30% of patients with mild TBI. Advanced MRI, including diffusion tensor imaging and volumetric analyses, can identify additional injuries not detectable by visual inspection of standard clinical MR images. Thus, the absence of CT abnormalities does not exclude structural damage—an observation relevant to litigation procedures, to management of mild TBI, and when CT scans are insufficient to explain the severity of the clinical condition.

Although blood-based protein biomarkers have been shown to have important roles in the evaluation of TBI, most available assays are for research use only. To date, there is only one vendor of such assays with regulatory clearance in Europe and the USA with an indication to rule out the need for CT imaging for patients with suspected TBI. Regulatory clearance is provided for a combination of biomarkers, although evidence is accumulating that a single biomarker can perform as well as a combination. Additional biomarkers and more clinical-use platforms are on the horizon, but cross-platform harmonisation of results is needed. Health-care efficiency would benefit from diversity in providers.

In the intensive care setting, automated analysis of blood pressure and intracranial pressure with calculation of derived parameters can help individualise management of TBI. Interest in the identification of subgroups of patients who might benefit more from some specific therapeutic approaches than others represents a welcome shift towards precision medicine. Comparative-effectiveness research to identify best practice has delivered on expectations for providing evidence in support of best practices, both in adult and paediatric patients with TBI.

Progress has also been made in improving outcome assessment after TBI. Key instruments have been translated into up to 20 languages and linguistically validated, and are now internationally available for clinical and research use. TBI affects multiple domains of functioning, and outcomes are affected by personal characteristics and life-course events, consistent with a multifactorial bio-psycho-socio-ecological model of TBI, as presented in the US National Academies of Sciences, Engineering, and Medicine (NASEM) 2022 report. Multidimensional assessment is desirable and might be best based on measurement of global functional impairment. More work is required to develop and implement recommendations for multidimensional assessment. Prediction of outcome is relevant to patients and their families, and can facilitate the benchmarking of quality of care. InTBIR studies have identified new building blocks (eg, blood biomarkers and quantitative CT analysis) to refine existing prognostic models. Further improvement in prognostication could come from MRI, genetics, and the integration of dynamic changes in patient status after presentation.

Neurotrauma researchers traditionally seek translation of their research findings through publications, clinical guidelines, and industry collaborations. However, to effectively impact clinical care and outcome, interactions are also needed with research funders, regulators, and policy makers, and partnership with patient organisations. Such interactions are increasingly taking place, with exemplars including interactions with the All Party Parliamentary Group on Acquired Brain Injury in the UK, the production of the NASEM report in the USA, and interactions with the US Food and Drug Administration. More interactions should be encouraged, and future discussions with regulators should include debates around consent from patients with acute mental incapacity and data sharing. Data sharing is strongly advocated by funding agencies. From January 2023, the US National Institutes of Health will require upload of research data into public repositories, but the EU requires data controllers to safeguard data security and privacy regulation. The tension between open data-sharing and adherence to privacy regulation could be resolved by cross-dataset analyses on federated platforms, with the data remaining at their original safe location. Tools already exist for conventional statistical analyses on federated platforms, however federated machine learning requires further development. Support for further development of federated platforms, and neuroinformatics more generally, should be a priority.

This update to the 2017 Commission presents new insights and challenges across a range of topics around TBI: epidemiology and prevention (section 1 ); system of care (section 2 ); clinical management (section 3 ); characterisation of TBI (section 4 ); outcome assessment (section 5 ); prognosis (Section 6 ); and new directions for acquiring and implementing evidence (section 7 ). Table 1 summarises key messages from this Commission and proposes recommendations for the way forward to advance research and clinical management of TBI.

Contrastive self-supervised learning from 100 million medical images with optional supervision

Purpose

Building accurate and robust artificial intelligence systems for medical image assessment requires the creation of large sets of annotated training examples. However, constructing such datasets is very costly due to the complex nature of annotation tasks, which often require expert knowledge (e.g., a radiologist). To counter this limitation, we propose a method to learn from medical images at scale in a self-supervised way.

Approach

Our approach, based on contrastive learning and online feature clustering, leverages training datasets of over 100,000,000 medical images of various modalities, including radiography, computed tomography (CT), magnetic resonance (MR) imaging, and ultrasonography (US). We propose to use the learned features to guide model training in supervised and hybrid self-supervised/supervised regime on various downstream tasks.

Results

We highlight a number of advantages of this strategy on challenging image assessment problems in radiography, CT, and MR: (1) significant increase in accuracy compared to the state-of-the-art (e.g., area under the curve boost of 3% to 7% for detection of abnormalities from chest radiography scans and hemorrhage detection on brain CT); (2) acceleration of model convergence during training by up to 85% compared with using no pretraining (e.g., 83% when training a model for detection of brain metastases in MR scans); and (3) increase in robustness to various image augmentations, such as intensity variations, rotations or scaling reflective of data variation seen in the field.

Conclusions

The proposed approach enables large gains in accuracy and robustness on challenging image assessment problems. The improvement is significant compared with other state-of-the-art approaches trained on medical or vision images (e.g., ImageNet).

Diagnose Like a Radiologist: Hybrid Neuro-Probabilistic Reasoning for Attribute-Based Medical Image Diagnosis

During clinical practice, radiologists often use attributes, e.g., morphological and appearance characteristics of a lesion, to aid disease diagnosis. Effectively modeling attributes as well as all relationships involving attributes could boost the generalization ability and verifiability of medical image diagnosis algorithms. In this paper, we introduce a hybrid neuro-probabilistic reasoning algorithm for verifiable attribute-based medical image diagnosis. There are two parallel branches in our hybrid algorithm, a Bayesian network branch performing probabilistic causal relationship reasoning and a graph convolutional network branch performing more generic relational modeling and reasoning using a feature representation. Tight coupling between these two branches is achieved via a cross-network attention mechanism and the fusion of their classification results. We have successfully applied our hybrid reasoning algorithm to two challenging medical image diagnosis tasks. On the LIDC-IDRI benchmark dataset for benign-malignant classification of pulmonary nodules in CT images, our method achieves a new state-of-the-art accuracy of 95.36% and an AUC of 96.54%. Our method also achieves a 3.24% accuracy improvement on an in-house chest X-ray image dataset for tuberculosis diagnosis. Our ablation study indicates that our hybrid algorithm achieves a much better generalization performance than a pure neural network architecture under very limited training data.

Automatic detection of stroke lesion from diffusion-weighted imaging via the improved YOLOv5

BACKGROUND AND OBJECTIVE

Stroke is the second most deadly disease globally and seriously endangers people's lives and health. The automatic detection of stroke lesions from diffusion-weighted imaging (DWI) can improve the diagnosis. Recently, automatic detection methods based on YOLOv5 have been utilized in medical images. However, most of them barely capture the stroke lesions because of their small size and fuzzy boundaries.

METHODS

To address this problem, a novel method for tracing the edge of the stroke lesion based on YOLOv5 (TE-YOLOv5) is proposed. Specifically, we constantly update the high-level features of the lesion using an aggregate pool (AP) module. Conversely, we feed the extracted feature into the reverse attention (RA) module to trace the edge relationship promptly. Overall, 1681 DWI images of 319 stroke patients have been collected, and experienced radiologists have marked the lesions. DWI images were randomly split into the training and test set at a ratio of 8:2. TE-YOLOv5 has been compared with the related models, and a detailed ablation analysis has been conducted to clarify the role of the RA and AP modules.

RESULTS

TE-YOLOv5 outperforms its counterparts and achieves competitive performance with a precision of 81.5%, a recall of 75.8%, and a mAP@0.5 of 80.7% (mean average precision while the intersection over union is 0.5) under the same backbone. At the patient level, the positive finding rate can reach 98.51%, while the confidence is set at 80.0%. After ablating RA, the mAP@0.5 decreases to 79.6%; after ablating RA and AP, the mAP@0.5 decreases to 78.1%.

CONCLUSIONS

The proposed TE-YOLOv5 can automatically and effectively detect stroke lesions from DWI images, especially for those with an extremely small size and blurred boundaries. AP and RA modules can aggregate multi-layer high-level features and concurrently track the edge relationship of stroke lesions. These detection methods might help radiologists improve stroke diagnosis and have great application potential in clinical practice.

Deep learning-based computed tomography image segmentation and volume measurement of intracerebral hemorrhage

The study aims to enhance the accuracy and practicability of CT image segmentation and volume measurement of ICH by using deep learning technology. A dataset including the brain CT images and clinical data of 1,027 patients with spontaneous ICHs treated from January 2010 to December 2020 were retrospectively analyzed, and a deep segmentation network (AttFocusNet) integrating the focus structure and the attention gate (AG) mechanism is proposed to enable automatic, accurate CT image segmentation and volume measurement of ICHs. In internal validation set, experimental results showed that AttFocusNet achieved a Dice coefficient of 0.908, an intersection-over-union (IoU) of 0.874, a sensitivity of 0.913, a positive predictive value (PPV) of 0.957, and a 95% Hausdorff distance (HD95) (mm) of 5.960. The intraclass correlation coefficient (ICC) of the ICH volume measurement between AttFocusNet and the ground truth was 0.997. The average time of per case achieved by AttFocusNet, Coniglobus formula and manual segmentation is 5.6, 47.7, and 170.1 s. In the two external validation sets, AttFocusNet achieved a Dice coefficient of 0.889 and 0.911, respectively, an IoU of 0.800 and 0.836, respectively, a sensitivity of 0.817 and 0.849, respectively, a PPV of 0.976 and 0.981, respectively, and a HD95 of 5.331 and 4.220, respectively. The ICC of the ICH volume measurement between AttFocusNet and the ground truth were 0.939 and 0.956, respectively. The proposed segmentation network AttFocusNet significantly outperforms the Coniglobus formula in terms of ICH segmentation and volume measurement by acquiring measurement results closer to the true ICH volume and significantly reducing the clinical workload.

Accuracy of artificial intelligence for the detection of intracranial hemorrhage and chronic cerebral microbleeds: a systematic review and pooled analysis

BACKGROUND

Artificial intelligence (AI)-driven software has been developed and become commercially available within the past few years for the detection of intracranial hemorrhage (ICH) and chronic cerebral microbleeds (CMBs). However, there is currently no systematic review that summarizes all of these tools or provides pooled estimates of their performance.

METHODS

In this PROSPERO-registered, PRISMA compliant systematic review, we sought to compile and review all MEDLINE and EMBASE published studies that have developed and/or tested AI algorithms for ICH detection on non-contrast CT scans (NCCTs) or MRI scans and CMBs detection on MRI scans.

RESULTS

In total, 40 studies described AI algorithms for ICH detection in NCCTs/MRIs and 19 for CMBs detection in MRIs. The overall sensitivity, specificity, and accuracy were 92.06%, 93.54%, and 93.46%, respectively, for ICH detection and 91.6%, 93.9%, and 92.7% for CMBs detection. Some of the challenges encountered in the development of these algorithms include the laborious work of creating large, labeled and balanced datasets, the volumetric nature of the imaging examinations, the fine tuning of the algorithms, and the reduction in false positives.

CONCLUSIONS

Numerous AI-driven software tools have been developed over the last decade. On average, they are characterized by high performance and expert-level accuracy for the diagnosis of ICH and CMBs. As a result, implementing these tools in clinical practice may improve workflow and act as a failsafe for the detection of such lesions. REGISTRATION-URL: https://www.crd.york.ac.uk/prospero/ Unique Identifier: CRD42021246848.

The 2021 landscape of FDA-approved artificial intelligence/machine learning-enabled medical devices: An analysis of the characteristics and intended use

BACKGROUND

Machine learning (ML), a type of artificial intelligence (AI) technology that uses a data-driven approach for pattern recognition, has been shown to be beneficial for many tasks across healthcare. To characterize the commercial availability of AI/ML applications in the clinic, we performed a detailed analysis of AI/ML-enabled medical devices approved/cleared by the US Food and Drug Administration (FDA) by June 2021.

METHODS/MATERIALS

The publicly available approval letters by the FDA on 343 AI/ML-enabled medical devices compiled by the agency were reviewed. The characteristics of the devices and the patterns of their intended use were analyzed, and basic descriptive statistical analysis was performed on the aggregated data.

RESULTS

Most devices were reviewed by radiology (70.3%) and cardiovascular (12.0%) medical specialty panels. The growth of these devices sharply rose since the mid-2010s. Most (95.0%) devices were cleared under the 510(k) premarket notification pathway, and 69.4% were software as a medical device (SaMD). Of the 241 radiology-related devices, the most common applications were for diagnostic assistance (48.5%) and image reconstruction (14.1%). Of the 117 radiology-related devices for diagnostic assistance, 20.5% were developed for breast lesion assessment and 14.5% for cardiac function assessment on echocardiogram. Of the 41 cardiology-related devices, the most common applications were electrocardiography-based arrhythmia detection (46.3%) and hemodynamics & vital signs monitoring (26.8%).

CONCLUSION

In this study, we characterized the patterns and trends of AI/ML-enabled medical devices approved or cleared by the FDA. To our knowledge, this is the most up-to-date and comprehensive analysis of the landscape as of 2021.

Intracerebral hemorrhage detection on computed tomography images using a residual neural network

Intracerebral hemorrhage (ICH) is a high mortality rate, critical medical injury, produced by the rupture of a blood vessel of the vascular system inside the skull. ICH can lead to paralysis and even death. Therefore, it is considered a clinically dangerous disease that needs to be treated quickly. Thanks to the advancement in machine learning and the computing power of today's microprocessors, deep learning has become an unbelievably valuable tool for detecting diseases, in particular from medical images. In this work, we are interested in differentiating computer tomography (CT) images of healthy brains and ICH using a ResNet-18, a deep residual convolutional neural network. In addition, the gradient-weighted class activation mapping (Grad-CAM) technique was employed to visually explore and understand the network's decisions. The generalizability of the detector was assessed through a 100-iteration Monte Carlo cross-validation (80% of the data for training and 20% for test). In a database with 200 CT images of brains (100 with ICH and 100 without ICH), the detector yielded, on average, 95.93%accuracy, 96.20% specificity, 95.65% sensitivity, 96.40% precision, and 95.91% F1-core, with an average computing time of 165.90 s to train the network (on 160 images) and 1.17 s to test it with 40 CT images. These results are comparable with the state of the art with a simpler and lower computational load approach. Our detector could assist physicians in their medical decision, in resource optimization and in reducing the time and error in the diagnosis of ICH.

Automatic CT Angiography Lesion Segmentation Compared to CT Perfusion in Ischemic Stroke Detection: a Feasibility Study

In stroke imaging, CT angiography (CTA) is used for detecting arterial occlusions. These images could also provide information on the extent of ischemia. The study aim was to develop and evaluate a convolutional neural network (CNN)-based algorithm for detecting and segmenting acute ischemic lesions from CTA images of patients with suspected middle cerebral artery stroke. These results were compared to volumes reported by widely used CT perfusion-based RAPID software (IschemaView). A 42-layer-deep CNN was trained on 50 CTA volumes with manually delineated targets. The lower bound for predicted lesion size to reliably discern stroke from false positives was estimated. The severity of false positives and false negatives was reviewed visually to assess the clinical applicability and to further guide the method development. The CNN model corresponded to the manual segmentations with voxel-wise sensitivity 0.54 (95% confidence interval: 0.44-0.63), precision 0.69 (0.60-0.76), and Sørensen-Dice coefficient 0.61 (0.52-0.67). Stroke/nonstroke differentiation accuracy 0.88 (0.81-0.94) was achieved when only considering the predicted lesion size (i.e., regardless of location). By visual estimation, 46% of cases showed some false findings, such as CNN highlighting chronic periventricular white matter changes or beam hardening artifacts, but only in 9% the errors were severe, translating to 0.91 accuracy. The CNN model had a moderately strong correlation to RAPID-reported T_max > 10 s volumes (Pearson's r = 0.76 (0.58-0.86)). The results suggest that detecting anterior circulation ischemic strokes from CTA using a CNN-based algorithm can be feasible when accompanied with physiological knowledge to rule out false positives.

Measures of Intracranial Injury Size Do Not Improve Clinical Decision Making for Children With Mild Traumatic Brain Injuries and Intracranial Injuries

BACKGROUND

When evaluating children with mild traumatic brain injuries (mTBIs) and intracranial injuries (ICIs), neurosurgeons intuitively consider injury size. However, the extent to which such measures (eg, hematoma size) improve risk prediction compared with the kids intracranial injury decision support tool for traumatic brain injury (KIIDS-TBI) model, which only includes the presence/absence of imaging findings, remains unknown.

OBJECTIVE

To determine the extent to which measures of injury size improve risk prediction for children with mild traumatic brain injuries and ICIs.

METHODS

We included children ≤18 years who presented to 1 of the 5 centers within 24 hours of TBI, had Glasgow Coma Scale scores of 13 to 15, and had ICI on neuroimaging. The data set was split into training (n = 1126) and testing (n = 374) cohorts. We used generalized linear modeling (GLM) and recursive partitioning (RP) to predict the composite of neurosurgery, intubation >24 hours, or death because of TBI. Each model's sensitivity/specificity was compared with the validated KIIDS-TBI model across 3 decision-making risk cutoffs (<1%, <3%, and <5% predicted risk).

RESULTS

The GLM and RP models included similar imaging variables (eg, epidural hematoma size) while the GLM model incorporated additional clinical predictors (eg, Glasgow Coma Scale score). The GLM (76%-90%) and RP (79%-87%) models showed similar specificity across all risk cutoffs, but the GLM model had higher sensitivity (89%-96% for GLM; 89% for RP). By comparison, the KIIDS-TBI model had slightly higher sensitivity (93%-100%) but lower specificity (27%-82%).

CONCLUSION

Although measures of ICI size have clear intuitive value, the tradeoff between higher specificity and lower sensitivity does not support the addition of such information to the KIIDS-TBI model.

Artificial Intelligence with Statistical Confidence Scores for Detection of Acute or Subacute Hemorrhage on Noncontrast CT Head Scans

Purpose

To present a method that automatically detects, subtypes, and locates acute or subacute intracranial hemorrhage (ICH) on noncontrast CT (NCCT) head scans; generates detection confidence scores to identify high-confidence data subsets with higher accuracy; and improves radiology worklist prioritization. Such scores may enable clinicians to better use artificial intelligence (AI) tools.

Materials and Methods

This retrospective study included 46 057 studies from seven "internal" centers for development (training, architecture selection, hyperparameter tuning, and operating-point calibration; n = 25 946) and evaluation (n = 2947) and three "external" centers for calibration (n = 400) and evaluation (n = 16764). Internal centers contributed developmental data, whereas external centers did not. Deep neural networks predicted the presence of ICH and subtypes (intraparenchymal, intraventricular, subarachnoid, subdural, and/or epidural hemorrhage) and segmentations per case. Two ICH confidence scores are discussed: a calibrated classifier entropy score and a Dempster-Shafer score. Evaluation was completed by using receiver operating characteristic curve analysis and report turnaround time (RTAT) modeling on the evaluation set and on confidence score-defined subsets using bootstrapping.

Results

The areas under the receiver operating characteristic curve for ICH were 0.97 (0.97, 0.98) and 0.95 (0.94, 0.95) on internal and external center data, respectively. On 80% of the data stratified by calibrated classifier and Dempster-Shafer scores, the system improved the Youden indexes, increasing them from 0.84 to 0.93 (calibrated classifier) and from 0.84 to 0.92 (Dempster-Shafer) for internal centers and increasing them from 0.78 to 0.88 (calibrated classifier) and from 0.78 to 0.89 (Dempster-Shafer) for external centers (P < .001). Models estimated shorter RTAT for AI-prioritized worklists with confidence measures than for AI-prioritized worklists without confidence measures, shortening RTAT by 27% (calibrated classifier) and 27% (Dempster-Shafer) for internal centers and shortening RTAT by 25% (calibrated classifier) and 27% (Dempster-Shafer) for external centers (P < .001).

Conclusion

AI that provided statistical confidence measures for ICH detection on NCCT scans reliably detected and subtyped hemorrhages, identified high-confidence predictions, and improved worklist prioritization in simulation.Keywords: CT, Head/Neck, Hemorrhage, Convolutional Neural Network (CNN) Supplemental material is available for this article. © RSNA, 2022.

External Validation of Deep Learning Algorithms for Radiologic Diagnosis: A Systematic Review

Purpose

To assess generalizability of published deep learning (DL) algorithms for radiologic diagnosis.

Materials and Methods

In this systematic review, the PubMed database was searched for peer-reviewed studies of DL algorithms for image-based radiologic diagnosis that included external validation, published from January 1, 2015, through April 1, 2021. Studies using nonimaging features or incorporating non-DL methods for feature extraction or classification were excluded. Two reviewers independently evaluated studies for inclusion, and any discrepancies were resolved by consensus. Internal and external performance measures and pertinent study characteristics were extracted, and relationships among these data were examined using nonparametric statistics.

Results

Eighty-three studies reporting 86 algorithms were included. The vast majority (70 of 86, 81%) reported at least some decrease in external performance compared with internal performance, with nearly half (42 of 86, 49%) reporting at least a modest decrease (≥0.05 on the unit scale) and nearly a quarter (21 of 86, 24%) reporting a substantial decrease (≥0.10 on the unit scale). No study characteristics were found to be associated with the difference between internal and external performance.

Conclusion

Among published external validation studies of DL algorithms for image-based radiologic diagnosis, the vast majority demonstrated diminished algorithm performance on the external dataset, with some reporting a substantial performance decrease.Keywords: Meta-Analysis, Computer Applications-Detection/Diagnosis, Neural Networks, Computer Applications-General (Informatics), Epidemiology, Technology Assessment, Diagnosis, Informatics Supplemental material is available for this article. © RSNA, 2022.