CT grading scales for splenic injury: why can't we agree?

An ASER AI/ML expert panel formative user research study for an interpretable interactive splenic AAST grading graphical user interface prototype

PURPOSE

The AAST Organ Injury Scale is widely adopted for splenic injury severity but suffers from only moderate inter-rater agreement. This work assesses SpleenPro, a prototype interactive explainable artificial intelligence/machine learning (AI/ML) diagnostic aid to support AAST grading, for effects on radiologist dwell time, agreement, clinical utility, and user acceptance.

METHODS

Two trauma radiology ad hoc expert panelists independently performed timed AAST grading on 76 admission CT studies with blunt splenic injury, first without AI/ML assistance, and after a 2-month washout period and randomization, with AI/ML assistance. To evaluate user acceptance, three versions of the SpleenPro user interface with increasing explainability were presented to four independent expert panelists with four example cases each. A structured interview consisting of Likert scales and free responses was conducted, with specific questions regarding dimensions of diagnostic utility (DU); mental support (MS); effort, workload, and frustration (EWF); trust and reliability (TR); and likelihood of future use (LFU).

RESULTS

SpleenPro significantly decreased interpretation times for both raters. Weighted Cohen's kappa increased from 0.53 to 0.70 with AI/ML assistance. During user acceptance interviews, increasing explainability was associated with improvement in Likert scores for MS, EWF, TR, and LFU. Expert panelists indicated the need for a combined early notification and grading functionality, PACS integration, and report autopopulation to improve DU.

CONCLUSIONS

SpleenPro was useful for improving objectivity of AAST grading and increasing mental support. Formative user research identified generalizable concepts including the need for a combined detection and grading pipeline and integration with the clinical workflow.

Interrater Agreement of CT Grading of Blunt Splenic Injuries: Does the AAST Grading Need to Be Reimagined?

Introduction: The Revised Organ Injury Scale (OIS) of the American Association for Surgery of Trauma (AAST) is the most widely accepted classification of splenic trauma. The objective of this study was to evaluate inter-rater agreement for CT grading of blunt splenic injuries. Methods: CT scans in adult patients with splenic injuries at a level 1 trauma centre were independently graded by 5 fellowship trained abdominal radiologists using the AAST OIS for splenic injuries - 2018 revision. The inter-rater agreement for AAST CT injury score, as well as low-grade (IIII) versus high-grade (IV-V) splenic injury was assessed. Disagreement in two key clinical scenarios (no injury versus injury, and high versus low grade) were qualitatively reviewed to identify possible sources of disagreement. Results: A total of 610 examinations were included. The inter-rater absolute agreement was low (Fleiss kappa statistic 0.38, P < 0.001), but improved when comparing agreement between low and high grade injuries (Fleiss kappa statistic of 0.77, P < .001). There were 34 cases (5.6%) of minimum two-rater disagreement about no injury vs injury (AAST grade ≥ I). There were 46 cases (7.5%) of minimum two-rater disagreement of low grade (AAST grade I-III) versus high grade (AAST grade IV-V) injuries. Likely sources of disagreement were interpretation of clefts versus lacerations, peri-splenic fluid versus subcapsular hematoma, application of adding multiple low grade injuries to higher grade injuries, and identification of subtle vascular injuries. Conclusion: There is low absolute agreement in grading of splenic injuries using the existing AAST OIS for splenic injuries.

Machine Learning Detection and Characterization of Splenic Injuries on Abdominal Computed Tomography

BACKGROUND

Multi-detector contrast-enhanced abdominal computed tomography (CT) allows for the accurate detection and classification of traumatic splenic injuries, leading to improved patient management. Their effective use requires rapid study interpretation, which can be a challenge on busy emergency radiology services. A machine learning system has the potential to automate the process, potentially leading to a faster clinical response. This study aimed to create such a system.

METHOD

Using the American Association for the Surgery of Trauma (AAST), spleen injuries were classified into 3 classes: normal, low-grade (AAST grade I-III) injuries, and high-grade (AAST grade IV and V) injuries. Employing a 2-stage machine learning strategy, spleens were initially segmented from input CT images and subsequently underwent classification via a 3D dense convolutional neural network (DenseNet).

RESULTS

This single-centre retrospective study involved trauma protocol CT scans performed between January 1, 2005, and July 31, 2021, totaling 608 scans with splenic injuries and 608 without. Five board-certified fellowship-trained abdominal radiologists utilizing the AAST injury scoring scale established ground truth labels. The model achieved AUC values of 0.84, 0.69, and 0.90 for normal, low-grade injuries, and high-grade splenic injuries, respectively.

CONCLUSIONS

Our findings demonstrate the feasibility of automating spleen injury detection using our method with potential applications in improving patient care through radiologist worklist prioritization and injury stratification. Future endeavours should concentrate on further enhancing and optimizing our approach and testing its use in a real-world clinical environment.

Artificial intelligence CAD tools in trauma imaging: a scoping review from the American Society of Emergency Radiology (ASER) AI/ML Expert Panel

BACKGROUND

AI/ML CAD tools can potentially improve outcomes in the high-stakes, high-volume model of trauma radiology. No prior scoping review has been undertaken to comprehensively assess tools in this subspecialty.

PURPOSE

To map the evolution and current state of trauma radiology CAD tools along key dimensions of technology readiness.

METHODS

Following a search of databases, abstract screening, and full-text document review, CAD tool maturity was charted using elements of data curation, performance validation, outcomes research, explainability, user acceptance, and funding patterns. Descriptive statistics were used to illustrate key trends.

RESULTS

A total of 4052 records were screened, and 233 full-text articles were selected for content analysis. Twenty-one papers described FDA-approved commercial tools, and 212 reported algorithm prototypes. Works ranged from foundational research to multi-reader multi-case trials with heterogeneous external data. Scalable convolutional neural network-based implementations increased steeply after 2016 and were used in all commercial products; however, options for explainability were narrow. Of FDA-approved tools, 9/10 performed detection tasks. Dataset sizes ranged from < 100 to > 500,000 patients, and commercialization coincided with public dataset availability. Cross-sectional torso datasets were uniformly small. Data curation methods with ground truth labeling by independent readers were uncommon. No papers assessed user acceptance, and no method included human-computer interaction. The USA and China had the highest research output and frequency of research funding.

CONCLUSIONS

Trauma imaging CAD tools are likely to improve patient care but are currently in an early stage of maturity, with few FDA-approved products for a limited number of uses. The scarcity of high-quality annotated data remains a major barrier.

Toward automated interpretable AAST grading for blunt splenic injury

BACKGROUND

The American Association for the Surgery of Trauma (AAST) splenic organ injury scale (OIS) is the most frequently used CT-based grading system for blunt splenic trauma. However, reported inter-rater agreement is modest, and an algorithm that objectively automates grading based on transparent and verifiable criteria could serve as a high-trust diagnostic aid.

PURPOSE

To pilot the development of an automated interpretable multi-stage deep learning-based system to predict AAST grade from admission trauma CT.

METHODS

Our pipeline includes 4 parts: (1) automated splenic localization, (2) Faster R-CNN-based detection of pseudoaneurysms (PSA) and active bleeds (AB), (3) nnU-Net segmentation and quantification of splenic parenchymal disruption (SPD), and (4) a directed graph that infers AAST grades from detection and segmentation results. Training and validation is performed on a dataset of adult patients (age ≥ 18) with voxelwise labeling, consensus AAST grading, and hemorrhage-related outcome data (n = 174).

RESULTS

AAST classification agreement (weighted κ) between automated and consensus AAST grades was substantial (0.79). High-grade (IV and V) injuries were predicted with accuracy, positive predictive value, and negative predictive value of 92%, 95%, and 89%. The area under the curve for predicting hemorrhage control intervention was comparable between expert consensus and automated AAST grading (0.83 vs 0.88). The mean combined inference time for the pipeline was 96.9 s.

CONCLUSIONS

The results of our method were rapid and verifiable, with high agreement between automated and expert consensus grades. Diagnosis of high-grade lesions and prediction of hemorrhage control intervention produced accurate results in adult patients.

Liver trauma: WSES 2020 guidelines

Liver injuries represent one of the most frequent life-threatening injuries in trauma patients. In determining the optimal management strategy, the anatomic injury, the hemodynamic status, and the associated injuries should be taken into consideration. Liver trauma approach may require non-operative or operative management with the intent to restore the homeostasis and the normal physiology. The management of liver trauma should be multidisciplinary including trauma surgeons, interventional radiologists, and emergency and ICU physicians. The aim of this paper is to present the World Society of Emergency Surgery (WSES) liver trauma management guidelines.

Splenic trauma: WSES classification and guidelines for adult and pediatric patients

Spleen injuries are among the most frequent trauma-related injuries. At present, they are classified according to the anatomy of the injury. The optimal treatment strategy, however, should keep into consideration the hemodynamic status, the anatomic derangement, and the associated injuries. The management of splenic trauma patients aims to restore the homeostasis and the normal physiopathology especially considering the modern tools for bleeding management. Thus, the management of splenic trauma should be ultimately multidisciplinary and based on the physiology of the patient, the anatomy of the injury, and the associated lesions. Lastly, as the management of adults and children must be different, children should always be treated in dedicated pediatric trauma centers. In fact, the vast majority of pediatric patients with blunt splenic trauma can be managed non-operatively. This paper presents the World Society of Emergency Surgery (WSES) classification of splenic trauma and the management guidelines.