Performance of AI to exclude normal chest radiographs to reduce radiologists' workload

INTRODUCTION

This study investigates the performance of a commercially available artificial intelligence (AI) system to identify normal chest radiographs and its potential to reduce radiologist workload.

METHODS

Retrospective analysis included consecutive chest radiographs from two medical centers between Oct 1, 2016 and Oct 14, 2016. Exclusions comprised follow-up exams within the inclusion period, bedside radiographs, incomplete images, imported radiographs, and pediatric radiographs. Three chest radiologists categorized findings into normal, clinically irrelevant, clinically relevant, urgent, and critical. A commercial AI system processed all radiographs, scoring 10 chest abnormalities on a 0-100 confidence scale. AI system performance was evaluated using the area under the ROC curve (AUC), assessing the detection of normal radiographs. Sensitivity was calculated for the default and a conservative operating point. the detection of negative predictive value (NPV) for urgent and critical findings, as well as the potential workload reduction, was calculated.

RESULTS

A total of 2603 radiographs were acquired in 2141 unique patients. Post-exclusion, 1670 radiographs were analyzed. Categories included 479 normal, 332 clinically irrelevant, 339 clinically relevant, 501 urgent, and 19 critical findings. The AI system achieved an AUC of 0.92. Sensitivity for normal radiographs was 92% at default and 53% at the conservative operating point. At the conservative operating point, NPV was 98% for urgent and critical findings, and could result in a 15% workload reduction.

CONCLUSION

A commercially available AI system effectively identifies normal chest radiographs and holds the potential to lessen radiologists' workload by omitting half of the normal exams from reporting.

CLINICAL RELEVANCE STATEMENT

The AI system is able to detect half of all normal chest radiographs at a clinically acceptable operating point, thereby potentially reducing the workload for the radiologists by 15%.

KEY POINTS

The AI system reached an AUC of 0.92 for the detection of normal chest radiographs. Fifty-three percent of normal chest radiographs were identified with a NPV of 98% for urgent findings. AI can reduce the workload of chest radiography reporting by 15%.

Students' and junior doctors' perspectives on radiology education in medical school: a qualitative study in the Netherlands

BACKGROUND

Modern medicine becomes more dependent on radiologic imaging techniques. Over the past decade, radiology has also gained more attention in the medical curricula. However, little is known with regard to students' perspectives on this subject. Therefore, this study aims to gain insight into the thoughts and ideas of medical students and junior doctors on radiology education in medical curricula.

METHODS

A qualitative, descriptive study was carried out at one medical university in the Netherlands. Participants were recruited on social media and were interviewed following a predefined topic list. The constant comparative method was applied in order to include new questions when unexpected topics arose during the interviews. All interviews were transcribed verbatim and coded. Codes were organized into categories and themes by discussion between researchers.

RESULTS

Fifteen participants (nine junior doctors and six students) agreed to join. From the coded interviews, four themes derived from fifteen categories arose: (1) The added value of radiology education in medical curricula, (2) Indispensable knowledge on radiology, (3) Organization of radiology education and (4) Promising educational innovations for the radiology curriculum.

CONCLUSION

This study suggests that medical students and junior doctors value radiology education. It provides insights in educational topics and forms for educational improvement for radiology educators.

Multireader image quality evaluation of dynamic myocardial computed tomography perfusion imaging with a novel four-dimensional noise reduction filter

BACKGROUND

Dynamic myocardial computed tomography perfusion (CTP) is a novel technique able to depict cardiac ischemia.

PURPOSE

To evaluate the impact of a four-dimensional noise reduction filter (similarity filter [4D-SF]) on image quality in dynamic CTP imaging, allowing for substantial radiation dose reduction.

MATERIAL AND METHODS

Dynamic CTP datasets of 30 patients (16 women) with suspected coronary artery disease, acquired with a 320-slice CT system, were retrieved, reconstructed with the deep learning-based algorithm of the system (DLR), and filtered with the 4D-SF. For each case, signal-to-noise ratio (SNR) and contrast-to-noise ratio (CNR) in six regions of interest (33-38mm²) were calculated before and after filtering, in four-chamber and short-axis views, and t-tested. Furthermore, six radiologists of different expertise evaluated subjective image preference by answering five visual grading analysis-type questions (regarding acceptable level of noise, absence of artifacts, natural appearance, cardiac contour sharpness, diagnostic acceptability) using a 5-point scale. The results were analyzed using visual grade characteristics (VGC) and intraclass correlation coefficient (ICC).

RESULTS

Mean SNR in four-chamber view (unfiltered vs. filtered) were: septum=4.1 ± 2.1 versus 7.6 ± 5.6; lateral wall=4.5 ± 2.0 versus 8.0 ± 4.9; CNRseptum=16.6 ± 8.9 versus 31.7 ± 28; lateral wall=16.2 ± 8.9 versus 31.3 ± 28.9. Similar results were obtained in short-axis view. The perceived filtered image quality indicated decreased noise (VGC_AUC=0.96) and artifacts (0.65), improved natural appearance (0.59), cardiac contour sharpness (0.74), and diagnostic acceptability (0.78). The inter-observer variability was excellent (ICC=0.79). All results were statistically significant (P < 0.05).

CONCLUSION

Similarity filtering after DLR improves image quality, possibly enabling dose reduction in dynamic CTP imaging in patient with suspected chronic coronary syndrome.

Malignancy risk estimation of pulmonary nodules in screening CTs: Comparison between a computer model and human observers

Purpose

To compare human observers to a mathematically derived computer model for differentiation between malignant and benign pulmonary nodules detected on baseline screening computed tomography (CT) scans.

Methods

A case-cohort study design was chosen. The study group consisted of 300 chest CT scans from the Danish Lung Cancer Screening Trial (DLCST). It included all scans with proven malignancies (n = 62) and two subsets of randomly selected baseline scans with benign nodules of all sizes (n = 120) and matched in size to the cancers, respectively (n = 118). Eleven observers and the computer model (PanCan) assigned a malignancy probability score to each nodule. Performances were expressed by area under the ROC curve (AUC). Performance differences were tested using the Dorfman, Berbaum and Metz method. Seven observers assessed morphological nodule characteristics using a predefined list. Differences in morphological features between malignant and size-matched benign nodules were analyzed using chi-square analysis with Bonferroni correction. A significant difference was defined at p < 0.004.

Results

Performances of the model and observers were equivalent (AUC 0.932 versus 0.910, p = 0.184) for risk-assessment of malignant and benign nodules of all sizes. However, human readers performed superior to the computer model for differentiating malignant nodules from size-matched benign nodules (AUC 0.819 versus 0.706, p < 0.001). Large variations between observers were seen for ROC areas and ranges of risk scores. Morphological findings indicative of malignancy referred to border characteristics (spiculation, p < 0.001) and perinodular architectural deformation (distortion of surrounding lung parenchyma architecture, p < 0.001; pleural retraction, p = 0.002).

Conclusions

Computer model and human observers perform equivalent for differentiating malignant from randomly selected benign nodules, confirming the high potential of computer models for nodule risk estimation in population based screening studies. However, computer models highly rely on size as discriminator. Incorporation of other morphological criteria used by human observers to superiorly discriminate size-matched malignant from benign nodules, will further improve computer performance.

Automatic Detection of Tuberculosis in Chest Radiographs Using a Combination of Textural, Focal, and Shape Abnormality Analysis

Tuberculosis (TB) is a common disease with high mortality and morbidity rates worldwide. Automatic systems to detect TB on chest radiographs (CXRs) can improve the efficiency of diagnostic algorithms for pulmonary TB. The diverse manifestation of TB on CXRs from different populations requires a system that can be adapted to deal with different types of abnormalities. A computer aided detection (CAD) system was developed which combines several subscores of supervised subsystems detecting textural, shape, and focal abnormalities into one TB score. A general framework was developed to combine an arbitrary number of subscores: subscores were normalized, collected in a feature vector and then combined using a supervised classifier into one combined score. The method was evaluated on two databases, both consisting of 200 digital CXRs, from: (A) Western high-risk group screening, (B) TB suspect screening in Africa. The subscores and combined score were compared to (1) an external, non-radiological, reference and (2) a radiological reference determined by a human expert. Performance was measured using Receiver Operator Characteristic (ROC) analysis. Different subscores performed best in the two databases. The combined TB score performed better than the individual subscores, except for the external reference in database B. The performances of the independent observer were slightly higher than the combined TB score. Compared to the external reference, differences in performance between the combined TB score and the independent observer were not significant in both databases. Supervised combination to compute an overall TB score allows for a necessary adaptation of the CAD system to different settings or different operational requirements.

Observer Variability for Classification of Pulmonary Nodules on Low-Dose CT Images and Its Effect on Nodule Management

PURPOSE

To examine the factors that affect inter- and intraobserver agreement for pulmonary nodule type classification on low-radiation-dose computed tomographic (CT) images, and their potential effect on patient management.

MATERIALS AND METHODS

Nodules (n = 160) were randomly selected from the Dutch-Belgian Lung Cancer Screening Trial cohort, with equal numbers of nodule types and similar sizes. Nodules were scored by eight radiologists by using morphologic categories proposed by the Fleischner Society guidelines for management of pulmonary nodules as solid, part solid with a solid component smaller than 5 mm, part solid with a solid component 5 mm or larger, or pure ground glass. Inter- and intraobserver agreement was analyzed by using Cohen κ statistics. Multivariate analysis of variance was performed to assess the effect of nodule characteristics and image quality on observer disagreement. Effect on nodule management was estimated by differentiating CT follow-up for ground-glass nodules, solid nodules 8 mm or smaller, and part-solid nodules smaller than 5 mm from immediate diagnostic work-up for solid nodules larger than 8 mm and part-solid nodules 5 mm or greater.

RESULTS

Pair-wise inter- and intraobserver agreement was moderate (mean κ, 0.51 [95% confidence interval, 0.30, 0.68] and 0.57 [95% confidence interval, 0.47, 0.71]). Categorization as part-solid nodules and location in the upper lobe significantly reduced observer agreement (P = .012 and P < .001, respectively). By considering all possible reading pairs (28 possible combinations of observer pairs × 160 nodules = 4480 possible agreements or disagreements), a discordant nodule classification was found in 36.4% (1630 of 4480), related to presence or size of a solid component in 88.7% (1446 of 1630). Two-thirds of these discrepant readings (1061 of 1630) would have potentially resulted in different nodule management.

CONCLUSION

There is moderate inter- and intraobserver agreement for nodule classification by using current recommendations for low-radiation-dose CT examinations of the chest. Discrepancies in nodule categorization were mainly caused by disagreement on the size and presence of a solid component, which may lead to different management in the majority of cases with such discrepancies. (©) RSNA, 2015.

Cavity contour segmentation in chest radiographs using supervised learning and dynamic programming

PURPOSE

Efficacy of tuberculosis (TB) treatment is often monitored using chest radiography. Monitoring size of cavities in pulmonary tuberculosis is important as the size predicts severity of the disease and its persistence under therapy predicts relapse. The authors present a method for automatic cavity segmentation in chest radiographs.

METHODS

A two stage method is proposed to segment the cavity borders, given a user defined seed point close to the center of the cavity. First, a supervised learning approach is employed to train a pixel classifier using texture and radial features to identify the border pixels of the cavity. A likelihood value of belonging to the cavity border is assigned to each pixel by the classifier. The authors experimented with four different classifiers:k-nearest neighbor (kNN), linear discriminant analysis (LDA), GentleBoost (GB), and random forest (RF). Next, the constructed likelihood map was used as an input cost image in the polar transformed image space for dynamic programming to trace the optimal maximum cost path. This constructed path corresponds to the segmented cavity contour in image space.

RESULTS

The method was evaluated on 100 chest radiographs (CXRs) containing 126 cavities. The reference segmentation was manually delineated by an experienced chest radiologist. An independent observer (a chest radiologist) also delineated all cavities to estimate interobserver variability. Jaccard overlap measure Ω was computed between the reference segmentation and the automatic segmentation; and between the reference segmentation and the independent observer's segmentation for all cavities. A median overlap Ω of 0.81 (0.76 ± 0.16), and 0.85 (0.82 ± 0.11) was achieved between the reference segmentation and the automatic segmentation, and between the segmentations by the two radiologists, respectively. The best reported mean contour distance and Hausdorff distance between the reference and the automatic segmentation were, respectively, 2.48 ± 2.19 and 8.32 ± 5.66 mm, whereas these distances were 1.66 ± 1.29 and 5.75 ± 4.88 mm between the segmentations by the reference reader and the independent observer, respectively. The automatic segmentations were also visually assessed by two trained CXR readers as "excellent," "adequate," or "insufficient." The readers had good agreement in assessing the cavity outlines and 84% of the segmentations were rated as "excellent" or "adequate" by both readers.

CONCLUSIONS

The proposed cavity segmentation technique produced results with a good degree of overlap with manual expert segmentations. The evaluation measures demonstrated that the results approached the results of the experienced chest radiologists, in terms of overlap measure and contour distance measures. Automatic cavity segmentation can be employed in TB clinics for treatment monitoring, especially in resource limited settings where radiologists are not available.

Detection of tuberculosis using digital chest radiography: automated reading vs. interpretation by clinical officers

SETTING

A busy urban health centre in Lusaka, Zambia.

OBJECTIVE

To compare the accuracy of automated reading (CAD4TB) with the interpretation of digital chest radiograph (CXR) by clinical officers for the detection of tuberculosis (TB).

DESIGN

A retrospective analysis was performed on 161 subjects enrolled in a TB specimen bank study. CXRs were analysed using CAD4TB, which computed an image abnormality score (0-100). Four clinical officers scored the CXRs for abnormalities consistent with TB. We compared the automated readings and the readings by clinical officers against the bacteriological and radiological results used as reference. We report here the area under the receiver operating characteristic curve (AUC) and kappa (κ) statistics.

RESULTS

Of 161 enrolled subjects, 97 had bacteriologically confirmed TB and 120 had abnormal CXR. The AUCs for CAD4TB and the clinical officers were respectively 0.73 and 0.65-0.75 in comparison with the bacteriological reference, and 0.91 and 0.89-0.94 in comparison with the radiological reference. P values indicated no significant differences, except for one clinical officer who performed significantly worse than CAD4TB (P < 0.05) using the bacteriological reference. κ values for CAD4TB and clinical officers with radiological reference were respectively 0.61 and 0.49-0.67.

CONCLUSION

CXR assessment using CAD4TB and by clinical officers is comparable. CAD4TB has potential as a point-of-care test and for the automated identification of subjects who require further examinations.

Bone suppressed images improve radiologists' detection performance for pulmonary nodules in chest radiographs

OBJECTIVES

To assess the effect of bone suppression imaging on observer performance in detecting lung nodules in chest radiographs.

MATERIALS AND METHODS

Posteroanterior (PA) and lateral digital chest radiographs of 111 (average age 65) patients with a CT proven solitary nodule (median diameter 15 mm), and 189 (average age 63) controls were read by 5 radiologists and 3 residents. Conspicuity of nodules on the radiographs was classified in obvious (n = 32), moderate (n = 32), subtle (n = 29) and very subtle (n = 18). Observers read the PA and lateral chest radiographs without and with an additional PA bone suppressed image (BSI) (ClearRead Bone Suppression 2.4, Riverain Technologies, Ohio) within one reading session. Multi reader multi case (MRMC) receiver operating characteristics (ROC) were used for statistical analysis.

RESULTS

ROC analysis showed improved detection with use of BSI compared to chest radiographs alone (AUC = 0.883 versus 0.855; p = 0.004). Performance also increased at high specificities exceeding 80% (pAUC = 0.136 versus 0.124; p = 0.0007). Operating at a specificity of 90%, sensitivity increased with BSI from 66% to 71% (p = 0.0004). Increase of detection performance was highest for nodules with moderate and subtle conspicuity (p = 0.02; p = 0.03).

CONCLUSION

Bone suppressed images improve radiologists' detection performance for pulmonary nodules, especially for those of moderate and subtle conspicuity.

18F-FDG PET early response evaluation of locally advanced non-small cell lung cancer treated with concomitant chemoradiotherapy

The potential of (18)F-FDG PET changes was evaluated for prediction of response to concomitant chemoradiotherapy in patients with locally advanced non-small cell lung cancer (NSCLC).

METHODS

For 28 patients, (18)F-FDG PET was performed before treatment, at the end of the second week of treatment, and at 2 wk and 3 mo after the completion of treatment. Standardized uptake value (SUV), maximum SUV, metabolic tumor volume (MTV), and total lesion glycolysis (TLG) were obtained. Early metabolic changes were defined as fractional change (ΔTLG) when (18)F-FDG PET at the end of the second week was compared with pretreatment (18)F-FDG PET. In-treatment metabolic changes, as measured by serial (18)F-FDG PET, were correlated with standard criteria of response evaluation of solid tumors by means of CT imaging (Response Evaluation Criteria In Solid Tumors 1.1). Parameters were analyzed for stratification in progression-free survival (PFS).

RESULTS

When compared with early metabolic nonresponders, a ΔTLG decrease of 38% or more was associated with a significantly longer PFS (1-y PFS 80% vs. 36%, P = 0.02). Pretreatment TLG was found to be a prognostic factor for PFS.

CONCLUSION

The degree of change in TLG was predictive for response to concomitant chemoradiotherapy as early as the end of the second week into treatment for patients with locally advanced NSCLC. Pretreatment TLG was prognostic for PFS.

Timeliness of lung cancer diagnosis and treatment in a rapid outpatient diagnostic program with combined 18FDG-PET and contrast enhanced CT scanning

INTRODUCTION

Delays in the diagnosis of lung cancer are under debate and may affect outcome. The objectives of this study were to compare various delays in a rapid outpatient diagnostic program (RODP) for suspected lung cancer patients with those described in literature and with guideline recommendations, to investigate the effects of referral route and symptoms on delays, and to establish whether delays were related to disease stage and outcome.

METHODS

A retrospective chart study was conducted of all patients with suspected lung cancer, referred to the RODP of our tertiary care university clinic between 1999 and 2009. Patient characteristics, tumor stage and different delays were analyzed.

RESULTS

Medical charts of 565 patients were retrieved. 290 patients (51.3%) were diagnosed with lung cancer, 48 (8.5%) with another type of malignancy, and in 111 patients (19.6%) the radiological anomaly was diagnosed as non-malignant. In 112 (19.8%) no immediate definite diagnosis was obtained, however in 82 of these cases (73.2%) the proposed follow-up strategy confirmed a benign outcome. The median first line delay was 54 days, IQR (interquartile range) 20-104 days, median patient delay 19 days (IQR 4-52 days), median referral delay was 7 days (IQR 5-9 days), median diagnostic delay 2 days (IQR 1-19 days). In 87% a diagnosis was obtained within 3 weeks after visiting a chest physician and 52.5% started curative therapy within 2 weeks after diagnosis. Patients presenting with hemoptysis had shorter first line delays. The RODP care was generally far more timely compared to literature and published guidelines, except for both referral and palliative therapeutic delay. No specific delay was significantly related to disease stage or survival.

CONCLUSIONS

An RODP results in a timely diagnosis well within guideline recommendations. Patient and first line delay account for most of total patient delay. Within the limitations of this retrospective study, we found no association with disease stage or survival.