Evaluation of a method of computer-aided detection (CAD) of pulmonary nodules with computed tomography

A Deep Learning-Based Assay for Programmed Death Ligand 1 Immunohistochemistry Scoring in Non-Small Cell Lung Carcinoma: Does it Help Pathologists Score?

Several studies have developed various artificial intelligence (AI) models for immunohistochemical analysis of programmed death ligand 1 (PD-L1) in patients with non-small cell lung carcinoma; however, none have focused on specific ways by which AI-assisted systems could help pathologists determine the tumor proportion score (TPS). In this study, we developed an AI model to calculate the TPS of the PD-L1 22C3 assay and evaluated whether and how this AI-assisted system could help pathologists determine the TPS and analyze how AI-assisted systems could affect pathologists' assessment accuracy. We assessed the 4 methods of the AI-assisted system: (1 and 2) pathologists first assessed and then referred to automated AI scoring results (1, positive tumor cell percentage; 2, positive tumor cell percentage and visualized overlay image) for final confirmation, and (3 and 4) pathologists referred to the automated AI scoring results (3, positive tumor cell percentage; 4, positive tumor cell percentage and visualized overlay image) while determining TPS. Mixed-model analysis was used to calculate the odds ratios (ORs) with 95% CI for AI-assisted TPS methods 1 to 4 compared with pathologists' scoring. For all 584 samples of the tissue microarray, the OR for AI-assisted TPS methods 1 to 4 was 0.94 to 1.07 and not statistically significant. Of them, we found 332 discordant cases, on which the pathologists' judgments were inconsistent; the ORs for AI-assisted TPS methods 1, 2, 3, and 4 were 1.28 (1.06-1.54; P = .012), 1.29 (1.06-1.55; P = .010), 1.28 (1.06-1.54; P = .012), and 1.29 (1.06-1.55; P = .010), respectively, which were statistically significant. For discordant cases, the OR for each AI-assisted TPS method compared with the others was 0.99 to 1.01 and not statistically significant. This study emphasized the usefulness of the AI-assisted system for cases in which pathologists had difficulty determining the PD-L1 TPS.

Artificial Intelligence Assisted Computational Tomographic Detection of Lung Nodules for Prognostic Cancer Examination: A Large-Scale Clinical Trial

Low-dose computed tomography (LDCT) has emerged as a standard method for detecting early-stage lung cancer. However, the tedious computer tomography (CT) slide reading, patient-by-patient check, and lack of standard criteria to determine the vague but possible nodule leads to variable outcomes of CT slide interpretation. To determine the artificial intelligence (AI)-assisted CT examination, AI algorithm-assisted CT screening was embedded in the hospital picture archiving and communication system, and a 200 person-scaled clinical trial was conducted at two medical centers. With AI algorithm-assisted CT screening, the sensitivity of detecting nodules sized 4−5 mm, 6~10 mm, 11~20 mm, and >20 mm increased by 41%, 11.2%, 10.3%, and 18.7%, respectively. Remarkably, the overall sensitivity of detecting varied nodules increased by 20.7% from 67.7% to 88.4%. Furthermore, the sensitivity increased by 18.5% from 72.5% to 91% for detecting ground glass nodules (GGN), which is challenging for radiologists and physicians. The free-response operating characteristic (FROC) AI score was ≥0.4, and the AI algorithm standalone CT screening sensitivity reached >95% with an area under the localization receiver operating characteristic curve (LROC-AUC) of >0.88. Our study demonstrates that AI algorithm-embedded CT screening significantly ameliorates tedious LDCT practices for doctors.

Performance and reading time of lung nodule identification on multidetector CT with or without an artificial intelligence-powered computer-aided detection system

AIM

To compare the performance and reading time of different readers using automatic artificial intelligence (AI)-powered computer-aided detection (CAD) to detect lung nodules in different reading modes.

MATERIALS AND METHODS

One hundred and fifty multidetector computed tomography (CT) datasets containing 340 nodules ≤10 mm in diameter were collected retrospectively. A CAD with vessel-suppressed function was used to interpret the images. Three junior and three senior readers were assigned to read (1) CT images without CAD, (2) second-read using CAD in which CAD was applied only after initial unassisted assessment, and (3) a concurrent read with CAD in which CAD was applied at the start of assessment. Diagnostic performances and reading times were compared using analysis of variance.

RESULTS

For all readers, the mean sensitivity improved from 64% (95% confidence interval [CI]: 62%, 66%) for the without-CAD mode to 82% (95% CI: 80%, 84%) for the second-reading mode and to 80% (95% CI: 79%, 82%) for the concurrent-reading mode (p<0.001). There was no significant difference between the two modes in terms of the mean sensitivity, specificity, and area under the receiver operating characteristic curve (AUC) for both junior and senior readers and all readers (p>0.05). The reading time of all readers was significantly shorter for the concurrent-reading mode (124 ± 25 seconds) compared to without CAD (156 ± 34 seconds; p<0.001) and the second-reading mode (197 ± 46 seconds; p<0.001).

CONCLUSION

In CAD for lung nodules at CT, the second-reading mode and concurrent-reading mode may improve detection performance for all readers in both screening and clinical routine practice. Concurrent use of CAD is more efficient for both junior and senior readers.

Incorporation of a computer-aided vessel-suppression system to detect lung nodules in CT images: effect on sensitivity and reading time in routine clinical settings

PURPOSE

To evaluate whether a computer-aided vessel-suppression system improves lung nodule detection in routine clinical settings.

MATERIALS AND METHODS

We used computer software that automatically suppresses pulmonary vessels on chest CT while preserving pulmonary nodules. Sixty-one chest CT images were included in our study. Three radiologists independently read either standard CT images alone or both computer-aided CT and standard CT images randomly to detect a pulmonary nodule ≥ 4 mm in diameter. After an interval of at least 15 days to avoid recall bias, the three radiologists interpreted the counterpart images of the same patients. The reference standard was decided by an expert panel. The primary endpoint was sensitivity. The secondary endpoint was interpretation time.

RESULTS

The average sensitivity improved with computer-aided CT (72% for standard CT vs. 84% for computer-aided CT, p = 0.02). There was no difference in the false-positive rate (21% for both standard CT and computer-aided CT, p = 0.98). Although the average reading time was 9.5% longer for computer-aided plus standard CT compared with standard CT alone, the difference was not significant (p = 0.11).

CONCLUSION

Vessel-suppressed CT images helped radiologists to improve the sensitivity of pulmonary nodule detection without compromising the false-positive rate.

JOURNAL CLUB: Computer-Aided Detection of Lung Nodules on CT With a Computerized Pulmonary Vessel Suppressed Function

OBJECTIVE

The purpose of this study is to evaluate radiologists' performance in detecting actionable nodules on chest CT when aided by a pulmonary vessel image-suppressed function and a computer-aided detection (CADe) system.

MATERIALS AND METHODS

A novel computerized pulmonary vessel image-suppressed function with a built-in CADe (VIS/CADe) system was developed to assist radiologists in interpreting thoracic CT images. Twelve radiologists participated in a comparative study without and with the VIS/CADe using 324 cases (involving 95 cancers and 83 benign nodules). The ratio of nodule-free cases to cases with nodules was 2:1 in the study. Localization ROC (LROC) methods were used for analysis.

RESULTS

In a stand-alone test, the VIS/CADe system detected 89.5% and 82.0% of malignant nodules and all nodules no smaller than 5 mm, respectively. The false-positive rate per CT study was 0.58. For the reader study, the mean area under the LROC curve (LROCAUC) for the detection of lung cancer significantly increased from 0.633 when unaided by VIS/CADe to 0.773 when aided by VIS/CADe (p < 0.01). For the detection of all clinically actionable nodules, the mean LROC-AUC significantly increased from 0.584 when unaided by VIS/CADe to 0.692 when detection was aided by VIS/CADe (p < 0.01). Radiologists detected 80.0% of cancers with VIS/CADe versus 64.45% of cancers unaided (p < 0.01); specificity decreased from 89.9% to 84.4% (p < 0.01). Radiologist interpretation time significantly decreased by 26%.

CONCLUSION

The VIS/CADe system significantly increased radiologists' detection of cancers and actionable nodules with somewhat lower specificity. With use of the VIS/CADe system, radiologists increased their interpretation speed by a factor of approximately one-fourth. Our study suggests that the technique has the potential to assist radiologists in the detection of additional actionable nodules on thoracic CT.

Missed lung cancer: when, where, and why?

Missed lung cancer is a source of concern among radiologists and an important medicolegal challenge. In 90% of the cases, errors in diagnosis of lung cancer occur on chest radiographs. It may be challenging for radiologists to distinguish a lung lesion from bones, pulmonary vessels, mediastinal structures, and other complex anatomical structures on chest radiographs. Nevertheless, lung cancer can also be overlooked on computed tomography (CT) scans, regardless of the context, either if a clinical or radiologic suspect exists or for other reasons. Awareness of the possible causes of overlooking a pulmonary lesion can give radiologists a chance to reduce the occurrence of this eventuality. Various factors contribute to a misdiagnosis of lung cancer on chest radiographs and on CT, often very similar in nature to each other. Observer error is the most significant one and comprises scanning error, recognition error, decision-making error, and satisfaction of search. Tumor characteristics such as lesion size, conspicuity, and location are also crucial in this context. Even technical aspects can contribute to the probability of skipping lung cancer, including image quality and patient positioning and movement. Albeit it is hard to remove missed lung cancer completely, strategies to reduce observer error and methods to improve technique and automated detection may be valuable in reducing its likelihood.

The impact of trained radiographers as concurrent readers on performance and reading time of experienced radiologists in the UK Lung Cancer Screening (UKLS) trial

Objectives

To compare radiologists’ performance reading CTs independently with their performance using radiographers as concurrent readers in lung cancer screening.

Methods

369 consecutive baseline CTs performed for the UK Lung Cancer Screening (UKLS) trial were double-read by radiologists reading either independently or concurrently with a radiographer. In concurrent reading, the radiologist reviewed radiographer-identified nodules and then detected any additional nodules. Radiologists recorded their independent and concurrent reading times. For each radiologist, sensitivity, average false-positive detections (FPs) per case and mean reading times for each method were calculated.

Results

694 nodules in 246/369 (66.7%) studies comprised the reference standard. Radiologists’ mean sensitivity and average FPs per case both increased with concurrent reading compared to independent reading (90.8 ± 5.6% vs. 77.5 ± 11.2%, and 0.60 ± 0.53 vs. 0.33 ± 0.20, respectively; p < 0.05 for 3/4 and 2/4 radiologists, respectively). The mean reading times per case decreased from 9.1 ± 2.3 min with independent reading to 7.2 ± 1.0 min with concurrent reading, decreasing significantly for 3/4 radiologists (p < 0.05).

Conclusions

The majority of radiologists demonstrated improved sensitivity, a small increase in FP detections and a statistically significantly reduced reading time using radiographers as concurrent readers.

Key Points

• Radiographers as concurrent readers could improve radiologists’ sensitivity in lung nodule detection.

• An increase in false-positive detections with radiographer-assisted concurrent reading occurred.

• The false-positive detection rate was still lower than reported for computer-aided detection.

• Concurrent reading with radiographers was also faster than single reading.

• The time saved per case using concurrently reading radiographers was relatively modest.

Electronic supplementary material

The online version of this article (doi:10.1007/s00330-017-4903-z) contains supplementary material, which is available to authorized users.