Comparison of sensitivity and reading time for the use of computer-aided detection (CAD) of pulmonary nodules at MDCT as concurrent or second reader

Deep generative abnormal lesion emphasization validated by nine radiologists and 1000 chest X-rays with lung nodules

A general-purpose method of emphasizing abnormal lesions in chest radiographs, named EGGPALE (Extrapolative, Generative and General-Purpose Abnormal Lesion Emphasizer), is presented. The proposed EGGPALE method is composed of a flow-based generative model and L-infinity-distance-based extrapolation in a latent space. The flow-based model is trained using only normal chest radiographs, and an invertible mapping function from the image space to the latent space is determined. In the latent space, a given unseen image is extrapolated so that the image point moves away from the normal chest X-ray hyperplane. Finally, the moved point is mapped back to the image space and the corresponding emphasized image is created. The proposed method was evaluated by an image interpretation experiment with nine radiologists and 1,000 chest radiographs, of which positive suspected lung cancer cases and negative cases were validated by computed tomography examinations. The sensitivity of EGGPALE-processed images showed +0.0559 average improvement compared with that of the original images, with -0.0192 deterioration of average specificity. The area under the receiver operating characteristic curve of the ensemble of nine radiologists showed a statistically significant improvement. From these results, the feasibility of EGGPALE for enhancing abnormal lesions was validated. Our code is available at https://github.com/utrad-ical/Eggpale.

Effects of artificial intelligence implementation on efficiency in medical imaging-a systematic literature review and meta-analysis

In healthcare, integration of artificial intelligence (AI) holds strong promise for facilitating clinicians' work, especially in clinical imaging. We aimed to assess the impact of AI implementation for medical imaging on efficiency in real-world clinical workflows and conducted a systematic review searching six medical databases. Two reviewers double-screened all records. Eligible records were evaluated for methodological quality. The outcomes of interest were workflow adaptation due to AI implementation, changes in time for tasks, and clinician workload. After screening 13,756 records, we identified 48 original studies to be incuded in the review. Thirty-three studies measured time for tasks, with 67% reporting reductions. Yet, three separate meta-analyses of 12 studies did not show significant effects after AI implementation. We identified five different workflows adapting to AI use. Most commonly, AI served as a secondary reader for detection tasks. Alternatively, AI was used as the primary reader for identifying positive cases, resulting in reorganizing worklists or issuing alerts. Only three studies scrutinized workload calculations based on the time saved through AI use. This systematic review and meta-analysis represents an assessment of the efficiency improvements offered by AI applications in real-world clinical imaging, predominantly revealing enhancements across the studies. However, considerable heterogeneity in available studies renders robust inferences regarding overall effectiveness in imaging tasks. Further work is needed on standardized reporting, evaluation of system integration, and real-world data collection to better understand the technological advances of AI in real-world healthcare workflows. Systematic review registration: Prospero ID CRD42022303439, International Registered Report Identifier (IRRID): RR2-10.2196/40485.

A Deep Learning-Based Assay for Programmed Death Ligand 1 Immunohistochemistry Scoring in Non-Small Cell Lung Carcinoma: Does it Help Pathologists Score?

Several studies have developed various artificial intelligence (AI) models for immunohistochemical analysis of programmed death ligand 1 (PD-L1) in patients with non-small cell lung carcinoma; however, none have focused on specific ways by which AI-assisted systems could help pathologists determine the tumor proportion score (TPS). In this study, we developed an AI model to calculate the TPS of the PD-L1 22C3 assay and evaluated whether and how this AI-assisted system could help pathologists determine the TPS and analyze how AI-assisted systems could affect pathologists' assessment accuracy. We assessed the 4 methods of the AI-assisted system: (1 and 2) pathologists first assessed and then referred to automated AI scoring results (1, positive tumor cell percentage; 2, positive tumor cell percentage and visualized overlay image) for final confirmation, and (3 and 4) pathologists referred to the automated AI scoring results (3, positive tumor cell percentage; 4, positive tumor cell percentage and visualized overlay image) while determining TPS. Mixed-model analysis was used to calculate the odds ratios (ORs) with 95% CI for AI-assisted TPS methods 1 to 4 compared with pathologists' scoring. For all 584 samples of the tissue microarray, the OR for AI-assisted TPS methods 1 to 4 was 0.94 to 1.07 and not statistically significant. Of them, we found 332 discordant cases, on which the pathologists' judgments were inconsistent; the ORs for AI-assisted TPS methods 1, 2, 3, and 4 were 1.28 (1.06-1.54; P = .012), 1.29 (1.06-1.55; P = .010), 1.28 (1.06-1.54; P = .012), and 1.29 (1.06-1.55; P = .010), respectively, which were statistically significant. For discordant cases, the OR for each AI-assisted TPS method compared with the others was 0.99 to 1.01 and not statistically significant. This study emphasized the usefulness of the AI-assisted system for cases in which pathologists had difficulty determining the PD-L1 TPS.

Imaging of Solid Pulmonary Nodules

Early detection with accurate classification of solid pulmonary nodules is critical in reducing lung cancer morbidity and mortality. Computed tomography (CT) remains the most widely used imaging examination for pulmonary nodule evaluation; however, other imaging modalities, such as PET/CT and MRI, are increasingly used for nodule characterization. Current advances in solid nodule imaging are largely due to developments in machine learning, including automated nodule segmentation and computer-aided detection. This review explores current multi-modality solid pulmonary nodule detection and characterization with discussion of radiomics and risk prediction models.

Effectiveness of FEES with artificial intelligence-assisted computer-aided diagnosis

OBJECTIVES

FEES is a standard procedure for diagnosing dysphagia. However, appropriate evaluation of FEES findings is difficult for inexperienced evaluators. Recent progress in deep learning has highlighted the use of artificial intelligence-assisted computer-aided diagnosis (AI-assisted CAD) in medical applications. We investigated the detection accuracy of FEES findings evaluated by inexperienced evaluators with and without the use of CAD.

METHODS

The algorithm for FEES-CAD was developed using 25,630 expert-annotated images. A total of 45 inexperienced evaluators from three groups of people (resident doctors, nurses, and medical students), evaluated 32 FEES videos from 32 patients. To confirm the effectiveness of FEES-CAD, first, 32 FEES videos were evaluated without the use of CAD. Second, one half was evaluated with, and one half without, the use of CAD. The detection accuracy of the FEES findings was investigated, and the evaluation results obtained with CAD were statistically compared with those obtained without CAD.

RESULTS

In the first FEES evaluation, the total detection accuracy was 82.2 %. In the second evaluation, the total detection accuracy with CAD was 84.3 %, and that without CAD was 81.7 %. The detection accuracies by the resident doctors, nurses, and medical students with CAD were 90.1 %, 82.6 %, and 79.4 %, respectively, and those without CAD were 83.7 %, 80.9 % and 80.1 %, respectively. In the resident doctors, the detection accuracy was significantly better when CAD was used for evaluation, compared with the non-CAD evaluations.

CONCLUSION

The present study demonstrated the effectiveness of FEES-CAD in improving the detection accuracy of resident doctors, however, the differences were small.

Evaluation of a Deep Learning-based Algorithm for Post-Radiotherapy Prostate Cancer Local Recurrence Detection Using Biparametric MRI

OBJECTIVE

To evaluate a biparametric MRI (bpMRI)-based artificial intelligence (AI) model for the detection of local prostate cancer (PCa) recurrence in patients with radiotherapy history.

MATERIALS AND METHODS

This study included post-radiotherapy patients undergoing multiparametric MRI and subsequent MRI/US fusion-guided and/or systematic biopsy. Histopathology results were used as ground truth. The recurrent cancer detection sensitivity of a bpMRI-based AI model, which was developed on a large dataset to primarily identify lesions in treatment-naïve patients, was compared to a prospective radiologist assessment using the Wald test. Subanalysis was conducted on patients stratified by the treatment modality (external beam radiation treatment [EBRT] and brachytherapy) and the prostate volume quartiles.

RESULTS

Of the 62 patients included (median age = 70 years; median PSA = 3.51 ng/ml; median prostate volume = 27.55 ml), 56 recurrent PCa foci were identified within 46 patients. The AI model detected 40 lesions in 35 patients. The AI model performance was lower than the prospective radiology interpretation (Rad) on a patient-(AI: 76.1% vs. Rad: 91.3%, p = 0.02) and lesion-level (AI: 71.4% vs. Rad: 87.5%, p = 0.01). The mean number of false positives per patient was 0.35 (range: 0-2). The AI model performance was higher in EBRT group both on patient-level (EBRT: 81.5% [22/27] vs. brachytherapy: 68.4% [13/19]) and lesion-level (EBRT: 79.4% [27/34] vs. brachytherapy: 59.1% [13/22]). In patients with gland volumes >34 ml (n = 25), detection sensitivities were 100% (11/11) and 94.1% (16/17) on patient- and lesion-level, respectively.

CONCLUSION

The reported bpMRI-based AI model detected the majority of locally recurrent prostate cancer after radiotherapy. Further testing including external validation of this model is warranted prior to clinical implementation.

Incorporation of CAD (computer-aided detection) with thin-slice lung CT in routine 18F-FDG PET/CT imaging read-out protocol for detection of lung nodules

OBJECTIVE

To evaluate the detection rate and performance of 18F-FDG PET alone (PET), the combination of PET and low-dose thick-slice CT (PET/lCT), PET and diagnostic thin-slice CT (PET/dCT), and additional computer-aided detection (PET/dCT/CAD) for lung nodules (LN)/metastases in tumor patients. Along with this, assessment of inter-reader agreement and time requirement for different techniques were evaluated as well.

METHODS

In 100 tumor patients (56 male, 44 female; age range: 22-93 years, mean age: 60 years) 18F-FDG PET images, low-dose CT with shallow breathing (5 mm slice thickness), and diagnostic thin-slice CT (1 mm slice thickness) in full inspiration were retrospectively evaluated by three readers with variable experience (junior, mid-level, and senior) for the presence of lung nodules/metastases and additionally analyzed with CAD. Time taken for each analysis and number of the nodules detected were assessed. Sensitivity, specificity, positive and negative predictive value, accuracy, and Receiver operating characteristic (ROC) analysis of each technique was calculated. Histopathology and/or imaging follow-up served as reference standard for the diagnosis of metastases.

RESULTS

Three readers, on an average, detected 40 LN in 17 patients with PET only, 121 LN in 37 patients using ICT, 283 LN in 60 patients with dCT, and 282 LN in 53 patients with CAD. On average, CAD detected 49 extra LN, missed by the three readers without CAD, whereas CAD overall missed 53 LN. There was very good inter-reader agreement regarding the diagnosis of metastases for all four techniques (kappa: 0.84-0.93). The average time required for the evaluation of LN in PET, lCT, dCT, and CAD was 25, 31, 60, and 40 s, respectively; the assistance of CAD lead to average 33% reduction in time requirement for evaluation of lung nodules compared to dCT. The time-saving effect was highest in the less experienced reader. Regarding the diagnosis of metastases, sensitivity and specificity combined of all readers were 47.8%/96.2% for PET, 80.0%/81.9% for PET/lCT, 100%/56.7% for PET/dCT, and 95.6%/64.3% for PET/CAD. No significant difference was observed regarding the ROC AUC (area under the curve) between the imaging methods.

CONCLUSION

Implementation of CAD for the detection of lung nodules/metastases in routine 18F-FDG PET/CT read-out is feasible. The combination of diagnostic thin-slice CT and CAD significantly increases the detection rate of lung nodules in tumor patients compared to the standard PET/CT read-out. PET combined with low-dose CT showed the best balance between sensitivity and specificity regarding the diagnosis of metastases per patient. CAD reduces the time required for lung nodule/metastasis detection, especially for less experienced readers.

The impact of AI suggestions on radiologists' decisions: a pilot study of explainability and attitudinal priming interventions in mammography examination

Various studies have shown that medical professionals are prone to follow the incorrect suggestions offered by algorithms, especially when they have limited inputs to interrogate and interpret such suggestions and when they have an attitude of relying on them. We examine the effect of correct and incorrect algorithmic suggestions on the diagnosis performance of radiologists when (1) they have no, partial, and extensive informational inputs for explaining the suggestions (study 1) and (2) they are primed to hold a positive, negative, ambivalent, or neutral attitude towards AI (study 2). Our analysis of 2760 decisions made by 92 radiologists conducting 15 mammography examinations shows that radiologists' diagnoses follow both incorrect and correct suggestions, despite variations in the explainability inputs and attitudinal priming interventions. We identify and explain various pathways through which radiologists navigate through the decision process and arrive at correct or incorrect decisions. Overall, the findings of both studies show the limited effect of using explainability inputs and attitudinal priming for overcoming the influence of (incorrect) algorithmic suggestions.

The value of deep learning-based computer aided diagnostic system in improving diagnostic performance of rib fractures in acute blunt trauma

BACKGROUND

To evaluate the value of a deep learning-based computer-aided diagnostic system (DL-CAD) in improving the diagnostic performance of acute rib fractures in patients with chest trauma.

MATERIALS AND METHODS

CT images of 214 patients with acute blunt chest trauma were retrospectively analyzed by two interns and two attending radiologists independently firstly and then with the assistance of a DL-CAD one month later, in a blinded and randomized manner. The consensusdiagnosis of fib fracture by another two senior thoracic radiologists was regarded as reference standard. The rib fracture diagnostic sensitivity, specificity, positive predictive value, diagnostic confidence and mean reading time with and without DL-CAD were calculated and compared.

RESULTS

There were 680 rib fracture lesions confirmed as reference standard among all patients. The diagnostic sensitivity and positive predictive value of interns weresignificantly improved from (68.82%, 84.50%) to (91.76%, 93.17%) with the assistance of DL-CAD, respectively. Diagnostic sensitivity and positive predictive value of attendings aided by DL-CAD (94.56%, 95.67%) or not aided (86.47%, 93.83%), respectively. In addition, when radiologists were assisted by DL-CAD, the mean reading time was significantly reduced, and diagnostic confidence was significantly enhanced.

CONCLUSIONS

DL-CAD improves the diagnostic performance of acute rib fracture in chest trauma patients, which increases the diagnostic confidence, sensitivity, and positive predictive value for radiologists. DL-CAD can advance the diagnostic consistency of radiologists with different experiences.

Implementation of artificial intelligence in thoracic imaging-a what, how, and why guide from the European Society of Thoracic Imaging (ESTI)

This statement from the European Society of Thoracic imaging (ESTI) explains and summarises the essentials for understanding and implementing Artificial intelligence (AI) in clinical practice in thoracic radiology departments. This document discusses the current AI scientific evidence in thoracic imaging, its potential clinical utility, implementation and costs, training requirements and validation, its' effect on the training of new radiologists, post-implementation issues, and medico-legal and ethical issues. All these issues have to be addressed and overcome, for AI to become implemented clinically in thoracic radiology. KEY POINTS: • Assessing the datasets used for training and validation of the AI system is essential. • A departmental strategy and business plan which includes continuing quality assurance of AI system and a sustainable financial plan is important for successful implementation. • Awareness of the negative effect on training of new radiologists is vital.

Artificial Intelligence Assisted Computational Tomographic Detection of Lung Nodules for Prognostic Cancer Examination: A Large-Scale Clinical Trial

Low-dose computed tomography (LDCT) has emerged as a standard method for detecting early-stage lung cancer. However, the tedious computer tomography (CT) slide reading, patient-by-patient check, and lack of standard criteria to determine the vague but possible nodule leads to variable outcomes of CT slide interpretation. To determine the artificial intelligence (AI)-assisted CT examination, AI algorithm-assisted CT screening was embedded in the hospital picture archiving and communication system, and a 200 person-scaled clinical trial was conducted at two medical centers. With AI algorithm-assisted CT screening, the sensitivity of detecting nodules sized 4−5 mm, 6~10 mm, 11~20 mm, and >20 mm increased by 41%, 11.2%, 10.3%, and 18.7%, respectively. Remarkably, the overall sensitivity of detecting varied nodules increased by 20.7% from 67.7% to 88.4%. Furthermore, the sensitivity increased by 18.5% from 72.5% to 91% for detecting ground glass nodules (GGN), which is challenging for radiologists and physicians. The free-response operating characteristic (FROC) AI score was ≥0.4, and the AI algorithm standalone CT screening sensitivity reached >95% with an area under the localization receiver operating characteristic curve (LROC-AUC) of >0.88. Our study demonstrates that AI algorithm-embedded CT screening significantly ameliorates tedious LDCT practices for doctors.

Detection of intracranial aneurysms using deep learning-based CAD system: usefulness of the scores of CNN's final layer for distinguishing between aneurysm and infundibular dilatation

PURPOSE

We evaluated the diagnostic performance of a clinically available deep learning-based computer-assisted diagnosis software for detecting unruptured aneurysms (UANs) using magnetic resonance angiography and assessed the functionality of the convolutional neural network (CNN) final layer score for distinguishing between UAN and infundibular dilatation (ID).

MATERIALS AND METHODS

EIRL brain aneurysm (EIRL_BA) was used in this study. The subjects were 117 UAN and/or ID cases including 100 UAN lesions (average sizes of 2.56 ± 1.45 mm) and 40 ID lesions (average sizes of 1.75 ± 0.41 mm) in any of internal carotid artery, middle cerebral artery, and anterior communicating artery, and 123 normal controls. The sensitivity, specificity, and accuracy of EIRL_BA were determined for UAN and ID or UAN only. Furthermore, the relationship between the lesion category and score was examined using a linear regression analysis model, and the receiver operating characteristic (ROC) analysis was used to assess whether the scores represent UAN-like characteristics.

RESULTS

EIRL_BA showed a total of 203 candidates (an average of 1.73/case) in UAN and/or ID cases and 98 candidates (an average of 0.80/case) in normal controls. For diagnosing either UAN/ID, EIRL_BA showed an overall sensitivity of 80%, specificity of 84.2%, and accuracy of 83.7%, resulting in the positive likelihood ratio of 5.0. For diagnosing UAN only, the overall sensitivity of 89.0, specificity of 82.6%, and accuracy of 83.2% resulting in the positive likelihood ratio of 5.1. In a linear regression analysis, the scores significantly increased in the candidates' first and second ranks in UAN (p < 0.05) but not in ID. An ROC analysis using the score for diagnosing UAN showed an area under the curve of 0.836.

CONCLUSION

EIRL_BA is applicable for detecting small UAN, and the CNN's final layer scores may be an effective index for discriminating UAN and ID and representing the likelihood of UAN.

Validation of deep learning-based computer-aided detection software use for interpretation of pulmonary abnormalities on chest radiographs and examination of factors that influence readers' performance and final diagnosis

PURPOSE

To evaluate the performance of a deep learning-based computer-aided detection (CAD) software for detecting pulmonary nodules, masses, and consolidation on chest radiographs (CRs) and to examine the effect of readers' experience and data characteristics on the sensitivity and final diagnosis.

MATERIALS AND METHODS

The CRs of 453 patients were retrospectively selected from two institutions. Among these CRs, 60 images with abnormal findings (pulmonary nodules, masses, and consolidation) and 140 without abnormal findings were randomly selected for sequential observer-performance testing. In the test, 12 readers (three radiologists, three pulmonologists, three non-pulmonology physicians, and three junior residents) interpreted 200 images with and without CAD, and the findings were compared. Weighted alternative free-response receiver operating characteristic (wAFROC) figure of merit (FOM) was used to analyze observer performance. The lesions that readers initially missed but CAD detected were stratified by anatomic location and degree of subtlety, and the adoption rate was calculated. Fisher's exact test was used for comparison.

RESULTS

The mean wAFROC FOM score of the 12 readers significantly improved from 0.746 to 0.810 with software assistance (P = 0.007). In the reader group with < 6 years of experience, the mean FOM score significantly improved from 0.680 to 0.779 (P = 0.011), while that in the reader group with ≥ 6 years of experience increased from 0.811 to 0.841 (P = 0.12). The sensitivity of the CAD software and the adoption rate for the lesions with subtlety level 2 or 3 (obscure) lesions were significantly lower than for level 4 or 5 (distinct) lesions (50% vs. 93%, P < 0.001; and 55% vs. 74%, P = 0.04, respectively).

CONCLUSION

CAD software use improved doctors' performance in detecting nodules/masses and consolidation on CRs, particularly for non-expert doctors, by preventing doctors from missing distinct lesions rather than helping them to detect obscure lesions.

Impact of Artificial Intelligence Assistance on Chest CT Interpretation Times: A Prospective Randomized Study

BACKGROUND. Deep learning-based convolutional neural networks have enabled major advances in development of artificial intelligence (AI) software applications. Modern AI applications offer comprehensive multiorgan evaluation. OBJECTIVE. The purpose of this article was to evaluate the impact of an automated AI platform integrated into clinical workflow for chest CT interpretation on radiologists' interpretation times when evaluated in a real-world clinical setting. METHODS. In this prospective single-center study, a commercial AI software solution was integrated into clinical workflow for chest CT interpretation. The software provided automated analysis of cardiac, pulmonary, and musculoskeletal findings, including labeling, segmenting, and measuring normal structures as well as detecting, labeling, and measuring abnormalities. AI-annotated images and autogenerated summary results were stored in the PACS and available to interpreting radiologists. A total of 390 patients (204 women, 186 men; mean age, 62.8 ± 13.3 [SD] years) who underwent out-patient chest CT between January 19, 2021, and January 28, 2021, were included. Scans were randomized using 1:1 allocation between AI-assisted and non-AI-assisted arms and were clinically interpreted by one of three cardiothoracic radiologists (65 scans per arm per radiologist; total of 195 scans per arm) who recorded interpretation times using a stopwatch. Findings were categorized according to review of report impressions. Interpretation times were compared between arms. RESULTS. Mean interpretation times were significantly shorter in the AI-assisted than in the non-AI-assisted arm for all three readers (289 ± 89 vs 344 ± 129 seconds, p < .001; 449 ± 110 vs 649 ± 82 seconds, p < .001; 281 ± 114 vs 348 ± 93 seconds, p = .01) and for readers combined (328 ± 122 vs 421 ± 175 seconds, p < .001). For readers combined, the mean difference was 93 seconds (95% CI, 63-123 seconds), corresponding with a 22.1% reduction in the AI-assisted arm. Mean interpretation time was also shorter in the AI-assisted arm compared with the non-AI-assisted arm for contrast-enhanced scans (83 seconds), noncontrast scans (104 seconds), negative scans (84 seconds), positive scans without significant new findings (117 seconds), and positive scans with significant new findings (92 seconds). CONCLUSION. Cardiothoracic radiologists exhibited a 22.1% reduction in chest CT interpretations times when they had access to results from an automated AI support platform during real-world clinical practice. CLINICAL IMPACT. Integration of the AI support platform into clinical workflow improved radiologist efficiency.

The role of computer-assisted radiographer reporting in lung cancer screening programmes

OBJECTIVES

Successful lung cancer screening delivery requires sensitive, timely reporting of low-dose computed tomography (LDCT) scans, placing a demand on radiology resources. Trained non-radiologist readers and computer-assisted detection (CADe) software may offer strategies to optimise the use of radiology resources without loss of sensitivity. This report examines the accuracy of trained reporting radiographers using CADe support to report LDCT scans performed as part of the Lung Screen Uptake Trial (LSUT).

METHODS

In this observational cohort study, two radiographers independently read all LDCT performed within LSUT and reported on the presence of clinically significant nodules and common incidental findings (IFs), including recommendations for management. Reports were compared against a 'reference standard' (RS) derived from nodules identified by study radiologists without CADe, plus consensus radiologist review of any additional nodules identified by the radiographers.

RESULTS

A total of 716 scans were included, 158 of which had one or more clinically significant pulmonary nodules as per our RS. Radiographer sensitivity against the RS was 68-73.7%, with specificity of 92.1-92.7%. Sensitivity for detection of proven cancers diagnosed from the baseline scan was 83.3-100%. The spectrum of IFs exceeded what could reasonably be covered in radiographer training.

CONCLUSION

Our findings highlight the complexity of LDCT reporting requirements, including the limitations of CADe and the breadth of IFs. We are unable to recommend CADe-supported radiographers as a sole reader of LDCT scans, but propose potential avenues for further research including initial triage of abnormal LDCT or reporting of follow-up surveillance scans.

KEY POINTS

• Successful roll-out of mass screening programmes for lung cancer depends on timely, accurate CT scan reporting, placing a demand on existing radiology resources. • This observational cohort study examines the accuracy of trained radiographers using computer-assisted detection (CADe) software to report lung cancer screening CT scans, as a potential means of supporting reporting workflows in LCS programmes. • CADe-supported radiographers were less sensitive than radiologists at identifying clinically significant pulmonary nodules, but had a low false-positive rate and good sensitivity for detection of confirmed cancers.

Comparative Performance of Deep Learning and Radiologists for the Diagnosis and Localization of Clinically Significant Prostate Cancer at MRI: A Systematic Review

Background: Deep learning (DL)-based models have demonstrated an ability to automatically diagnose clinically significant prostate cancer (PCa) on MRI scans and are regularly reported to approach expert performance. The aim of this work was to systematically review the literature comparing deep learning (DL) systems to radiologists in order to evaluate the comparative performance of current state-of-the-art deep learning models and radiologists. Methods: This systematic review was conducted in accordance with the 2020 Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) checklist. Studies investigating DL models for diagnosing clinically significant (cs) PCa on MRI were included. The quality and risk of bias of each study were assessed using the checklist for AI in medical imaging (CLAIM) and QUADAS-2, respectively. Patient level and lesion-based diagnostic performance were separately evaluated by comparing the sensitivity achieved by DL and radiologists at an identical specificity and the false positives per patient, respectively. Results: The final selection consisted of eight studies with a combined 7337 patients. The median study quality with CLAIM was 74.1% (IQR: 70.6–77.6). DL achieved an identical patient-level performance to the radiologists for PI-RADS ≥ 3 (both 97.7%, SD = 2.1%). DL had a lower sensitivity for PI-RADS ≥ 4 (84.2% vs. 88.8%, p = 0.43). The sensitivity of DL for lesion localization was also between 2% and 12.5% lower than that of the radiologists. Conclusions: DL models for the diagnosis of csPCa on MRI appear to approach the performance of experts but currently have a lower sensitivity compared to experienced radiologists. There is a need for studies with larger datasets and for validation on external data.

Higher agreement between readers with deep learning CAD software for reporting pulmonary nodules on CT

Purpose

The aim was to evaluate the impact of CAD software on the pulmonary nodule management recommendations of radiologists in a cohort of patients with incidentally detected nodules on CT.

Methods

For this retrospective study, two radiologists independently assessed 50 chest CT cases for pulmonary nodules to determine the appropriate management recommendation, twice, unaided and aided by CAD with a 6-month washout period. Management recommendations were given in a 4-point grade based on the BTS guidelines. Both reading sessions were recorded to determine the reading times per case. A reduction in reading times per session was tested with a one-tailed paired t-test, and a linear weighted kappa was calculated to assess interobserver agreement.

Results

The mean age of the included patients was 65.0 ± 10.9. Twenty patients were male (40 %). For both readers 1 and 2, a significant reduction of reading time was observed of 33.4 % and 42.6 % (p < 0.001, p < 0.001). The linear weighted kappa between readers unaided was 0.61. Readers showed a better agreement with the aid of CAD, namely by a kappa of 0.84. The mean reading time per case was 226.4 ± 113.2 and 320.8 ± 164.2 s unaided and 150.8 ± 74.2 and 184.2 ± 125.3 s aided by CAD software for readers 1 and 2, respectively.

Conclusion

A dedicated CAD system for aiding in pulmonary nodule reporting may help improve the uniformity of management recommendations in clinical practice.

Solitary Pulmonary Nodule Evaluation: Pearls and Pitfalls

Lung nodules are frequently encountered while interpreting chest CTs and are challenging to detect, characterize, and manage given they can represent both benign or malignant etiologies. An understanding of features associated with malignancy and causes of interpretive pitfalls is helpful to avoid misdiagnoses. This review addresses pertinent topics related to the etiologies for missed lung nodules on radiography and CT. Additionally, CT imaging technical pitfalls and challenges in addition to issues in the evaluation of nodule morphology, attenuation, and size will be discussed. Nodule management guidelines will be addressed as well as recent investigations that further our understanding of lung nodules.

The Added Effect of Artificial Intelligence on Physicians' Performance in Detecting Thoracic Pathologies on CT and Chest X-ray: A Systematic Review

Our systematic review investigated the additional effect of artificial intelligence-based devices on human observers when diagnosing and/or detecting thoracic pathologies using different diagnostic imaging modalities, such as chest X-ray and CT. Peer-reviewed, original research articles from EMBASE, PubMed, Cochrane library, SCOPUS, and Web of Science were retrieved. Included articles were published within the last 20 years and used a device based on artificial intelligence (AI) technology to detect or diagnose pulmonary findings. The AI-based device had to be used in an observer test where the performance of human observers with and without addition of the device was measured as sensitivity, specificity, accuracy, AUC, or time spent on image reading. A total of 38 studies were included for final assessment. The quality assessment tool for diagnostic accuracy studies (QUADAS-2) was used for bias assessment. The average sensitivity increased from 67.8% to 74.6%; specificity from 82.2% to 85.4%; accuracy from 75.4% to 81.7%; and Area Under the ROC Curve (AUC) from 0.75 to 0.80. Generally, a faster reading time was reported when radiologists were aided by AI-based devices. Our systematic review showed that performance generally improved for the physicians when assisted by AI-based devices compared to unaided interpretation.

Performance and reading time of lung nodule identification on multidetector CT with or without an artificial intelligence-powered computer-aided detection system

AIM

To compare the performance and reading time of different readers using automatic artificial intelligence (AI)-powered computer-aided detection (CAD) to detect lung nodules in different reading modes.

MATERIALS AND METHODS

One hundred and fifty multidetector computed tomography (CT) datasets containing 340 nodules ≤10 mm in diameter were collected retrospectively. A CAD with vessel-suppressed function was used to interpret the images. Three junior and three senior readers were assigned to read (1) CT images without CAD, (2) second-read using CAD in which CAD was applied only after initial unassisted assessment, and (3) a concurrent read with CAD in which CAD was applied at the start of assessment. Diagnostic performances and reading times were compared using analysis of variance.

RESULTS

For all readers, the mean sensitivity improved from 64% (95% confidence interval [CI]: 62%, 66%) for the without-CAD mode to 82% (95% CI: 80%, 84%) for the second-reading mode and to 80% (95% CI: 79%, 82%) for the concurrent-reading mode (p<0.001). There was no significant difference between the two modes in terms of the mean sensitivity, specificity, and area under the receiver operating characteristic curve (AUC) for both junior and senior readers and all readers (p>0.05). The reading time of all readers was significantly shorter for the concurrent-reading mode (124 ± 25 seconds) compared to without CAD (156 ± 34 seconds; p<0.001) and the second-reading mode (197 ± 46 seconds; p<0.001).

CONCLUSION

In CAD for lung nodules at CT, the second-reading mode and concurrent-reading mode may improve detection performance for all readers in both screening and clinical routine practice. Concurrent use of CAD is more efficient for both junior and senior readers.

Can a Novel Deep Neural Network Improve the Computer-Aided Detection of Solid Pulmonary Nodules and the Rate of False-Positive Findings in Comparison to an Established Machine Learning Computer-Aided Detection?

OBJECTIVE

The aim of this study was to compare the performance of 2 approved computer-aided detection (CAD) systems for detection of pulmonary solid nodules (PSNs) in an oncologic cohort. The first CAD system is based on a conventional machine learning approach (VD10F), and the other is based on a deep 3D convolutional neural network (CNN) CAD software (VD20A).

METHODS AND MATERIALS

Nine hundred sixty-seven patients with a total of 2451 PSNs were retrospectively evaluated using the 2 different CAD systems. All patients had thin-slice chest computed tomography (0.6 mm) using 100 kV and 100 mAs and a high-resolution kernel (I50f). The CAD images generated by VD10F were transferred to the PACS for evaluation. The images generated by VD20A were evaluated using a Web browser-based viewer. Finally, a senior radiologist who was blinded for the CAD results examined the thin-slice images of every patient (ground truth).

RESULTS

A total of 2451 PSNs were detected by the senior radiologist. CAD-VD10F detected 1401 true-positive, 143 false-negative, 565 false-positive (FP), and 342 true-negative PSNs, resulting in sensitivity of 90.7%, specificity of 37.7%, positive predictive value of 0.71, and negative predictive value of 0.70. CAD-VD20A detected 1381 true-positive, 163 false-negative, 337 FP, and 570 true-negative PSNs, resulting in sensitivity of 89.4%, specificity of 62.8%, positive predictive value of 0.80, and negative predictive value 0.77, respectively. The rate of FP per scan was 0.6 for CAD-VD10F and 0.3 for CAD-VD20A.

CONCLUSIONS

The new deep learning-based CAD software (VD20A) shows similar sensitivity with the conventional CAD software (VD10F), but a significantly higher specificity.

Incorporation of a computer-aided vessel-suppression system to detect lung nodules in CT images: effect on sensitivity and reading time in routine clinical settings

PURPOSE

To evaluate whether a computer-aided vessel-suppression system improves lung nodule detection in routine clinical settings.

MATERIALS AND METHODS

We used computer software that automatically suppresses pulmonary vessels on chest CT while preserving pulmonary nodules. Sixty-one chest CT images were included in our study. Three radiologists independently read either standard CT images alone or both computer-aided CT and standard CT images randomly to detect a pulmonary nodule ≥ 4 mm in diameter. After an interval of at least 15 days to avoid recall bias, the three radiologists interpreted the counterpart images of the same patients. The reference standard was decided by an expert panel. The primary endpoint was sensitivity. The secondary endpoint was interpretation time.

RESULTS

The average sensitivity improved with computer-aided CT (72% for standard CT vs. 84% for computer-aided CT, p = 0.02). There was no difference in the false-positive rate (21% for both standard CT and computer-aided CT, p = 0.98). Although the average reading time was 9.5% longer for computer-aided plus standard CT compared with standard CT alone, the difference was not significant (p = 0.11).

CONCLUSION

Vessel-suppressed CT images helped radiologists to improve the sensitivity of pulmonary nodule detection without compromising the false-positive rate.

Toward standardized premarket evaluation of computer aided diagnosis/detection products: insights from FDA-approved products

INTRODUCTION

Computer aided detection and diagnosis (CADe and CADx) products are an emerging branch of medical device industry. However, limited technical standard has been developed for product verification and validation. It will be helpful to investigate the current practice of preclinical and clinical evaluation of approved products and provide insights for future standardization.

AREAS COVERED

Document review was conducted on 56 products approved by the United States Food and Drug Administration, including Summary of Safety and Effectiveness Data, 510(k) decision and de novo decision summaries. Key parameters describing product characteristics, preclinical studies and clinical studies were collected. Evaluation strategies for CADe/CADx products were analyzed and assessed.

EXPERT OPINION

Preclinical studies were widely adopted in the verification of CADe/CADx products. Standalone performance testing was a common procedure, but the selection of testing dataset and performance metrics showed significant variability and flexibility among manufacturers. Clinical studies were reported by all class III products and some class II products, and Multi-Reader Multi-Case design was commonly used. However, statistical analysis and presentation/interpretation of results was oftentimes incomplete. To resolve above issues, systematic development of standards of CADe/CADx is encouraged, which can be implemented at different aspects through the product lifecycle.

A practical and adaptive approach to lung cancer screening: a review of international evidence and position on CT lung cancer screening in the Singaporean population by the College of Radiologists Singapore

Lung cancer is the leading cause of cancer-related death around the world, being the top cause of cancer-related deaths among men and the second most common cause of cancer-related deaths among women in Singapore. Currently, no screening programme for lung cancer exists in Singapore. Since there is mounting evidence indicating a different epidemiology of lung cancer in Asian countries, including Singapore, compared to the rest of the world, a unique and adaptive approach must be taken for a screening programme to be successful at reducing mortality while maintaining cost-effectiveness and a favourable risk-benefit ratio. This review article promotes the use of low-dose computed tomography of the chest and explores the radiological challenges and future directions.

AI-based computer-aided diagnosis (AI-CAD): the latest review to read first

The third artificial intelligence (AI) boom is coming, and there is an inkling that the speed of its evolution is quickly increasing. In games like chess, shogi, and go, AI has already defeated human champions, and the fact that it is able to achieve autonomous driving is also being realized. Under these circumstances, AI has evolved and diversified at a remarkable pace in medical diagnosis, especially in diagnostic imaging. Therefore, this commentary focuses on AI in medical diagnostic imaging and explains the recent development trends and practical applications of computer-aided detection/diagnosis using artificial intelligence, especially deep learning technology, as well as some topics surrounding it.

Integration of fully automated computer-aided pulmonary nodule detection into CT pulmonary angiography studies in the emergency department: effect on workflow and diagnostic accuracy

PURPOSE

To assess the feasibility of implementing fully automated computer-aided diagnosis (CAD) for detection of pulmonary nodules on CT pulmonary angiography (CTPA) studies in emergency setting.

MATERIALS AND METHODS

CTPA of 48 emergency patients was retrospectively reviewed. Fully automated CAD nodule detection was performed at the scanner and results were automatically submitted to PACS. A third-year radiology resident (RAD1) and a cardiothoracic radiologist with 6 years' experience (RAD2) reviewed the scans independently to detect pulmonary nodules in two different sessions 8 weeks apart: session 1, CAD was reviewed first and then all images were reviewed; session 2, CAD was reviewed last after all images were reviewed. Time spent by RAD to evaluate image sets was measured for each case. Fisher's exact test and t test were used.

RESULTS

There were 17 male and 31 female patients with mean ± SD age of 48.7 ± 16.4 years. Using CAD at the beginning was associated with lower average reading time for both readers. However, difference in reading time did not reach statistical significance for RAD1 (RAD1 94.6 s vs. 102.7 s, P > 0.05; RAD2 61.1 s vs. 76.5 s, P < 0.05). Using CAD at the end significantly increased rate of RAD1 and RAD2 nodule detection by 34% (2.52 vs. 2.12 nodule/scan, P < 0.05) and 27% (2.23 vs. 1.81 nodule/scan, P < 0.05), respectively.

CONCLUSION

Routine utilization of CAD in emergency setting is feasible and can improve detection rate of pulmonary nodules significantly. Different methods of incorporating CAD in detecting pulmonary nodules can improve both the rate of detection and interpretation speed.

Performance and Reading Time of Automated Breast US with or without Computer-aided Detection

BackgroundComputer-aided detection (CAD) systems may be used to help radiologists interpret automated breast (AB) US images. However, the optimal use of CAD with AB US has, to the knowledge of the authors, not been determined.PurposeTo compare the performance and reading time of different readers by using AB US CAD system to detect breast cancer in different reading modes.Materials and MethodsIn this retrospective study, 1485 AB US images (282 with malignant lesions, 695 with benign lesions, and 508 healthy) in 1452 women (mean age, 43.7 years; age range, 19-82 years) including 529 (36.4%) women who were asymptomatic were collected between 2016 and 2017. A CAD system was used to interpret the images. Three novice readers with 1-3 years of US experience and three experienced readers with 5-10 years of US experience were assigned to read AB US images without CAD, at a second reading (after the reader completed a full unaided interpretation), and at concurrent reading (use of CAD at the start of the assessment). Diagnostic performances and reading times were compared by using analysis of variance.ResultsFor all readers, the mean area under the receiver operating characteristic curve improved from 0.88 (95% confidence interval [CI]: 0.85, 0.91) at without-CAD mode to 0.91 (95% CI: 0.89, 0.92; P < .001) at the second-reading mode and 0.90 (95% CI: 0.89, 0.92; P = .002) at the concurrent-reading mode. The mean sensitivity of novice readers in women who were asymptomatic improved from 67% (95% CI: 63%, 74%) at without-CAD mode to 88% (95% CI: 84%, 89%) at both the second-reading mode and the concurrent-reading mode (P = .003). Compared with the without-CAD and second-reading modes, the mean reading time per volume of concurrent reading was 16 seconds (95% CI: 11, 22; P < .001) and 27 seconds (95% CI: 21, 32; P < .001) shorter, respectively.ConclusionComputer-aided detection (CAD) was helpful for novice readers to improve cancer detection at automated breast US in women who were asymptomatic. CAD was more efficient when used concurrently for all readers.© RSNA, 2019Online supplemental material is available for this article.See also the editorial by Slanetz in this issue.

Integration of Chest CT CAD into the Clinical Workflow and Impact on Radiologist Efficiency

RATIONALE AND OBJECTIVES

The purpose of this paper is to describe the integration of a commercial chest CT computer-aided detection (CAD) system into the clinical radiology reporting workflow and perform an initial investigation of its impact on radiologist efficiency. It seeks to complement research into CAD sensitivity and specificity of stand-alone systems, by focusing on report generation time when the CAD is integrated into the clinical workflow.

MATERIALS AND METHODS

A commercial chest CT CAD software that provides automated detection and measurement of lung nodules, ascending and descending aorta, and pleural effusion was integrated with a commercial radiology report dictation application. The CAD system automatically prepopulated a radiology report template, thus offering the potential for increased efficiency. The integrated system was evaluated using 40 scans from a publicly available lung nodule database. Each scan was read using two methods: (1) without CAD analytics, i.e., manually populated report with measurements using electronic calipers, and (2) with CAD analytics to prepopulate the report for reader review and editing. Three radiologists participated as readers in this study.

RESULTS

CAD assistance reduced reading times by 7%-44%, relative to the conventional manual method, for the three radiologists from opening of the case to signing of the final report.

CONCLUSION

This study provides an investigation of the impact of CAD and measurement on chest CTs within a clinical reporting workflow. Prepopulation of a report with automated nodule and aorta measurements yielded substantial time savings relative to manual measurement and entry.

The Added Value of Computer-aided Detection of Small Pulmonary Nodules and Missed Lung Cancers

Lung cancer at its earliest stage is typically manifested on computed tomography as a pulmonary nodule, which could be detected by low-dose multidetector computed tomography technology and the use of thinner collimation. Within the last 2 decades, computer-aided detection (CAD) of pulmonary nodules has been developed to meet the increasing demand for lung cancer screening computed tomography with a larger set of images per scan. This review introduced the basic techniques and then summarized the up-to-date applications of CAD systems in clinical and research programs and in the low-dose lung cancer screening trials, especially in the detection of small pulmonary nodules and missed lung cancers. Many studies have already shown that the CAD systems could increase the sensitivity and reduce the false-positive rate in the diagnosis of pulmonary nodules, especially for the small and isolated nodules. Further improvements to the current CAD schemes are needed to detect nodules accurately, particularly for subsolid nodules.

Performance and clinical impact of machine learning based lung nodule detection using vessel suppression in melanoma patients

PURPOSE

To evaluate performance and the clinical impact of a novel machine learning based vessel-suppressing computer-aided detection (CAD) software in chest computed tomography (CT) of patients with malignant melanoma.

MATERIALS AND METHODS

We retrospectively included consecutive malignant melanoma patients with a chest CT between 01/2015 and 01/2016. Machine learning based CAD software was used to reconstruct additional vessel-suppressed axial images. Three radiologists independently reviewed a maximum of 15 lung nodules per patient. Vessel-suppressed reconstructions were reviewed independently and results were compared. Follow-up CT examinations and clinical follow-up were used to assess the outcome. Impact of additional nodules on clinical management was assessed.

RESULTS

In 46 patients, vessel-suppressed axial images led to the detection of additional nodules in 25/46 (54.3%) patients. CT or clinical follow up was available in 25/25 (100%) patients with additionally detected nodules. 2/25 (8%) of these patients developed new pulmonary metastases. None of the additionally detected nodules were found to be metastases. None of the lung nodules detected by the radiologists was missed by the CAD software. The mean diameter of the 92 additional nodules was 1.5 ± 0.8 mm. The additional nodules did not affect therapeutic management. However, in 14/46 (30.4%) of patients the additional nodules might have had an impact on the radiological follow-up recommendations.

CONCLUSION

Machine learning based vessel suppression led to the detection of significantly more lung nodules in melanoma patients. Radiological follow-up recommendations were altered in 30% of the patients. However, all lung nodules turned out to be non-malignant on follow-up.

Pulmonary quantitative CT imaging in focal and diffuse disease: current research and clinical applications

The frenetic development of imaging technology-both hardware and software-provides exceptional potential for investigation of the lung. In the last two decades, CT was exploited for detailed characterization of pulmonary structures and description of respiratory disease. The introduction of volumetric acquisition allowed increasingly sophisticated analysis of CT data by means of computerized algorithm, namely quantitative CT (QCT). Hundreds of thousands of CTs have been analysed for characterization of focal and diffuse disease of the lung. Several QCT metrics were developed and tested against clinical, functional and prognostic descriptors. Computer-aided detection of nodules, textural analysis of focal lesions, densitometric analysis and airway segmentation in obstructive pulmonary disease and textural analysis in interstitial lung disease are the major chapters of this discipline. The validation of QCT metrics for specific clinical and investigational needs prompted the translation of such metrics from research field to patient care. The present review summarizes the state of the art of QCT in both focal and diffuse lung disease, including a dedicated discussion about application of QCT metrics as parameters for clinical care and outcomes in clinical trials.

JOURNAL CLUB: Computer-Aided Detection of Lung Nodules on CT With a Computerized Pulmonary Vessel Suppressed Function

OBJECTIVE

The purpose of this study is to evaluate radiologists' performance in detecting actionable nodules on chest CT when aided by a pulmonary vessel image-suppressed function and a computer-aided detection (CADe) system.

MATERIALS AND METHODS

A novel computerized pulmonary vessel image-suppressed function with a built-in CADe (VIS/CADe) system was developed to assist radiologists in interpreting thoracic CT images. Twelve radiologists participated in a comparative study without and with the VIS/CADe using 324 cases (involving 95 cancers and 83 benign nodules). The ratio of nodule-free cases to cases with nodules was 2:1 in the study. Localization ROC (LROC) methods were used for analysis.

RESULTS

In a stand-alone test, the VIS/CADe system detected 89.5% and 82.0% of malignant nodules and all nodules no smaller than 5 mm, respectively. The false-positive rate per CT study was 0.58. For the reader study, the mean area under the LROC curve (LROCAUC) for the detection of lung cancer significantly increased from 0.633 when unaided by VIS/CADe to 0.773 when aided by VIS/CADe (p < 0.01). For the detection of all clinically actionable nodules, the mean LROC-AUC significantly increased from 0.584 when unaided by VIS/CADe to 0.692 when detection was aided by VIS/CADe (p < 0.01). Radiologists detected 80.0% of cancers with VIS/CADe versus 64.45% of cancers unaided (p < 0.01); specificity decreased from 89.9% to 84.4% (p < 0.01). Radiologist interpretation time significantly decreased by 26%.

CONCLUSION

The VIS/CADe system significantly increased radiologists' detection of cancers and actionable nodules with somewhat lower specificity. With use of the VIS/CADe system, radiologists increased their interpretation speed by a factor of approximately one-fourth. Our study suggests that the technique has the potential to assist radiologists in the detection of additional actionable nodules on thoracic CT.

The impact of trained radiographers as concurrent readers on performance and reading time of experienced radiologists in the UK Lung Cancer Screening (UKLS) trial

Objectives

To compare radiologists’ performance reading CTs independently with their performance using radiographers as concurrent readers in lung cancer screening.

Methods

369 consecutive baseline CTs performed for the UK Lung Cancer Screening (UKLS) trial were double-read by radiologists reading either independently or concurrently with a radiographer. In concurrent reading, the radiologist reviewed radiographer-identified nodules and then detected any additional nodules. Radiologists recorded their independent and concurrent reading times. For each radiologist, sensitivity, average false-positive detections (FPs) per case and mean reading times for each method were calculated.

Results

694 nodules in 246/369 (66.7%) studies comprised the reference standard. Radiologists’ mean sensitivity and average FPs per case both increased with concurrent reading compared to independent reading (90.8 ± 5.6% vs. 77.5 ± 11.2%, and 0.60 ± 0.53 vs. 0.33 ± 0.20, respectively; p < 0.05 for 3/4 and 2/4 radiologists, respectively). The mean reading times per case decreased from 9.1 ± 2.3 min with independent reading to 7.2 ± 1.0 min with concurrent reading, decreasing significantly for 3/4 radiologists (p < 0.05).

Conclusions

The majority of radiologists demonstrated improved sensitivity, a small increase in FP detections and a statistically significantly reduced reading time using radiographers as concurrent readers.

Key Points

• Radiographers as concurrent readers could improve radiologists’ sensitivity in lung nodule detection.

• An increase in false-positive detections with radiographer-assisted concurrent reading occurred.

• The false-positive detection rate was still lower than reported for computer-aided detection.

• Concurrent reading with radiographers was also faster than single reading.

• The time saved per case using concurrently reading radiographers was relatively modest.

Electronic supplementary material

The online version of this article (doi:10.1007/s00330-017-4903-z) contains supplementary material, which is available to authorized users.

Computerized detection of lung nodules through radiomics

PURPOSE

Lung cancer is a major cause of cancer deaths, and the 5-year survival rate of stage IV lung cancer patients is only 2%. However, the 5-year survival rate of stage I lung cancer patients significantly increases to 50%. As such, spiral computed tomography (CT) scans are necessary to diagnose high-risk lung cancer patients in early stages. In this study, a computer-aided detection (CAD) system with radiomics was proposed. This system could automatically detect pulmonary nodules and reduce radiologists' workloads and human errors.

METHODS

In the proposed scheme, a nodular enhancement filter was used to segment nodule candidates and extract radiomic features. A synthetic minority over-sampling technique was also applied to balance the samples, and a random forest method was utilized to distinguish between real nodules and false positive detections. The radiomics approach quantified intratumor heterogeneity and multifrequency information, which are highly correlated with lung nodules.

RESULTS

The proposed method was used to evaluate 1004 CT cases from the well-known Lung Image Database Consortium, and 88.9% sensitivity with four false positive detections per CT scan was obtained by randomly selecting 502 cases for training and 502 other cases for testing.

CONCLUSIONS

The proposed scheme yielded a high performance on the LIDC database. Therefore, the proposed scheme is possibly effective for various CT configurations used in routine diagnosis and lung cancer screening.

Role of Computer Aided Diagnosis (CAD) in the detection of pulmonary nodules on 64 row multi detector computed tomography

AIMS AND OBJECTIVES

To determine the overall performance of an existing CAD algorithm with thin-section computed tomography (CT) in the detection of pulmonary nodules and to evaluate detection sensitivity at a varying range of nodule density, size, and location.

MATERIALS AND METHODS

A cross-sectional prospective study was conducted on 20 patients with 322 suspected nodules who underwent diagnostic chest imaging using 64-row multi-detector CT. The examinations were evaluated on reconstructed images of 1.4 mm thickness and 0.7 mm interval. Detection of pulmonary nodules, initially by a radiologist of 2 years experience (RAD) and later by CAD lung nodule software was assessed. Then, CAD nodule candidates were accepted or rejected accordingly. Detected nodules were classified based on their size, density, and location. The performance of the RAD and CAD system was compared with the gold standard that is true nodules confirmed by consensus of senior RAD and CAD together. The overall sensitivity and false-positive (FP) rate of CAD software was calculated.

OBSERVATIONS AND RESULTS

Of the 322 suspected nodules, 221 were classified as true nodules on the consensus of senior RAD and CAD together. Of the true nodules, the RAD detected 206 (93.2%) and 202 (91.4%) by the CAD. CAD and RAD together picked up more number of nodules than either CAD or RAD alone. Overall sensitivity for nodule detection with the CAD program was 91.4%, and FP detection per patient was 5.5%. The CAD showed comparatively higher sensitivity for nodules of size 4-10 mm (93.4%) and nodules in hilar (100%) and central (96.5%) location when compared to RAD's performance.

CONCLUSION

CAD performance was high in detecting pulmonary nodules including the small size and low-density nodules. CAD even with relatively high FP rate, assists and improves RAD's performance as a second reader, especially for nodules located in the central and hilar region and for small nodules by saving RADs time.

Automatic detection of large pulmonary solid nodules in thoracic CT images

PURPOSE

Current computer-aided detection (CAD) systems for pulmonary nodules in computed tomography (CT) scans have a good performance for relatively small nodules, but often fail to detect the much rarer larger nodules, which are more likely to be cancerous. We present a novel CAD system specifically designed to detect solid nodules larger than 10 mm.

METHODS

The proposed detection pipeline is initiated by a three-dimensional lung segmentation algorithm optimized to include large nodules attached to the pleural wall via morphological processing. An additional preprocessing is used to mask out structures outside the pleural space to ensure that pleural and parenchymal nodules have a similar appearance. Next, nodule candidates are obtained via a multistage process of thresholding and morphological operations, to detect both larger and smaller candidates. After segmenting each candidate, a set of 24 features based on intensity, shape, blobness, and spatial context are computed. A radial basis support vector machine (SVM) classifier was used to classify nodule candidates, and performance was evaluated using ten-fold cross-validation on the full publicly available lung image database consortium database.

RESULTS

The proposed CAD system reaches a sensitivity of 98.3% (234/238) and 94.1% (224/238) large nodules at an average of 4.0 and 1.0 false positives/scan, respectively.

CONCLUSIONS

The authors conclude that the proposed dedicated CAD system for large pulmonary nodules can identify the vast majority of highly suspicious lesions in thoracic CT scans with a small number of false positives.

Hybrid detection of lung nodules on CT scan images

PURPOSE

The diversity of lung nodules poses difficulty for the current computer-aided diagnostic (CAD) schemes for lung nodule detection on computed tomography (CT) scan images, especially in large-scale CT screening studies. We proposed a novel CAD scheme based on a hybrid method to address the challenges of detection in diverse lung nodules.

METHODS

The hybrid method proposed in this paper integrates several existing and widely used algorithms in the field of nodule detection, including morphological operation, dot-enhancement based on Hessian matrix, fuzzy connectedness segmentation, local density maximum algorithm, geodesic distance map, and regression tree classification. All of the adopted algorithms were organized into tree structures with multi-nodes. Each node in the tree structure aimed to deal with one type of lung nodule.

RESULTS

The method has been evaluated on 294 CT scans from the Lung Image Database Consortium (LIDC) dataset. The CT scans were randomly divided into two independent subsets: a training set (196 scans) and a test set (98 scans). In total, the 294 CT scans contained 631 lung nodules, which were annotated by at least two radiologists participating in the LIDC project. The sensitivity and false positive per scan for the training set were 87% and 2.61%. The sensitivity and false positive per scan for the testing set were 85.2% and 3.13%.

CONCLUSIONS

The proposed hybrid method yielded high performance on the evaluation dataset and exhibits advantages over existing CAD schemes. We believe that the present method would be useful for a wide variety of CT imaging protocols used in both routine diagnosis and screening studies.

Maximum-Intensity-Projection and Computer-Aided-Detection Algorithms as Stand-Alone Reader Devices in Lung Cancer Screening Using Different Dose Levels and Reconstruction Kernels

OBJECTIVE

The objective of our study was to evaluate lung nodule detection rates on standard and microdose chest CT with two different computer-aided detection systems (SyngoCT-CAD, VA 20, Siemens Healthcare [CAD1]; Lung CAD, IntelliSpace Portal DX Server, Philips Healthcare [CAD2]) as well as maximum-intensity-projection (MIP) images. We also assessed the impact of different reconstruction kernels.

MATERIALS AND METHODS

Standard and microdose CT using three reconstruction kernels (i30, i50, i70) was performed with an anthropomorphic chest phantom. We placed 133 ground-glass and 133 solid nodules (diameters of 5 mm, 8 mm, 10 mm, and 12 mm) in 55 phantoms. Four blinded readers evaluated the MIP images; one recorded the results of CAD1 and CAD2. Sensitivities for CAD and MIP nodule detection on standard dose and microdose CT were calculated for each reconstruction kernel.

RESULTS

Dose for microdose CT was significantly less than that for standard-dose CT (0.1323 mSv vs 1.65 mSv; p < 0.0001). CAD1 delivered superior results compared with CAD2 for standard-dose and microdose CT (p < 0.0001). At microdose level, the best stand-alone sensitivity (97.6%) was comparable with CAD1 sensitivity (96.0%; p = 0.36; both with i30 reconstruction kernel). Pooled sensitivities for all nodules, doses, and reconstruction kernels on CAD1 ranged from 88.9% to 97.3% versus 49.6% to 73.9% for CAD2. The best sensitivity was achieved with standard-dose CT, i50 kernel, and CAD1 (97.3%) versus 96% with microdose CT, i30 or i50 kernel, and CAD1. MIP images and CAD1 had similar performance at both dose levels (p = 0.1313 and p = 0.48).

CONCLUSION

Submillisievert CT is feasible for detecting solid and ground-glass nodules that require soft-tissue kernels for MIP and CAD systems to achieve acceptable sensitivities. MIP reconstructions remain a valuable adjunct to the interpretation of chest CT for increasing sensitivity and have the advantage of significantly lower false-positive rates.

[Computed tomography with computer-assisted detection of pulmonary nodules in dogs and cats]

OBJECTIVE

The aim of this study was to assess the potential benefit of computer-assisted detection (CAD) of pulmonary nodules in veterinary medicine. Therefore, the CAD rate was compared to the detection rates of two individual examiners in terms of its sensitivity and false-positive findings.

MATERIALS AND METHODS

We included 51 dogs and 16 cats with pulmonary nodules previously diagnosed by computed tomography. First, the number of nodules ≥ 3 mm was recorded for each patient by two independent examiners. Subsequently, each examiner used the CAD software for automated nodule detection. With the knowledge of the CAD results, a final consensus decision on the number of nodules was achieved. The software used was a commercially available CAD program.

RESULTS

The sensitivity of examiner 1 was 89.2%, while that of examiner 2 reached 87.4%. CAD had a sensitivity of 69.4%. With CAD, the sensitivity of examiner 1 increased to 94.7% and that of examiner 2 to 90.8%.

CONCLUSION AND CLINICAL RELEVANCE

The CAD-system, which we used in our study, had a moderate sensitivity of 69.4%. Despite its severe limitations, with a high level of false-positive and false-negative results, CAD increased the examiners' sensitivity. Therefore, its supportive role in diagnostics appears to be evident.

Improving the radiologist-CAD interaction: designing for appropriate trust

Computer-aided diagnosis (CAD) has great potential to improve radiologists' diagnostic performance. However, the reported performance of the radiologist-CAD team is lower than what might be expected based on the performance of the radiologist and the CAD system in isolation. This indicates that the interaction between radiologists and the CAD system is not optimal. An important factor in the interaction between humans and automated aids (such as CAD) is trust. Suboptimal performance of the human-automation team is often caused by an inappropriate level of trust in the automation. In this review, we examine the role of trust in the radiologist-CAD interaction and suggest ways to improve the output of the CAD system so that it allows radiologists to calibrate their trust in the CAD system more effectively. Observer studies of the CAD systems show that radiologists often have an inappropriate level of trust in the CAD system. They sometimes under-trust CAD, thereby reducing its potential benefits, and sometimes over-trust it, leading to diagnostic errors they would not have made without CAD. Based on the literature on trust in human-automation interaction and the results of CAD observer studies, we have identified four ways to improve the output of CAD so that it allows radiologists to form a more appropriate level of trust in CAD. Designing CAD systems for appropriate trust is important and can improve the performance of the radiologist-CAD team. Future CAD research and development should acknowledge the importance of the radiologist-CAD interaction, and specifically the role of trust therein, in order to create the perfect artificial partner for the radiologist. This review focuses on the role of trust in the radiologist-CAD interaction. The aim of the review is to encourage CAD developers to design for appropriate trust and thereby improve the performance of the radiologist-CAD team.

Fate of pulmonary nodules detected by computer-aided diagnosis and physician review on the computed tomography simulation images for hepatocellular carcinoma

Purpose

To investigate the frequency and clinical significance of detected incidental lung nodules found on computed tomography (CT) simulation images for hepatocellular carcinoma (HCC) using computer-aided diagnosis (CAD) and a physician review.

Materials and Methods

Sixty-seven treatment-naïve HCC patients treated with transcatheter arterial chemoembolization and radiotherapy (RT) were included for the study. Portal phase of simulation CT images was used for CAD analysis and a physician review for lung nodule detection. For automated nodule detection, a commercially available CAD system was used. To assess the performance of lung nodule detection for lung metastasis, the sensitivity, negative predictive value (NPV), and positive predictive value (PPV) were calculated.

Results

Forty-six patients had incidental nodules detected by CAD with a total of 109 nodules. Only 20 (18.3%) nodules were considered to be significant nodules by a physician review. The number of significant nodules detected by both of CAD or a physician review was 24 in 9 patients. Lung metastases developed in 11 of 46 patients who had any type of nodule. The sensitivities were 58.3% and 100% based on patient number and on the number of nodules, respectively. The NPVs were 91.4% and 100%, respectively. And the PPVs were 77.8% and 91.7%, respectively.

Conclusion

Incidental detection of metastatic nodules was not an uncommon event. From our study, CAD could be applied to CT simulation images allowing for an increase in detection of metastatic nodules.

CT screening for lung cancer: countdown to implementation

Implementation of lung cancer CT screening is currently the subject of a major policy decision within the USA. Findings of the US National Lung Screening Trial showed a 20% reduction in lung cancer mortality and a 6·7% decrease in all-cause mortality; subsequently, five US professional and clinical organisations and the US Preventive Services Task Force recommended that screening should be implemented. Should national health services in Europe follow suit? The European community awaits mortality and cost-effectiveness data from the NELSON trial in 2015-16 and pooled findings of European trials. In the intervening years, a recommendation is proposed that a demonstration trial is done in the UK. In this Review, we summarise the existing evidence and identify questions that remain to be answered before the implementation of international lung cancer screening programmes.

Development and evaluation of a software tool for the generation of virtual liver lesions in multidetector-row CT datasets

RATIONALE AND OBJECTIVES

Development and evaluation of a software tool for the insertion of simulated hypodense liver lesions in multidetector-row computed tomography (CT) datasets.

MATERIALS AND METHODS

Forty software-generated hypodense liver lesions were inserted at random locations in 20 CT datasets by using the "alpha blending" technique and compared with 40 real metastatic lesions. The location, diameter (5-20 mm) and density of the simulated lesions were individually adjusted to closely resemble real lesions in each patient. Three blinded readers evaluated all 80 lesions twice in a 2-week interval using a five-point Likert confidence scale under standardized conditions. Nonparametric tests were used to statistically evaluate possible differences in scoring between real and simulated lesions. The correctness of the observer rating for real and simulated lesions was compared to chance distribution using the chi-squared statistics. The inter- and intraobserver variability was determined using Kendall's coefficient of concordance.

RESULTS

The observer study did not reveal significant differences between the scoring for real versus simulated lesions for any of the readers (P > .05). The distribution of correct and false scoring of the lesions was not significantly different from chance distribution (P > .05). Inter- and intraobserver agreement was poor (Kendall W coefficient = 0.12/0.13).

CONCLUSION

The proposed algorithm is suitable for creating realistic virtual liver lesions in CT datasets.

High performance lung nodule detection schemes in CT using local and global information

PURPOSE

A key issue in current computer-aided diagnostic (CAD) schemes for nodule detection in CT is the large number of false positives, because current schemes use only global three-dimensional (3D) information to detect nodules and discard useful local two-dimensional (2D) information. Thus, the authors integrated local and global information to markedly improve the performance levels of CAD schemes.

METHODS

Our database was obtained from the standard CT lung nodule database created by the Lung Image Database Consortium (LIDC). It consisted of 85 CT scans with 111 nodules of 3 mm or larger in diameter. The 111 nodules were confirmed by at least two of the four radiologists participated in the LIDC. Twenty-six nodules were missed by two of the four radiologists and were thus very difficult to detect. The authors developed five CAD schemes for nodule detection in CT using global 3D information (3D scheme), local 2D information (2D scheme), and both local and global information (2D + 3D scheme, 2D - 3D scheme, and 3D - 2D scheme). The 3D scheme, which was developed previously, used only global 3D information and discarded local 2D information, as other CAD schemes did. The 2D scheme used a uniform viewpoint reformation technique to decompose a 3D nodule candidate into a set of 2D reformatted images generated from representative viewpoints, and selected and used "effective" 2D reformatted images to remove false positives. The 2D + 3D scheme, 2D - 3D scheme, and 3D - 2D scheme used complementary local and global information in different ways to further improve the performance of lung nodule detection. The authors employed a leave-one-scan-out testing method for evaluation of the performance levels of the five CAD schemes.

RESULTS

At the sensitivities of 85%, 80%, and 75%, the existing 3D scheme reported 17.3, 7.4, and 2.8 false positives per scan, respectively; the 2D scheme improved the detection performance and reduced the numbers of false positives to 7.6, 2.5, and 1.3 per scan; the 2D + 3D scheme further reduced those to 2.7, 1.9, and 0.6 per scan; the 2D - 3D scheme reduced those to 7.6, 2.1, and 0.8 per scan; and the 3D - 2D scheme reduced those to 17.3, 1.6, and 1.0 per scan.

CONCLUSIONS

The local 2D information appears to be more useful than the global 3D information for nodule detection, particularly, when it is integrated with 3D information.

Performance of computer-aided detection of pulmonary nodules in low-dose CT: comparison with double reading by nodule volume

Objective

To evaluate performance of computer-aided detection (CAD) beyond double reading for pulmonary nodules on low-dose computed tomography (CT) by nodule volume.

Methods

A total of 400 low-dose chest CT examinations were randomly selected from the NELSON lung cancer screening trial. CTs were evaluated by two independent readers and processed by CAD. A total of 1,667 findings marked by readers and/or CAD were evaluated by a consensus panel of expert chest radiologists. Performance was evaluated by calculating sensitivity of pulmonary nodule detection and number of false positives, by nodule characteristics and volume.

Results

According to the screening protocol, 90.9 % of the findings could be excluded from further evaluation, 49.2 % being small nodules (less than 50 mm³). Excluding small nodules reduced false-positive detections by CAD from 3.7 to 1.9 per examination. Of 151 findings that needed further evaluation, 33 (21.9 %) were detected by CAD only, one of them being diagnosed as lung cancer the following year. The sensitivity of nodule detection was 78.1 % for double reading and 96.7 % for CAD. A total of 69.7 % of nodules undetected by readers were attached nodules of which 78.3 % were vessel-attached.

Conclusions

CAD is valuable in lung cancer screening to improve sensitivity of pulmonary nodule detection beyond double reading, at a low false-positive rate when excluding small nodules.

Key Points

• Computer-aided detection (CAD) has known advantages for computed tomography (CT).

• Combined CAD/nodule size cut-off parameters assist CT lung cancer screening.

• This combination improves the sensitivity of pulmonary nodule detection by CT.

• It increases the positive predictive value for cancer detection.