Computer-aided diagnosis for detection of interstitial opacities on chest radiographs

Automated detection of acute respiratory distress syndrome from chest X-Rays using Directionality Measure and deep learning features

Acute respiratory distress syndrome (ARDS) is a life-threatening lung injury with global prevalence and high mortality. Chest x-rays (CXR) are critical in the early diagnosis and treatment of ARDS. However, imaging findings may not result in proper identification of ARDS due to a number of reasons, including nonspecific appearance of radiological features, ambiguity in a patient's case due to the pathological stage of the disease, and poor inter-rater reliability from interpretations of CXRs by multiple clinical experts. This study demonstrates the potential capability of methodologies in artificial intelligence, machine learning, and image processing to overcome these challenges and quantitatively assess CXRs for presence of ARDS. We propose and describe Directionality Measure, a novel feature engineering technique used to capture the "cloud-like" appearance of diffuse alveolar damage as a mathematical concept. This study also examines the effectiveness of using an off-the-shelf, pretrained deep learning model as a feature extractor in addition to standard features extracted from the histogram and gray-level co-occurrence matrix (GLCM). Data was collected from hospitalized patients at Michigan Medicine's intensive care unit and the cohort's inclusion criteria was specifically designed to be representative of patients at risk of developing ARDS. Multiple machine learning models were used to evaluate these features with 5-fold cross-validation and the final performance was reported on a hold-out, temporally distinct test set. With AdaBoost, Directionality Measure achieved an accuracy of 78% and AUC of 74% - outperforming classification results using features from the histogram (75% accuracy and 73% AUC), GLCM (76% accuracy and 73% AUC), and ResNet-50 (77% accuracy and 73% AUC). Further experimental results demonstrated that using all feature sets in combination achieved the best overall performance, yielding an accuracy of 83% and AUC of 79% with AdaBoost. These results demonstrate the potential capability of using the proposed methodologies to complement current clinical analysis for detection of ARDS from CXRs.

Automated Cerebral Hemorrhage Detection Using RAPID

BACKGROUND AND PURPOSE

Intracranial hemorrhage (ICH) is an important event that is diagnosed on head NCCT. Increased NCCT utilization in busy hospitals may limit timely identification of ICH. RAPID ICH is an automated hybrid 2D-3D convolutional neural network application designed to detect ICH that may allow for expedited ICH diagnosis. We determined the accuracy of RAPID ICH for ICH detection and ICH volumetric quantification on NCCT.

MATERIALS AND METHODS

NCCT scans were evaluated for ICH by RAPID ICH. Consensus detection of ICH by 3 neuroradiology experts was used as the criterion standard for RAPID ICH comparison. ICH volume was also automatically determined by RAPID ICH in patients with intraparenchymal or intraventricular hemorrhage and compared with manually segmented ICH volumes by a single neuroradiology expert. ICH detection accuracy, sensitivity, specificity, positive predictive value, negative predictive value, and positive and negative likelihood ratios by RAPID ICH were determined.

RESULTS

We included 308 studies. RAPID ICH correctly identified 151/158 ICH cases and 143/150 ICH-negative cases, which resulted in high sensitivity (0.956, CI: 0.911-0.978), specificity (0.953, CI: 0.907-0.977), positive predictive value (0.956, CI: 0.911-0.978), and negative predictive value (0.953, CI: 0.907-0.977) for ICH detection. The positive likelihood ratio (20.479, CI 9.928-42.245) and negative likelihood ratio (0.046, CI 0.023-0.096) for ICH detection were similarly favorable. RAPID ICH volumetric quantification for intraparenchymal and intraventricular hemorrhages strongly correlated with expert manual segmentation (correlation coefficient r = 0.983); the median absolute error was 3 mL.

CONCLUSIONS

RAPID ICH is highly accurate in the detection of ICH and in the volumetric quantification of intraparenchymal and intraventricular hemorrhages.

Transfer-to-Transfer Learning Approach for Computer Aided Detection of COVID-19 in Chest Radiographs

The coronavirus disease 2019 (COVID-19) global pandemic has severely impacted lives across the globe. Respiratory disorders in COVID-19 patients are caused by lung opacities similar to viral pneumonia. A Computer-Aided Detection (CAD) system for the detection of COVID-19 using chest radiographs would provide a second opinion for radiologists. For this research, we utilize publicly available datasets that have been marked by radiologists into two-classes (COVID-19 and non-COVID-19). We address the class imbalance problem associated with the training dataset by proposing a novel transfer-to-transfer learning approach, where we break a highly imbalanced training dataset into a group of balanced mini-sets and apply transfer learning between these. We demonstrate the efficacy of the method using well-established deep convolutional neural networks. Our proposed training mechanism is more robust to limited training data and class imbalance. We study the performance of our algorithm(s) based on 10-fold cross validation and two hold-out validation experiments to demonstrate its efficacy. We achieved an overall sensitivity of 0.94 for the hold-out validation experiments containing 2265 and 2139 marked as COVID-19 chest radiographs, respectively. For the 10-fold cross validation experiment, we achieve an overall Area under the Receiver Operating Characteristic curve (AUC) value of 0.996 for COVID-19 detection. This paper serves as a proof-of-concept that an automated detection approach can be developed with a limited set of COVID-19 images, and in areas with scarcity of trained radiologists.

Use of deep learning to detect cardiomegaly on thoracic radiographs in dogs

The purpose of this study was to develop a computer-aided detection (CAD) device based on convolutional neural networks (CNNs) to detect cardiomegaly from plain radiographs in dogs. Right lateral chest radiographs (n = 1465) were retrospectively selected from archives. The radiographs were classified as having a normal cardiac silhouette (No-vertebral heart scale [VHS]-Cardiomegaly) or an enlarged cardiac silhouette (VHS-Cardiomegaly) based on the breed-specific VHS. The database was divided into a training set (1153 images) and a test set (315 images). The diagnostic accuracy of four different CNN models in the detection of cardiomegaly was calculated using the test set. All tested models had an area under the curve >0.9, demonstrating high diagnostic accuracy. There was a statistically significant difference between Model C and the remainder models (Model A vs. Model C, P = 0.0298; Model B vs. Model C, P = 0.003; Model C vs. Model D, P = 0.0018), but there were no significant differences between other combinations of models (Model A vs. Model B, P = 0.395; Model A vs. Model D, P = 0.128; Model B vs. Model D, P = 0.373). Convolutional neural networks could therefore assist veterinarians in detecting cardiomegaly in dogs from plain radiographs.

Visualization and Interpretation of Convolutional Neural Network Predictions in Detecting Pneumonia in Pediatric Chest Radiographs

Pneumonia affects 7% of the global population, resulting in 2 million pediatric deaths every year. Chest X-ray (CXR) analysis is routinely performed to diagnose the disease. Computer-aided diagnostic (CADx) tools aim to supplement decision-making. These tools process the handcrafted and/or convolutional neural network (CNN) extracted image features for visual recognition. However, CNNs are perceived as black boxes since their performance lack explanations. This is a serious bottleneck in applications involving medical screening/diagnosis since poorly interpreted model behavior could adversely affect the clinical decision. In this study, we evaluate, visualize, and explain the performance of customized CNNs to detect pneumonia and further differentiate between bacterial and viral types in pediatric CXRs. We present a novel visualization strategy to localize the region of interest (ROI) that is considered relevant for model predictions across all the inputs that belong to an expected class. We statistically validate the models' performance toward the underlying tasks. We observe that the customized VGG16 model achieves 96.2% and 93.6% accuracy in detecting the disease and distinguishing between bacterial and viral pneumonia respectively. The model outperforms the state-of-the-art in all performance metrics and demonstrates reduced bias and improved generalization.

Advanced machine learning in action: identification of intracranial hemorrhage on computed tomography scans of the head with clinical workflow integration

Intracranial hemorrhage (ICH) requires prompt diagnosis to optimize patient outcomes. We hypothesized that machine learning algorithms could automatically analyze computed tomography (CT) of the head, prioritize radiology worklists and reduce time to diagnosis of ICH. 46,583 head CTs (~2 million images) acquired from 2007–2017 were collected from several facilities across Geisinger. A deep convolutional neural network was trained on 37,074 studies and subsequently evaluated on 9499 unseen studies. The predictive model was implemented prospectively for 3 months to re-prioritize “routine” head CT studies as “stat” on realtime radiology worklists if an ICH was detected. Time to diagnosis was compared between the re-prioritized “stat” and “routine” studies. A neuroradiologist blinded to the study reviewed false positive studies to determine whether the dictating radiologist overlooked ICH. The model achieved an area under the ROC curve of 0.846 (0.837–0.856). During implementation, 94 of 347 “routine” studies were re-prioritized to “stat”, and 60/94 had ICH identified by the radiologist. Five new cases of ICH were identified, and median time to diagnosis was significantly reduced (p < 0.0001) from 512 to 19 min. In particular, one outpatient with vague symptoms on anti-coagulation was found to have an ICH which was treated promptly with reversal of anticoagulation, resulting in a good clinical outcome. Of the 34 false positives, the blinded over-reader identified four probable ICH cases overlooked in original interpretation. In conclusion, an artificial intelligence algorithm can prioritize radiology worklists to reduce time to diagnosis of new outpatient ICH by 96% and may also identify subtle ICH overlooked by radiologists. This demonstrates the positive impact of advanced machine learning in radiology workflow optimization.

A computer program that automatically analyzes brain images from patients undergoing CT scans of the head can reliably flag those with signs of hemorrhage. A team of researchers from Geisinger in Danville, Pennsylvania, USA, trained and tested a machine-learning algorithm using 46,583 computed tomography imaging studies of the head. Subsequently, they implemented the model into routine care for 3 months to help prioritize radiology worklists. Of 347 routine cases, the computer identified 94 as having an intracranial hemorrhage, two-thirds of which were confirmed by a radiologist, including five among patients who had a new diagnosis of a brain bleed. The algorithm reduced the average time in which a radiologist diagnosed these patients from around 8.5 h to just 19 min, demonstrating the positive impact of incorporating artificial intelligence into radiology workflow.

Computer-Aided Diagnosis for Chest Radiographs in Intensive Care

The chest radiograph is an essential tool for the diagnosis of several lung diseases in intensive care units (ICU). However, several factors make the interpretation of the chest radiograph difficult including the number of X-rays done daily in ICU, the quality of the chest radiograph, and the lack of a standardized interpretation. To overcome these limitations in the interpretation of chest radiographs, researchers have developed computer-aided diagnosis (CAD) systems. In this review, the authors report the methodology used to develop CAD systems including identification of the region of interest, analysis of these regions, and classification. Currently, only a few CAD systems for chest X-ray interpretation are commercially available. Some promising research is ongoing, but the involvement of the pediatric research community is needed for the development and validation of such CAD systems dedicated to pediatric intensive care.

Computer-aided detection system for chest radiography: reducing report turnaround times of examinations with abnormalities

BACKGROUND

The ability to give high priority to examinations with pathological findings could be very useful to radiologists with large work lists who wish to first evaluate the most critical studies. A computer-aided detection (CAD) system for identifying chest examinations with abnormalities has therefore been developed.

PURPOSE

To evaluate the effectiveness of a CAD system on report turnaround times of chest examinations with abnormalities.

MATERIAL AND METHODS

The CAD system was designed to automatically mark chest examinations with possible abnormalities in the work list of radiologists interpreting chest examinations. The system evaluation was performed in two phases: two radiologists interpreted the chest examinations without CAD in phase 1 and with CAD in phase 2. The time information recorded by the radiology information system was then used to calculate the turnaround times. All chest examinations were reviewed by two other radiologists and were divided into normal and abnormal groups. The turnaround times for the examinations with pathological findings with and without the CAD system assistance were compared.

RESULTS

The sensitivity and specificity of the CAD for chest abnormalities were 0.790 and 0.697, respectively, and use of the CAD system decreased the turnaround time for chest examinations with abnormalities by 44%.

CONCLUSION

The turnaround times required for radiologists to identify chest examinations with abnormalities could be reduced by using the CAD system. This system could be useful for radiologists with large work lists who wish to first evaluate the most critical studies.

Potential clinical impact of advanced imaging and computer-aided diagnosis in chest radiology: importance of radiologist's role and successful observer study

This review paper is based on our research experience in the past 30 years. The importance of radiologists' role is discussed in the development or evaluation of new medical images and of computer-aided detection (CAD) schemes in chest radiology. The four main topics include (1) introducing what diseases can be included in a research database for different imaging techniques or CAD systems and what imaging database can be built by radiologists, (2) understanding how radiologists' subjective judgment can be combined with technical objective features to improve CAD performance, (3) sharing our experience in the design of successful observer performance studies, and (4) finally, discussing whether the new images and CAD systems can improve radiologists' diagnostic ability in chest radiology. In conclusion, advanced imaging techniques and detection/classification of CAD systems have a potential clinical impact on improvement of radiologists' diagnostic ability, for both the detection and the differential diagnosis of various lung diseases, in chest radiology.

Development of CAD based on ANN analysis of power spectra for pneumoconiosis in chest radiographs: effect of three new enhancement methods

We have been developing a computer-aided detection (CAD) scheme for pneumoconiosis based on a rule-based plus artificial neural network (ANN) analysis of power spectra. In this study, we have developed three enhancement methods for the abnormal patterns to reduce false-positive and false-negative values. The image database consisted of 2 normal and 15 abnormal chest radiographs. The International Labour Organization standard chest radiographs with pneumoconiosis were categorized as subcategory, size, and shape of pneumoconiosis. Regions of interest (ROIs) with a matrix size of 32 × 32 were selected from normal and abnormal lungs. Three new enhanced methods were obtained by window function, top-hat transformation, and gray-level co-occurrence matrix analysis. We calculated the power spectrum (PS) of all ROIs by Fourier transform. For the classification between normal and abnormal ROIs, we applied a combined analysis using the ruled-based plus the ANN method. To evaluate the overall performance of this CAD scheme, we employed ROC analysis for distinguishing between normal and abnormal ROIs. On the chest radiographs of the highest categories (severe pneumoconiosis) and the lowest categories (early pneumoconiosis), this CAD scheme achieved area under the curve (AUC) values of 0.93 ± 0.02 and 0.72 ± 0.03. The combined rule-based plus ANN method with the three new enhanced methods obtained the highest classification performance for distinguishing between abnormal and normal ROIs. Our CAD system based on the three new enhanced methods would be useful in assisting radiologists in the classification of pneumoconiosis.

Computerized analysis of pneumoconiosis in digital chest radiography: effect of artificial neural network trained with power spectra

It is difficult for radiologists to classify pneumoconiosis with small nodules on chest radiographs. Therefore, we have developed a computer-aided diagnosis (CAD) system based on the rule-based plus artificial neural network (ANN) method for distinction between normal and abnormal regions of interest (ROIs) selected from chest radiographs with and without pneumoconiosis. The image database consists of 11 normal and 12 abnormal chest radiographs. These abnormal cases included five silicoses, four asbestoses, and three other pneumoconioses. ROIs (matrix size, 32 × 32) were selected from normal and abnormal lungs. We obtained power spectra (PS) by Fourier transform for the frequency analysis. A rule-based method using PS values at 0.179 and 0.357 cycles per millimeter, corresponding to the spatial frequencies of nodular patterns, were employed for identification of obviously normal or obviously abnormal ROIs. Then, ANN was applied for classification of the remaining normal and abnormal ROIs, which were not classified as obviously abnormal or normal by the rule-based method. The classification performance was evaluated by the area under the receiver operating characteristic curve (Az value). The Az value was 0.972 ± 0.012 for the rule-based plus ANN method, which was larger than that of 0.961 ± 0.016 for the ANN method alone (P ≤ 0.15) and that of 0.873 for the rule-based method alone. We have developed a rule-based plus pattern recognition technique based on the ANN for classification of pneumoconiosis on chest radiography. Our CAD system based on PS would be useful to assist radiologists in the classification of pneumoconiosis.

Computer-aided diagnosis in medical imaging: historical review, current status and future potential

Computer-aided diagnosis (CAD) has become one of the major research subjects in medical imaging and diagnostic radiology. In this article, the motivation and philosophy for early development of CAD schemes are presented together with the current status and future potential of CAD in a PACS environment. With CAD, radiologists use the computer output as a "second opinion" and make the final decisions. CAD is a concept established by taking into account equally the roles of physicians and computers, whereas automated computer diagnosis is a concept based on computer algorithms only. With CAD, the performance by computers does not have to be comparable to or better than that by physicians, but needs to be complementary to that by physicians. In fact, a large number of CAD systems have been employed for assisting physicians in the early detection of breast cancers on mammograms. A CAD scheme that makes use of lateral chest images has the potential to improve the overall performance in the detection of lung nodules when combined with another CAD scheme for PA chest images. Because vertebral fractures can be detected reliably by computer on lateral chest radiographs, radiologists' accuracy in the detection of vertebral fractures would be improved by the use of CAD, and thus early diagnosis of osteoporosis would become possible. In MRA, a CAD system has been developed for assisting radiologists in the detection of intracranial aneurysms. On successive bone scan images, a CAD scheme for detection of interval changes has been developed by use of temporal subtraction images. In the future, many CAD schemes could be assembled as packages and implemented as a part of PACS. For example, the package for chest CAD may include the computerized detection of lung nodules, interstitial opacities, cardiomegaly, vertebral fractures, and interval changes in chest radiographs as well as the computerized classification of benign and malignant nodules and the differential diagnosis of interstitial lung diseases. In order to assist in the differential diagnosis, it would be possible to search for and retrieve images (or lesions) with known pathology, which would be very similar to a new unknown case, from PACS when a reliable and useful method has been developed for quantifying the similarity of a pair of images for visual comparison by radiologists.

Computer-aided diagnosis in chest radiography

We have developed computer-aided diagnosis (CAD) schemes for the detection of lung nodules, interstitial lung diseases, interval changes, and asymmetric opacities, and also for the differential diagnosis of lung nodules and interstitial lung diseases on chest radiographs. Observer performance studies indicate clearly that radiologists' diagnostic accuracy was improved significantly when radiologists used a computer output in their interpretations of chest radiographs. In addition, the automated recognition methods for the patient and the projection view by use of chest radiographs were useful for integrating the chest CAD schemes into the picture-archiving and communication system (PACS).

Recent Advances in Chest Radiography

There have been many remarkable advances in conventional thoracic imaging over the past decade. Perhaps the most remarkable is the rapid conversion from film-based to digital radiographic systems. Computed radiography is now the preferred imaging modality for bedside chest imaging. Direct radiography is rapidly replacing film-based chest units for in-department posteroanterior and lateral examinations. An exciting aspect of the conversion to digital radiography is the ability to enhance the diagnostic capabilities and influence of chest radiography. Opportunities for direct computer-aided detection of various lesions may enhance the radiologist's accuracy and improve efficiency. Newer techniques such as dual-energy and temporal subtraction radiography show promise for improved detection of subtle and often obscured or overlooked lung lesions. Digital tomosynthesis is a particularly promising technique that allows reconstruction of multisection images from a short acquisition at very low patient dose. Preliminary data suggest that, compared with conventional radiography, tomosynthesis may also improve detection of subtle lung lesions. The ultimate influence of these new technologies will, of course, depend on the outcome of rigorous scientific validation.

Competency in chest radiography. A comparison of medical students, residents, and fellows

BACKGROUND

Accurate interpretation of chest radiographs (CXR) is essential as clinical decisions depend on readings.

OBJECTIVE

We sought to evaluate CXR interpretation ability at different levels of training and to determine factors associated with successful interpretation.

DESIGN

Ten CXR were selected from the teaching file of the internal medicine (IM) department. Participants were asked to record the most important diagnosis, their certainty in that diagnosis, interest in a pulmonary career and adequacy of CXR training. Two investigators independently scored each CXR on a scale of 0 to 2.

PARTICIPANTS

Participants (n=145) from a single teaching hospital were third year medical students (MS) (n=25), IM interns (n=44), IM residents (n=45), fellows from the divisions of cardiology and pulmonary/critical care (n=16), and radiology residents (n=15).

RESULTS

The median overall score was 11 of 20. An increased level of training was associated with overall score (MS 8, intern 10, IM resident 13, fellow 15, radiology resident 18, P<.001). Overall certainty was significantly correlated with overall score (r=.613, P<.001). Internal medicine interns and residents interested in a pulmonary career scored 14 of 20 while those not interested scored 11 (P=.027). Pneumothorax, misplaced central line, and pneumoperitoneum were diagnosed correctly 9%, 26%, and 46% of the time, respectively. Only 20 of 131 (15%) participants felt their CXR training sufficient.

CONCLUSION

We identified factors associated with successful CXR interpretation, including level of training, field of training, interest in a pulmonary career and overall certainty. Although interpretation improved with training, important diagnoses were missed.

Comparison of independent double readings and computer-aided diagnosis (CAD) for the diagnosis of breast calcifications

RATIONALE AND OBJECTIVES

The aim of the study is to compare independent double readings by radiologists and computer-aided diagnosis (CAD) in diagnostic interpretation of mammographic calcifications.

MATERIALS AND METHODS

Ten radiologists independently interpreted 104 mammograms containing clustered microcalcifications. Forty-six of these were malignant and 58 were benign at biopsy. Radiologists read the images with and without a computer aid by using a counterbalanced study design. Sensitivity and specificity were calculated from observer biopsy recommendations, and receiver operating characteristic (ROC) curves were computed from their diagnostic confidence ratings. Unaided double-reading sensitivity and specificity values were derived post hoc by using three different objective rules and an additional rule of simulated-optimal double reading that assumed that consultations for resolving two radiologists' different independent diagnoses always produce the correct clinical recommendation. ROC curves of unaided double readings were obtained according to the literature.

RESULTS

Single reading without computer aid yielded 74% sensitivity and 32% specificity, whereas CAD reading yielded 87% sensitivity and 42% specificity and appeared on a higher ROC curve (P < .0001). Three methods of formulating independent double readings generated sensitivities between 59% and 89%, specificities between 50% and 13%, and operating points that moved essentially along the average unaided single-reading ROC curve. ROC curves of unaided independent double readings showed small, statistically insignificant improvement over those of unaided single readings. Results of the simulated-optimal double reading were similar to CAD: 89% sensitivity and 50% specificity.

CONCLUSION

Independent double readings of mammographic calcifications may not improve diagnostic performance. CAD reading improves diagnostic performance to an extent approaching the maximum possible performance.

Current status and future potential of computer-aided diagnosis in medical imaging

Computer-aided diagnosis (CAD) has become one of the major research subjects in medical imaging and diagnostic radiology. The basic concept of CAD is to provide a computer output as a second opinion to assist radiologists' image interpretation by improving the accuracy and consistency of radiological diagnosis and also by reducing the image reading time. In this article, a number of CAD schemes are presented, with emphasis on potential clinical applications. These schemes include: (1) detection and classification of lung nodules on digital chest radiographs; (2) detection of nodules in low dose CT; (3) distinction between benign and malignant nodules on high resolution CT; (4) usefulness of similar images for distinction between benign and malignant lesions; (5) quantitative analysis of diffuse lung diseases on high resolution CT; and (6) detection of intracranial aneurysms in magnetic resonance angiography. Because CAD can be applied to all imaging modalities, all body parts and all kinds of examinations, it is likely that CAD will have a major impact on medical imaging and diagnostic radiology in the 21st century.

CAD techniques, challenges, and controversies in computed tomographic colonography

Computer-aided diagnosis (CAD) for computed tomographic colonography (CTC) automatically detects the locations of suspicious polyps and masses on CTC and provides radiologists with a second opinion. CAD has the potential to increase radiologists' diagnostic performance in the detection of polyps and masses and to decrease variability of the diagnostic accuracy among readers without significantly increasing the reading time. Technical developments have advanced CAD substantially during the past several years, and a fundamental scheme for the detection of polyps has been established. The most recent CAD systems based on this scheme produce a clinically acceptable high sensitivity and a low false-positive rate. However, CAD for CTC is still under active development, and the technology needs to be improved further. This report describes the expected benefits, the current fundamental scheme, the key techniques used for detection of polyps and masses on CTC, the current detection performance, as well as the pitfalls, challenges, controversies, and the future of CAD.

Overview on research and development of computer-aided diagnostic schemes

Computer-aided diagnosis (CAD) has become a practical clinical approach in diagnostic radiology, although at present only in the area of detection of breast cancer in mammograms. Current research efforts have been focused on detection and classification of images of many different types of lesions in a number of organs, obtained with various imaging modalities. It is likely that the present results of CAD are only at the tip of the iceberg. Although automated computer diagnosis is a concept based on computer algorithms only, CAD is a concept established by taking into account equally the roles of physicians and computers. The effect of CAD on differential diagnosis has already indicated that the performance level is high, and that CAD would be ready for clinical trials and commercialization efforts. The presentation of images similar to those of an unknown case may be useful as a supplemental tool for CAD in the differential diagnosis.

Computer-aided diagnosis in chest radiology

Chest radiography is still a useful examination in various situations, although CT has become a modality of choice as a diagnostic examination in many cases. Current computer-aided diagnosis (CAD) schemes for chest radiographs include nodule detection, interstitial disease detection, temporal subtraction, differential diagnosis of interstitial disease, and distinction between benign and malignant pulmonary nodules. All of these schemes are demonstrated as providing potentially useful tools for radiologists when the output of these schemes is used as a "second opinion." There are some commercially available products for these schemes and more are expected to be available in the near future. The current status of CAD for CT is also discussed briefly in this article.

Quantitative CT of emphysema and the airways

Chronic lung disease is a leading cause of morbidity and mortality in the United States. Quantitative techniques for assessing emphysema and related airway disease have been slow to gain acceptance among radiologists, who have traditionally used description of structural changes to evaluate these diseases. This review provides an overview of these quantitative techniques.

Detection of lung cancer on radiographs: receiver operating characteristic analyses of radiologists', pulmonologists', and anesthesiologists' performance

PURPOSE

To compare and quantify, by means of receiver operating characteristic (ROC) and localization ROC analyses, the performance of radiologists, pulmonologists, and anesthesiologists (residents and staff) in the detection of missed lung cancer.

MATERIALS AND METHODS

The study was approved by the institutional review board, and informed consent was not required or obtained for review of radiographs. A set of 60 posteroanterior chest radiographs was presented to 36 observers: 12 radiologists, 12 pulmonologists, and 12 anesthesiologists. Each of these three observer categories included six residents and six staff. Thirty of the radiographs each depicted one lung cancer that was overlooked at prospective image interpretation; the other 30 were normal radiographs matched for age and smoking history. Observers were asked to rate their degree of suspicion concerning the presence of lung cancer by using a visual analog scale and to point out the zone of suspicion on a schematic of the lung. These data were used to generate combined ROC-localization ROC curves and to assess performance. Intraobserver consistency was evaluated by using intraclass correlation coefficients and weighted kappa statistics.

RESULTS

Areas under the ROC curves indicated better performance for radiologists and pulmonologists compared with anesthesiologists (P < .002) and for staff compared with residents (P < .022). Performance was lower for all categories of observers when localization ROC curves were used. Radiologists and staff pulmonologists showed a higher degree of confidence in the assessment of normality than did other categories of physicians. Intraobserver consistency was poor.

CONCLUSION

Experienced readers showed better ability to distinguish normality from abnormality. Combined ROC and localization ROC analyses gave a more reliable quantification of observer performance than did ROC analysis alone.

Application of an Artificial Neural Network to High-Resolution CT: Usefulness in Differential Diagnosis of Diffuse Lung Disease

OBJECTIVE

The purpose of our study was to evaluate the diagnostic performance of an artificial neural network (ANN) in differentiating among certain diffuse lung diseases using high-resolution CT (HRCT) and the effect of ANN output on radiologists' diagnostic performance.

MATERIALS AND METHODS

We selected 130 clinical cases of diffuse lung disease. We used a single three-layer, feed-forward ANN with a back-propagation algorithm. The ANN was designed to differentiate among 11 diffuse lung diseases by using 10 clinical parameters and 23 HRCT features. Therefore, the ANN consisted of 33 input units and 11 output units. Subjective ratings for 23 HRCT features were provided independently by eight radiologists. All clinical cases were used for training and testing of the ANN by implementing a round-robin technique. In the observer test, a subset of 45 cases was selected from the database of 130 cases. HRCT images were viewed by eight radiologists first without and then with ANN output. The radiologists' performance was evaluated with receiver operating characteristic (ROC) analysis with a continuous rating scale.

RESULTS

The average area under the ROC curve for ANN performance obtained with all clinical parameters and HRCT features was 0.956. The diagnostic performance of four chest radiologists and four general radiologists was increased from 0.986 to 0.992 (p = 0.071) and 0.958 and 0.971 (p < 0.001), respectively, when they used the ANN output based on their own feature ratings.

CONCLUSION

The ANN can provide a useful output as a second opinion to improve general radiologists' diagnostic performance in the differential diagnosis of certain diffuse lung diseases using HRCT.

Evaluating detection and diagnostic decision support systems for bioterrorism response

We evaluated the usefulness of detection systems and diagnostic decision support systems for bioterrorism response. We performed a systematic review by searching relevant databases (e.g., MEDLINE) and Web sites for reports of detection systems and diagnostic decision support systems that could be used during bioterrorism responses. We reviewed over 24,000 citations and identified 55 detection systems and 23 diagnostic decision support systems. Only 35 systems have been evaluated: 4 reported both sensitivity and specificity, 13 were compared to a reference standard, and 31 were evaluated for their timeliness. Most evaluations of detection systems and some evaluations of diagnostic systems for bioterrorism responses are critically deficient. Because false-positive and false-negative rates are unknown for most systems, decision making on the basis of these systems is seriously compromised. We describe a framework for the design of future evaluations of such systems.

Automated storage and retrieval of thin-section CT images to assist diagnosis: system description and preliminary assessment

A software system and database for computer-aided diagnosis with thin-section computed tomographic (CT) images of the chest was designed and implemented. When presented with an unknown query image, the system uses pattern recognition to retrieve visually similar images with known diagnoses from the database. A preliminary validation trial was conducted with 11 volunteers who were asked to select the best diagnosis for a series of test images, with and without software assistance. The percentage of correct answers increased from 29% to 62% with computer assistance. This finding suggests that this system may be useful for computer-assisted diagnosis.

Computer-aided diagnosis to distinguish benign from malignant solitary pulmonary nodules on radiographs: ROC analysis of radiologists' performance--initial experience

PURPOSE

To evaluate radiologists' performance for determining a distinction between benign and malignant pulmonary nodules on chest radiographs without and with use of a computer-aided diagnosis scheme.

MATERIALS AND METHODS

Fifty-three chest radiographs that depicted 31 primary lung cancers and 22 benign nodules were used. The likelihood measure of malignancy for each nodule was determined by using an automated computerized scheme. Sixteen radiologists (nine attending radiologists and seven radiology residents) participated in an observer study in which cases were interpreted first without and then with use of the scheme. The radiologists' performance was evaluated with receiver operating characteristic analysis.

RESULTS

The mean area under the best-fit binormal receiver operating characteristic curve plotted in the unit square (Az) values of radiologists who interpreted images without and with the scheme were 0.743 and 0.817, respectively. The performance of radiologists was improved significantly when the scheme was used (P =.002). However, the performance (Az = 0.889) of the computer alone exceeded these results by a substantial margin. The average change in radiologists' confidence level for interpretation without and with the scheme was highly correlated (r = 0.845) with the likelihood measure of malignancy, which was presented as computer output.

CONCLUSION

This scheme for computer-aided diagnosis has the potential to improve the accuracy of radiologists' performance in the classification of benign and malignant solitary pulmonary nodules.

Computer-aided diagnosis in chest radiography: results of large-scale observer tests at the 1996-2001 RSNA scientific assemblies

Since 1996, computer-aided diagnosis (CAD) schemes have been presented as interactive demonstrations on computer workstations at each scientific assembly of the Radiological Society of North America. The schemes involved (a) detection of pulmonary nodules, (b) temporal subtraction, (c) detection of interstitial lung disease, (d) differential diagnosis of interstitial lung disease, and (e) distinction between benign and malignant pulmonary nodules on chest radiographs. Large-scale observer tests were carried out to examine how radiologists can benefit from CAD systems. Observer performance was evaluated by analysis of receiver operating characteristic (ROC) curves. The statistical significance of the difference between the areas under the ROC curves without and with CAD was analyzed with the Student t test. In all of the tests, the diagnostic accuracy of the radiologists in total improved significantly when CAD was used. This result provides additional evidence that CAD has the potential to improve the performance of radiologists in their decision-making process in interpreting chest radiographs.

Temporal subtraction for the detection of hazy pulmonary opacities on chest radiography

OBJECTIVE

The purpose of this study was to evaluate the accuracy of temporal subtraction with a commercially available computer-assisted diagnosis system for the detection of multifocal hazy pulmonary opacities on chest radiographs, which are sometimes difficult to detect directly on chest radiographs.

MATERIALS AND METHODS

Thirty healthy patients and 30 patients with new multifocal hazy pulmonary opacities that were confirmed by serial chest CT examinations were evaluated with and without temporal subtraction images. Chest radiographs were taken from either film-screen or digital radiography images and were digitized with a spatial resolution of 0.171 mm per pixel. Temporal subtraction images were produced by an iterative image-warping technique. We designed an observer performance study in which observers (six chest radiologists and four residents) indicated their confidence level for the presence or absence of hazy pulmonary opacities on two sets of images, current and previous radiographs only (set A), and current and previous radiographs with temporal subtraction images (set B). Receiver operating characteristic curves were generated.

RESULTS

For chest radiologists, observer performance with set B (with temporal subtraction images; A(z) = 0.947) was superior to that with set A (without temporal subtraction images; A(z) = 0.916) (p < 0.05). For residents, no statistically significant difference was found between sets A and B.

CONCLUSION

The temporal subtraction technique clearly improves diagnostic accuracy for the detection of multifocal hazy pulmonary opacities on chest radiographs, especially when the observers are experienced chest radiologists who have sufficient skill to evaluate the patient's condition as revealed on the images.

International Labor Office Classification System in the age of imaging: relevant or redundant

The 1980 International Labor Office International Classification of Radiographs of Pneumoconioses is a widely used epidemiologic tool with a storied past. This article reviews its development and examines its applications to occupational lung disease and the controversies generated in that process. The question of its relevancy to current imaging practices is discussed.

Automatic detection of abnormalities in chest radiographs using local texture analysis

A fully automatic method is presented to detect abnormalities in frontal chest radiographs which are aggregated into an overall abnormality score. The method is aimed at finding abnormal signs of a diffuse textural nature, such as they are encountered in mass chest screening against tuberculosis (TB). The scheme starts with automatic segmentation of the lung fields, using active shape models. The segmentation is used to subdivide the lung fields into overlapping regions of various sizes. Texture features are extracted from each region, using the moments of responses to a multiscale filter bank. Additional "difference features" are obtained by subtracting feature vectors from corresponding regions in the left and right lung fields. A separate training set is constructed for each region. All regions are classified by voting among the k nearest neighbors, with leave-one-out. Next, the classification results of each region are combined, using a weighted multiplier in which regions with higher classification reliability weigh more heavily. This produces an abnormality score for each image. The method is evaluated on two databases. The first database was collected from a TB mass chest screening program, from which 147 images with textural abnormalities and 241 normal images were selected. Although this database contains many subtle abnormalities, the classification has a sensitivity of 0.86 at a specificity of 0.50 and an area under the receiver operating characteristic (ROC) curve of 0.820. The second database consist of 100 normal images and 100 abnormal images with interstitial disease. For this database, the results were a sensitivity of 0.97 at a specificity of 0.90 and an area under the ROC curve of 0.986.

Quantitative CT of the lung

Lung disease is a leading cause of morbidity and mortality. HRCT, currently the best test to assess lung involvement in emphysema and interstitial lung disease, relies on abnormalities being detected when there is sufficient morphologic distortion to result in visually identified changes that, for the most part, correlate poorly with conventional lung function tests and outcome. QIA offers a technique to assess both structure and function on a regional and global basis. With the advent of user-friendly software packages, this approach is finding application in clinical practice and in clinical studies of new treatment alternatives for diffuse lung disease

Computer-aided diagnosis in chest radiography: a survey

The traditional chest radiograph is still ubiquitous in clinical practice, and will likely remain so for quite some time. Yet, its interpretation is notoriously difficult. This explains the continued interest in computer-aided diagnosis for chest radiography. The purpose of this survey is to categorize and briefly review the literature on computer analysis of chest images, which comprises over 150 papers published in the last 30 years. Remaining challenges are indicated and some directions for future research are given.

Potential of computer-aided diagnosis to reduce variability in radiologists' interpretations of mammograms depicting microcalcifications

PURPOSE

To evaluate whether computer-aided diagnosis can reduce interobserver variability in the interpretation of mammograms.

MATERIALS AND METHODS

Ten radiologists interpreted mammograms showing clustered microcalcifications in 104 patients. Decisions for biopsy or follow-up were made with and without a computer aid, and these decisions were compared. The computer was used to estimate the likelihood that a microcalcification cluster was due to a malignancy. Variability in the radiologists' recommendations for biopsy versus follow-up was then analyzed.

RESULTS

Variation in the radiologists' accuracy, as measured with the SD of the area under the receiver operating characteristic curve, was reduced by 46% with computer aid. Access to the computer aid increased the agreement among all observers from 13% to 32% of the total cases (P <.001), while the kappa value increased from 0.19 to 0.41 (P <.05). Use of computer aid eliminated two-thirds of the substantial disagreements in which two radiologists recommended biopsy and routine screening in the same patient (P <.05).

CONCLUSION

In addition to its demonstrated potential to improve diagnostic accuracy, computer-aided diagnosis has the potential to reduce the variability among radiologists in the interpretation of mammograms.

Automatic detection and quantification of ground-glass opacities on high-resolution CT using multiple neural networks: comparison with a density mask

OBJECTIVE

We compared multiple neural networks with a density mask for the automatic detection and quantification of ground-glass opacities on high-resolution CT under clinical conditions.

SUBJECTS AND METHODS

Eighty-four patients (54 men and 30 women; age range, 18-82 years; mean age, 49 years) with a total of 99 consecutive high-resolution CT scans were enrolled in the study. The neural network was designed to detect ground-glass opacities with high sensitivity and to omit air-tissue interfaces to increase specificity. The results of the neural network were compared with those of a density mask (thresholds, -750/-300 H), with a radiologist serving as the gold standard.

RESULTS

The neural network classified 6% of the total lung area as ground-glass opacities. The density mask failed to detect 1.3%, and this percentage represented the increase in sensitivity that was achieved by the neural network. The density mask identified another 17.3% of the total lung area to be ground-glass opacities that were not detected by the neural network. This area represented the increase in specificity achieved by the neural network. Related to the extent of the ground-glass opacities as classified by the radiologist, the neural network (density mask) reached a sensitivity of 99% (89%), specificity of 83% (55%), positive predictive value of 78% (18%), negative predictive value of 99% (98%), and accuracy of 89% (58%).

CONCLUSION

Automatic segmentation and quantification of ground-glass opacities on high-resolution CT by a neural network are sufficiently accurate to be implemented for the preinterpretation of images in a clinical environment; it is superior to a double-threshold density mask.