Detection of lung cancer on radiographs: receiver operating characteristic analyses of radiologists', pulmonologists', and anesthesiologists' performance

Evaluation of the clinical utility of temporal subtraction using bone suppression processing in digital chest radiography

Rationale and objectives

To evaluate the usefulness of temporal subtraction using the bone suppression method in digital chest radiography for the detection of pulmonary lesions.

Materials and methods

The images of 31 patients with pulmonary lesions and 19 normal cases were included in the study. Conventional and bone suppression temporal subtraction were performed in the 50 cases selected and used for an observer performance study. Five radiologists participated in the study, and the differences between using conventional and bone suppression temporal subtraction were assessed using jackknife free-response receiver operating characteristic analysis.

Results

The average figure-of-merit values for all radiologists increased significantly using the bone suppression method, from 0.619 (conventional) to 0.696 (p = 0.032). The average sensitivity for detecting pulmonary lesions improved from 67.9% to 75.4%, and the average number of false-positive per case decreased from 0.336 to 0.252 using bone suppression temporal subtraction.

Conclusion

Bone suppression temporal subtraction processing can assist with the detection of subtle pulmonary lesions in digital chest radiographs.

Realistic High-Resolution Body Computed Tomography Image Synthesis by Using Progressive Growing Generative Adversarial Network: Visual Turing Test

BACKGROUND

Generative adversarial network (GAN)-based synthetic images can be viable solutions to current supervised deep learning challenges. However, generating highly realistic images is a prerequisite for these approaches.

OBJECTIVE

The aim of this study was to investigate and validate the unsupervised synthesis of highly realistic body computed tomography (CT) images by using a progressive growing GAN (PGGAN) trained to learn the probability distribution of normal data.

METHODS

We trained the PGGAN by using 11,755 body CT scans. Ten radiologists (4 radiologists with <5 years of experience [Group I], 4 radiologists with 5-10 years of experience [Group II], and 2 radiologists with >10 years of experience [Group III]) evaluated the results in a binary approach by using an independent validation set of 300 images (150 real and 150 synthetic) to judge the authenticity of each image.

RESULTS

The mean accuracy of the 10 readers in the entire image set was higher than random guessing (1781/3000, 59.4% vs 1500/3000, 50.0%, respectively; P<.001). However, in terms of identifying synthetic images as fake, there was no significant difference in the specificity between the visual Turing test and random guessing (779/1500, 51.9% vs 750/1500, 50.0%, respectively; P=.29). The accuracy between the 3 reader groups with different experience levels was not significantly different (Group I, 696/1200, 58.0%; Group II, 726/1200, 60.5%; and Group III, 359/600, 59.8%; P=.36). Interreader agreements were poor (κ=0.11) for the entire image set. In subgroup analysis, the discrepancies between real and synthetic CT images occurred mainly in the thoracoabdominal junction and in the anatomical details.

CONCLUSIONS

The GAN can synthesize highly realistic high-resolution body CT images that are indistinguishable from real images; however, it has limitations in generating body images of the thoracoabdominal junction and lacks accuracy in the anatomical details.

Value of bone suppression software in chest radiographs for improving image quality and reducing radiation dose

OBJECTIVES

To compare image quality and radiation dose between dual-energy subtraction (DES)-based bone suppression images (D-BSIs) and software-based bone suppression images (S-BSIs).

METHODS

Chest radiographs (CXRs) of forty adult patients were obtained with the two X-ray devices, one with DES and one with bone suppression software. Three image quality metrics (relative mean absolute error (RMAE), peak signal-to-noise ratio (PSNR), and structural similarity index (SSIM)) between original CXR and BSI for each of D-BSI and S-SBI groups were calculated for each bone and soft tissue areas. Two readers rated the visual image quality for original CXR and BSI for each of D-BSI and S-SBI groups. The dose area product (DAP) values were recorded. Paired t test was used to compare the image quality and DAP values between D-BSI and S-BSI groups.

RESULTS

In bone areas, S-BSIs had better SSIM values than D-BSI (94.57 vs. 87.77) but worse RMAE and PSNR values (0.50 vs. 0.20; 20.93 vs. 34.37) (all p < 0.001). In soft tissue areas, S-BSIs had better SSIM values than D-BSI (97.56 vs. 91.42) but similar RMAE and PSNR values (0.29 vs. 0.27; 31.35 vs. 29.87) (all p < 0.001). Both readers gave S-BSIs significantly higher image quality scores than D-BSI (p < 0.001). The mean DAP in software-related images (0.98 dGy·cm²) was significantly lower than that in the DES-related images (1.48 dGy·cm²) (p < 0.001).

CONCLUSION

Bone suppression software significantly improved the image quality of bone suppression images with a relatively lower radiation dose, compared with dual-energy subtraction technique.

KEY POINTS

• Bone suppression software preserves structure similarity of soft tissues better than dual-energy subtraction technique in bone suppression images. • Bone suppression software achieves superior image quality for lung lesions than dual-energy subtraction technique in bone suppression images. • Bone suppression software can decrease the radiation dose over the hardware-based image processing technique.

Measuring the Effects of Education in Detecting Lung Cancer on Chest Radiographs: Utilization of a New Assessment Tool

This study was designed to evaluate the effect of group and individualized educational lectures to accurately interpret chest radiographs of lung cancer patients and to introduce a new educational tool in evaluating skills for reading chest radiographs. Utilizing "hotspot" technology will be instrumental in measuring the effect of education in interpreting chest radiographs. There were 48 participants in the study. Chest radiographs of 100 lung cancer patients and 11 healthy patients taken at various time points were used for evaluation. Using "hotspot" technology, lesions on each radiograph were outlined. Values were taken at baseline, after which the group received lectures. Several days later, they underwent exam 2. Exam 3 was conducted after individualized lectures. A final exam was taken after the participants underwent individualized training within 2 months. Scores significantly improved after the individual lessons (p < 0.001). This improvement in performance decreased in the final examination. Statistically significant differences were observed between exam 2 vs. exam 3 and exam 3 vs. the final exam (p < 0.001, p < 0.001). Participants demonstrated more improvement in detecting lesions in abnormal chest radiographs than in identifying normal ones. Although there was significant improvement in detecting abnormal radiographs by the end of the study (p < 0.001), no improvement was observed in detecting normal ones. We measured lung cancer detection rate using a new "hotspot" detection tool for chest radiographs. With the proposed scoring system, this tool could be objectively used in evaluating the educational effects.

Added Value of Bone Suppression Image in the Detection of Subtle Lung Lesions on Chest Radiographs with Regard to Reader's Expertise

BACKGROUND

Chest radiographs (CXR) are the most commonly used imaging techniques by various clinicians and radiologists. However, detecting lung lesions on CXR depends largely on the reader's experience level, so there have been several trials to overcome this problem using post-processing of CXR. We investigated the added value of bone suppression image (BSI) in detecting various subtle lung lesions on CXR with regard to reader's expertise.

METHODS

We applied a software program to generate BSI in 1,600 patients in the emergency department. Of them, 80 patients with subtle lung lesions and 80 patients with negative finding on CXR were retrospectively selected based on the subtlety scores on CXR and CT findings. Ten readers independently rated their confidence in deciding the presence or absence of a lung lesion at each of 960 lung regions on the two separated imaging sessions: CXR alone vs. CXR with BSI.

RESULTS

The additional use of BSI for all readers significantly increased the mean area under the curve (AUC) in detecting subtle lung lesions (0.663 vs. 0.706; P < 0.001). The less experienced readers were, the more AUC differences increased: 0.067 (P < 0.001) for junior radiology residents; 0.064 (P < 0.001) for non-radiology clinicians; 0.044 (P < 0.001) for senior radiology residents; and 0.019 (P = 0.041) for chest radiologists. The additional use of BSI significantly increased the mean confidence regarding the presence or absence of lung lesions for 213 positive lung regions (2.083 vs. 2.357; P < 0.001) and for 747 negative regions (1.217 vs. 1.195; P = 0.008).

CONCLUSION

The use of BSI increases diagnostic performance and confidence, regardless of reader's expertise, reduces the impact of reader's expertise and can be helpful for less experienced clinicians and residents in the detection of subtle lung lesions.

Influence of radiologic expertise in detecting lung tumors on chest radiographs

PURPOSE

To analyze the influence of radiologic expertise in detecting lung tumors on chest radiographs.

MATERIALS AND METHODS

We retrieved posteroanterior chest radiographs and CT examination obtained from 283 patients with solitary primary malignant lung tumors who underwent surgical resection. There were 176 men and 107 women with a mean age of 67.0±9.1 (SD) years (range: 33-88 years). Thirteen first-year post-graduate (PGY-1) trainees and nine pulmonary specialists (three radiologists, three thoracic surgeons, and three pulmonologists) interpreted the chest radiographs. Detection rates among trainees and specialists were compared using Student t test.

RESULTS

The total numbers of detected tumors ranged from 103 (36.4%) to 136 (48.1%) with a mean of 127.9±9.1 (45.2±3.2%) in the trainee group, and 137 (48.4%) to 182 (64.3%) with a mean of 161.6±13.1 (57.1±4.6%) in the specialist group; the intergroup difference was statistically significant (P<0.001). Significant intergroup detectability differences of >10% were noted for tumors in the peripheral zone with (i) ground glass opacity (GGO) ratio ≥10% and <70% and any size, or (ii) GGO ratio <10% and size ≤2cm; and for tumors hidden by the mediastinum, heart, or diaphragm with (i) GGO ratio ≥10% and <30% and size >3cm, or (ii) GGO ratio <10% and size >2cm.

CONCLUSION

Our study demonstrates significant differences in lung tumor detectability on chest radiographs between PGY-1 trainees and pulmonary specialists according to tumor size, extent of GGO, and tumor location.

Does periodic lung screening of films meets standards?

Objective:

To determine whether the workers’ periodic chest x-ray screening techniques in accordance with the quality standards is the responsibility of physicians. Evaluation of differences of interpretations by physicians in different levels of education and the importance of standardization of interpretation.

Methods:

Previously taken chest radiographs of 400 workers who are working in a factory producing the glass run channels were evaluated according to technical and quality standards by three observers (pulmonologist, radiologist, pulmonologist assistant). There was a perfect concordance between radiologist and pulmonologist for the underpenetrated films. Whereas there was perfect concordance between pulmonologist and pulmonologist assistant for over penetrated films.

Results:

Pulmonologist (52%) has interpreted the dose of the films as regular more than other observers (radiologist; 44.3%, pulmonologist assistant; 30.4%). The frequency of interpretation of the films as taken in inspiratory phase by the pulmonologist (81.7%) was less than other observers (radiologist; 92.1%, pulmonologist assistant; 92.6%). The rate of the pulmonologist (53.5%) was higher than the other observers (radiologist; 44.6%, pulmonologist assistant; 41.8%) for the assessment of the positioning of the patients as symmetrical. Pulmonologist assistant (15.3%) was the one who most commonly reported the parenchymal findings (radiologist; 2.2%, pulmonologist; 12.9%).

Conclusion:

It is necessary to reorganize the technical standards and exposure procedures for improving the quality of the chest radiographs. The reappraisal of all interpreters and continuous training of technicians is required.

Computer-aided nodule detection system: results in an unselected series of consecutive chest radiographs

RATIONALE AND OBJECTIVES

To evaluate the performance of a computer-aided detection (CAD) system with bone suppression imaging when applied to unselected consecutive chest radiographs (CXRs) with computed tomography (CT) correlation.

MATERIALS AND METHODS

This study included 586 consecutive patients with standard or portable CXRs who had a chest CT scan on the same day. Among the 586 CXRs, 438 had various abnormalities, including 46 CXRs with 66 lung nodules, and 148 CXRs had no significant abnormalities. A commercially available CAD system was applied to all 586 CXRs. True nodules and false positives (FPs) marked on CXRs by the CAD system were evaluated based on the corresponding chest CT findings.

RESULTS

The CAD system marked 47 of 66 (71%) lung nodules in this consecutive series of CXRs. The mean FP rate per image was 1.3 across all 586 CXRs, with 1.5 FPs per image on the 438 abnormal CXRs and 0.8 FPs per image on the 148 normal CXRs. A total of 41% of the 752 FP marks were related to non-nodule pathologic findings.

CONCLUSIONS

A currently available CAD system marked 71% of radiologist-identified lung nodules in a large consecutive series of CXRs, and 41% of "false" marks were caused by pathologic findings.

Digital tomosynthesis for evaluating metastatic lung nodules: nodule visibility, learning curves, and reading times

OBJECTIVE

To evaluate nodule visibility, learning curves, and reading times for digital tomosynthesis (DT).

MATERIALS AND METHODS

We included 80 patients who underwent computed tomography (CT) and DT before pulmonary metastasectomy. One experienced chest radiologist annotated all visible nodules on thin-section CT scans using computer-aided detection software. Two radiologists used CT as the reference standard and retrospectively graded the visibility of nodules on DT. Nodule detection performance was evaluated in four sessions of 20 cases each by six readers. After each session, readers were unblinded to the DT images by revealing the true-positive markings and were instructed to self-analyze their own misreads. Receiver-operating-characteristic curves were determined.

RESULTS

Among 414 nodules on CT, 53.3% (221/414) were visible on DT. The main reason for not seeing a nodule on DT was small size (93.3%, ≤ 5 mm). DT revealed a substantial number of malignant nodules (84.1%, 143/170). The proportion of malignant nodules among visible nodules on DT was significantly higher (64.7%, 143/221) than that on CT (41.1%, 170/414) (p < 0.001). Area under the curve (AUC) values at the initial session were > 0.8, and the average detection rate for malignant nodules was 85% (210/246). The inter-session analysis of the AUC showed no significant differences among the readers, and the detection rate for malignant nodules did not differ across sessions. A slight improvement in reading times was observed.

CONCLUSION

Most malignant nodules > 5 mm were visible on DT. As nodule detection performance was high from the initial session, DT may be readily applicable for radiology residents and board-certified radiologists.

A comparison of computer-aided detection (CAD) effectiveness in pulmonary nodule identification using different methods of bone suppression in chest radiographs

This study aimed to compare the diagnostic effectiveness of computer-aided detection (CAD) software (OnGuard™ 5.2) in combination with hardware-based bone suppression (dual-energy subtraction radiography (DESR)), software-based bone suppression (SoftView™, version 2.4), and standard posteroanterior images with no bone suppression. A retrospective pilot study compared the diagnostic performance of two commercially available methods of bone suppression when used with commercially available CAD software. Chest images from 27 patients with computed tomography (CT) and pathology-proven malignant pulmonary nodules (8-34 mm) and 25 CT-negative patient controls were used for analysis. The Friedman, McNemar, and chi-square tests were used to compare diagnostic performance and the kappa statistic was used to evaluate method agreement. The average number of regions of interest and false-positives per image identified by CAD were not found to be significantly different regardless of the bone suppression methods evaluated. Similarly, the sensitivity, specificity, and test efficiency were not found to be significantly different. Agreement between the methods was between poor and excellent. The accuracy of CAD (OnGuard™, version 5.2) is not statistically different with either DESR or SoftView™ (version 2.4) bone suppression technology in digital chest images for pulmonary nodule identification. Low values for sensitivity (<80 %) and specificity (<50 %) may limit their utility for clinical radiology.

Improved detection of focal pneumonia by chest radiography with bone suppression imaging

OBJECTIVE

To evaluate radiologists' ability to detect focal pneumonia by use of standard chest radiographs alone compared with standard plus bone-suppressed chest radiographs.

METHODS

Standard chest radiographs in 36 patients with 46 focal airspace opacities due to pneumonia (10 patients had bilateral opacities) and 20 patients without focal opacities were included in an observer study. A bone suppression image processing system was applied to the 56 radiographs to create corresponding bone suppression images. In the observer study, eight observers, including six attending radiologists and two radiology residents, indicated their confidence level regarding the presence of a focal opacity compatible with pneumonia for each lung, first by use of standard images, then with the addition of bone suppression images. Receiver operating characteristic (ROC) analysis was used to evaluate the observers' performance.

RESULTS

The mean value of the area under the ROC curve (AUC) for eight observers was significantly improved from 0.844 with use of standard images alone to 0.880 with standard plus bone suppression images (P < 0.001) based on 46 positive lungs and 66 negative lungs.

CONCLUSION

Use of bone suppression images improved radiologists' performance for detection of focal pneumonia on chest radiographs.

KEY POINTS

Bone suppression image processing can be applied to conventional digital radiography systems. Bone suppression imaging (BSI) produces images that appear similar to dual-energy soft tissue images. BSI improves the conspicuity of focal lung disease by minimizing bone opacity. BSI can improve the accuracy of radiologists in detecting focal pneumonia.

A study to determine the added value of 740 screening panoramic radiographs compared to intraoral radiography in the management of adult (>18 years) dentate patients in a primary care setting

OBJECTIVES

To measure the added value of panoramic radiography in new dentate patients attending for routine treatment.

METHODS

Thirty-seven general dental practitioners using panoramic radiographs routinely were recruited. Twenty dentate patients were identified prospectively by each participating dentist if they were new to the practice, attending for an examination and requesting any treatment deemed necessary. A panoramic radiograph was taken with appropriate intraoral radiographs in line with national guidelines. Each dentist completed a radiological report for the panoramic radiograph only and these 20 reports were forwarded to the researchers along with the 20 panoramic radiographs, their accompanying bitewing and periapical radiographs and twenty completed clinical assessment sheets.

RESULTS

740 panoramic, 1418 bitewing and 325 periapical radiographs were assessed by the researchers. Only 32 panoramic films provided any additional diagnostic value when compared to intraoral films when guidelines had been observed resulting from the poor technical and processing quality of the accompanying intraoral films. Assessment of the number of caries and periapical lesions and the degree of periodontal bone loss from the intraoral films provided a greater diagnostic yield at the p<0.001 level of significance. The research found that dentists underestimated the number of caries lesions present and level of periodontal bone loss when compared to the researchers but overestimated the presence of periapical pathology, at the level of significance at p<0.001.

CONCLUSIONS

The study found that there was no support for the use of panoramic radiographs in routine screening as there was no net diagnostic benefit to the patient.

Flat-panel detector computed tomography imaging: observer performance in detecting pulmonary nodules in comparison with conventional chest radiography and multidetector computed tomography

PURPOSE

The aim of this study was to compare the detectability of lung nodules on images obtained with a flat-panel detector computed tomography (FPD-CT) system and by chest radiographs (CXRs) using receiver-operating characteristic (ROC) analysis.

MATERIALS AND METHODS

FPD-CT was conducted with the patients in the sitting position. For the CXR study, the patients stood erect. Our study population consisted of 26 individuals ranging in age from 50 to 83 years. The reference standard was based on the interpretations obtained by consensus of 2 radiologists on multidetector CT images for the presence or absence of nodules. Four other radiologists independently assessed and recorded the absence or presence of lung nodules and their location on FPD-CT and CXR images. ROC analysis was used to evaluate lung nodule detectability by both imaging modalities.

RESULTS

Two radiologists identified 34 nodules whose diameter was 5 to 42 mm (mean, 19.3 mm) in 23 of the 26 study participants on the multidetector CT images. Overall, analysis of variance for ROC analysis showed that FPD-CT was significantly better in detecting nodules than CXR (P=0.02). The estimated mean Az value was 0.9818±0.0083 with FPD-CT and 0.7610±0.0908 with CXR. The sensitivity for nodule detection on FPD-CT and CXR images was 79.4% and 33.8%, respectively.

CONCLUSION

The detectability of pulmonary nodules was better on images of FPD-CT than on CXRs.

Small lung cancers: improved detection by use of bone suppression imaging--comparison with dual-energy subtraction chest radiography

PURPOSE

To determine whether use of bone suppression (BS) imaging, used together with a standard radiograph, could improve radiologists' performance for detection of small lung cancers compared with use of standard chest radiographs alone and whether BS imaging would provide accuracy equivalent to that of dual-energy subtraction (DES) radiography.

MATERIALS AND METHODS

Institutional review board approval was obtained. The requirement for informed consent was waived. The study was HIPAA compliant. Standard and DES chest radiographs of 50 patients with 55 confirmed primary nodular cancers (mean diameter, 20 mm) as well as 30 patients without cancers were included in the observer study. A new BS imaging processing system that can suppress the conspicuity of bones was applied to the standard radiographs to create corresponding BS images. Ten observers, including six experienced radiologists and four radiology residents, indicated their confidence levels regarding the presence or absence of a lung cancer for each lung, first by using a standard image, then a BS image, and finally DES soft-tissue and bone images. Receiver operating characteristic (ROC) analysis was used to evaluate observer performance.

RESULTS

The average area under the ROC curve (AUC) for all observers was significantly improved from 0.807 to 0.867 with BS imaging and to 0.916 with DES (both P < .001). The average AUC for the six experienced radiologists was significantly improved from 0.846 with standard images to 0.894 with BS images (P < .001) and from 0.894 to 0.945 with DES images (P = .001).

CONCLUSION

Use of BS imaging together with a standard radiograph can improve radiologists' accuracy for detection of small lung cancers on chest radiographs. Further improvements can be achieved by use of DES radiography but with the requirement for special equipment and a potential small increase in radiation dose.

Usefulness of computerized method for lung nodule detection on digital chest radiographs using similar subtraction images from different patients

PURPOSE

The purpose of this study is to evaluate the usefulness of a novel computerized method to select automatically the similar chest radiograph for image subtraction in the patients who have no previous chest radiographs and to assist the radiologists' interpretation by presenting the "similar subtraction image" from different patients.

MATERIALS AND METHODS

Institutional review board approval was obtained, and the requirement for informed patient consent was waived. A large database of approximately 15,000 normal chest radiographs was used for searching similar images of different patients. One hundred images of candidates were selected according to two clinical parameters and similarity of the lung field in the target image. We used the correlation value of chest region in the 100 images for searching the most similar image. The similar subtraction images were obtained by subtracting the similar image selected from the target image. Thirty cases with lung nodules and 30 cases without lung nodules were used for an observer performance test. Four attending radiologists and four radiology residents participated in this observer performance test.

RESULTS

The AUC for all radiologists increased significantly from 0.925 to 0.974 with the CAD (P=.004). When the computer output images were available, the average AUC for the residents was more improved (0.960 vs. 0.890) than for the attending radiologists (0.987 vs. 0.960).

CONCLUSION

The novel computerized method for lung nodule detection using similar subtraction images from different patients would be useful to detect lung nodules on digital chest radiographs, especially for less experienced readers.

Reader performance in radiographic diagnosis of signs of mitral regurgitation in cavalier King Charles spaniels

OBJECTIVES

To measure accuracy and variability of diagnosis by radiography of heart enlargement (HE) and heart failure (HF) in mitral regurgitation (MR).

METHODS

Sixteen readers representing four levels of experience evaluated 50 sets of radiographs with varying severity of MR for presence or absence of HE, left atrial enlargement (LAE) and HF. The performance of the readers was compared with a reference standard, using area under the curve (AUC) of receiver operating characteristic (ROC) curves. The interreader agreement value kappa (K) was calculated. A subset of difficult cases of HF was analysed before and after removing an outlying reader from each group.

RESULTS

AUC for HE was 0.89, for LAE it was 0.93 and for HF it was 0.92. Experience increased certainty of diagnosis but not accuracy. K ranges were HE, 0.53 to 0.67; LAE, 0.61 to 0.69 and HF, 0.49 to 0.58. When only difficult cases of HF were read, accuracy decreased and experienced readers performed better than inexperienced. When outlying readers were excluded, the differences between experienced and inexperienced readers increased.

CLINICAL SIGNIFICANCE

LAE, not HE, should be used to evaluate the heart size and indirectly the severity of MR on radiographs. For HF, agreement among individual readers was only moderate. Studies of reader accuracy should consider the effects of interreader variability.

Binary and multi-category ratings in a laboratory observer performance study: a comparison

The authors investigated radiologists, performances during retrospective interpretation of screening mammograms when using a binary decision whether to recall a woman for additional procedures or not and compared it with their receiver operating characteristic (ROC) type performance curves using a semi-continuous rating scale. Under an Institutional Review Board approved protocol nine experienced radiologists independently rated an enriched set of 155 examinations that they had not personally read in the clinic, mixed with other enriched sets of examinations that they had individually read in the clinic, using both a screening BI-RADS rating scale (recall/not recall) and a semi-continuous ROC type rating scale (0 to 100). The vertical distance, namely the difference in sensitivity levels at the same specificity levels, between the empirical ROC curve and the binary operating point were computed for each reader. The vertical distance averaged over all readers was used to assess the proximity of the performance levels under the binary and ROC-type rating scale. There does not appear to be any systematic tendency of the readers towards a better performance when using either of the two rating approaches, namely four readers performed better using the semi-continuous rating scale, four readers performed better with the binary scale, and one reader had the point exactly on the empirical ROC curve. Only one of the nine readers had a binary "operating point" that was statistically distant from the same reader's empirical ROC curve. Reader-specific differences ranged from -0.046 to 0.128 with an average width of the corresponding 95% confidence intervals of 0.2 and p-values ranging for individual readers from 0.050 to 0.966. On average, radiologists performed similarly when using the two rating scales in that the average distance between the run in individual reader's binary operating point and their ROC curve was close to zero. The 95% confidence interval for the fixed-reader average (0.016) was (-0.0206, 0.0631) (two-sided p-value 0.35). In conclusion the authors found that in retrospective observer performance studies the use of a binary response or a semi-continuous rating scale led to consistent results in terms of performance as measured by sensitivity-specificity operating points.

Improved detection of small lung cancers with dual-energy subtraction chest radiography

OBJECTIVE

The objective of our study was to retrospectively evaluate whether the use of dual-energy subtraction chest radiographs can improve radiologists' performance for the detection of small previously missed lung cancers.

MATERIALS AND METHODS

Dual-energy subtraction chest radiographs of 19 patients with previously missed nodular cancers, in which the radiology report did not mention a nodule that was visible in retrospect, were selected. Dual-energy subtraction radiographs of 19 patients with cancer and 16 patients without cancer were used for an observer study. Six radiologists indicated their confidence level regarding the presence of a lung cancer and, if they thought a cancer was present, also marked the most likely position for each lung, first using standard posteroanterior and lateral chest radiographs and then using both soft-tissue and bone dual-energy subtraction images along with standard radiographs. Receiver operating characteristic (ROC) curves were used to evaluate the observers' performance. The indicated locations of cancers and false-positives were also analyzed.

RESULTS

The average area under the ROC curve (A(z)) value for the six radiologists was improved from 0.718 to 0.816, a statistically significant amount (p = 0.004), and the average sensitivity (correct localizations) for 19 previously missed cancers was also significantly improved from 40% to 59% (p = 0.008) with the aid of dual-energy subtraction images. The average number of false-positive (incorrect) localizations on 70 lungs was 10 without and nine with dual-energy subtraction images (p = 0.785).

CONCLUSION

Dual-energy subtraction chest radiography has the potential to improve radiologists' performance for the detection of small missed lung cancers.

Effect of multiscale processing in digital chest radiography on automated detection of lung nodule with a computer assistance system

The aim of this study is to evaluate the effect of multiscale processing in digital chest radiography on automated detection of lung nodule with a computer-aided diagnosis (CAD) system. The study involved 58 small-nodule patient cases and 58 normal cases. The 58 patient cases included a total of 64 noncalcified lung nodules up to 15 mm in diameter. Each case underwent an examination with a digital radiography system (Digital Diagnost, Philips Medical Systems), and the acquired image was processed by the following three types of multiscale processing (Unique Image Processing Package, Philips Medical Systems) respectively: (1) standard image from the default processing parameter (structure preference, 0.0), (2) high-pass image with structure preference of 0.4, (3) low-pass image with structure preference of -0.4. The CAD output images were produced with a real-time computer assistance system (IQQAtrade mark-Chest, EDDA Technology). Two experienced chest radiologists established the nodule gold standard by consensus reading according to computed tomography results, and analyzed and recorded the detection of lung nodules and false-positive detections of these CAD output images. For the entire cases involved (each case with three types of different processing), a total of 348 observations were evaluated by the receiver operating characteristic (ROC) analysis. The mean area under the ROC curve (A ( z )) value was 0.700 for the standard images, 0.587 for the high-pass images, and 0.783 for the low-pass images. There were statistically significant A (z) values among these three types of processed images (p < 0.01). Multiscale processing in digital chest radiography can affect the automated detection of lung nodule by CAD, which is consistent with effects from visual inspection.

Lung cancers missed on chest radiographs: results obtained with a commercial computer-aided detection program

PURPOSE

To retrospectively determine the sensitivity of and number of false-positive marks made by a commercially available computer-aided detection (CAD) system for identifying lung cancers previously missed on chest radiographs by radiologists, with histopathologic results as the reference standard.

MATERIALS AND METHODS

Institutional review board approval was obtained for this HIPAA-compliant study; the requirement for informed patient consent was waived. A CAD nodule detection program was applied to 34 posteroanterior digital chest radiographs obtained in 34 patients (21 men, 13 women; mean age, 69 years). All 34 radiographs showed a nodular lung cancer that was apparent in retrospect but had not been mentioned in the report. Two radiologists identified these radiologist-missed cancers on the chest radiographs and graded them for visibility, location, subtlety (extremely subtle to extremely obvious on a 10-point scale), and actionability (actionable or not actionable according to whether the radiologists probably would have recommended follow-up if the nodule had been detected). The CAD results were analyzed to determine the numbers of cancers and false-positive nodules marked and to correlate the CAD results with the nodule grades for subtlety and actionability. The chi2 test or Fisher exact test for independence was used to compare CAD sensitivity between the very subtle (grade 1-3) and relatively obvious (grade > 3) cancers and between the actionable and not actionable cancers.

RESULTS

The CAD program had an overall sensitivity of 35% (12 of 34 cancers), identifying seven (30%) of 23 very subtle and five (45%) of 11 relatively obvious radiologist-missed cancers (P = .21) and detecting two (25%) of eight missed not actionable and ten (38%) of 26 missed actionable cancers (P = .33). The CAD program made an average of 5.9 false-positive marks per radiograph.

CONCLUSION

The described CAD system can mark a substantial proportion of visually subtle lung cancers that are likely to be missed by radiologists.

A review of research into the development of radiologic expertise: implications for computer-based training

RATIONALE AND OBJECTIVES

Studies of radiologic error reveal high levels of variation between radiologists. Although it is known that experts outperform novices, we have only limited knowledge about radiologic expertise and how it is acquired.

MATERIALS AND METHODS

This review identifies three areas of research: studies of the impact of experience and related factors on the accuracy of decision-making; studies of the organization of expert knowledge; and studies of radiologists' perceptual processes.

RESULTS AND CONCLUSION

Interpreting evidence from these three paradigms in the light of recent research into perceptual learning and studies of the visual pathway has a number of conclusions for the training of radiologists, particularly for the design of computer-based learning programs that are able to illustrate the similarities and differences between diagnoses, to give access to large numbers of cases and to help identify weaknesses in the way trainees build up a global representation from fixated regions.

Measuring Observer Performance in Chest Radiology: Some Experiences

All decisions made under conditions of uncertainty have error rates. All meaningful decisions are made under conditions of uncertainty. Can this uncertainty be measured? Can variations in how different observers deal with this uncertainty be ascertained? The ability to measure observer performance in diagnostic imaging was one of the issues that initiated the field of medical decision analysis. This article exemplifies an approach and is worth discussing as a preamble to presenting our long-term project of measuring variations in observer performance. The paper focuses on the interpretation of chest x-ray images, although the principles and findings described can be applied to nearly every radiologic modality and interpretation task.

Competency in chest radiography. A comparison of medical students, residents, and fellows

BACKGROUND

Accurate interpretation of chest radiographs (CXR) is essential as clinical decisions depend on readings.

OBJECTIVE

We sought to evaluate CXR interpretation ability at different levels of training and to determine factors associated with successful interpretation.

DESIGN

Ten CXR were selected from the teaching file of the internal medicine (IM) department. Participants were asked to record the most important diagnosis, their certainty in that diagnosis, interest in a pulmonary career and adequacy of CXR training. Two investigators independently scored each CXR on a scale of 0 to 2.

PARTICIPANTS

Participants (n=145) from a single teaching hospital were third year medical students (MS) (n=25), IM interns (n=44), IM residents (n=45), fellows from the divisions of cardiology and pulmonary/critical care (n=16), and radiology residents (n=15).

RESULTS

The median overall score was 11 of 20. An increased level of training was associated with overall score (MS 8, intern 10, IM resident 13, fellow 15, radiology resident 18, P<.001). Overall certainty was significantly correlated with overall score (r=.613, P<.001). Internal medicine interns and residents interested in a pulmonary career scored 14 of 20 while those not interested scored 11 (P=.027). Pneumothorax, misplaced central line, and pneumoperitoneum were diagnosed correctly 9%, 26%, and 46% of the time, respectively. Only 20 of 131 (15%) participants felt their CXR training sufficient.

CONCLUSION

We identified factors associated with successful CXR interpretation, including level of training, field of training, interest in a pulmonary career and overall certainty. Although interpretation improved with training, important diagnoses were missed.

A preliminary clinical scale to predict the risk of in-hospital death after carotid endarterectomy

OBJECTIVE

Carotid endarterectomy (CEA) remains the gold standard for the treatment of carotid disease, with mortality rates generally at 0.4% to 1.7%. Controversy remains with regards to its role in the treatment of the high-risk surgical population. We developed a new clinical scale incorporating weighted risk factors into a single numerical score that correlates with the risk of in-hospital death after CEA. We propose that this tool may serve to prospectively identify the high-risk patient.

METHODS

We performed a retrospective analysis of 10 years (1994 to 2003) of the Maryland hospital discharge database. Included in the analysis were patients with (1) International Classification of Diseases, 9th Revision, Clinical Modification (ICD-9-CM) procedure code 38.12 (endarterectomy of the vessels of the head and neck other than intracranial vessels) in the primary coding position but not in any secondary position, or (2) Diagnosis Code 433.00 to 433.91 (occlusion/stenosis, precerebral artery), or (3) the Diagnosis-Related Group 5 (extracranial vascular procedure). ICD codes representing preoperative conditions of the patients were identified and evaluated with stepwise regression modeling techniques for association with in-hospital deaths. Different regression models were evaluated and compared by discriminative power as measured by receiver operating characteristics (ROC) and goodness-of-fit to data as measured by r(2) and the Hosmer-Lemeshow statistic. A numeric index correlating with the risk of in-hospital death was constructed by rounding the correlation coefficients for the statistically significant variables from the logistic regression.

RESULTS

We identified 23,237 cases. The mean age of patients was 70.6 years, with 54.7% male patients. There were 125 in-hospital deaths (0.54%). Patient age and four patient medical conditions emerged with significant associations with in-hospital deaths after CEA, and their relationships can be summarized in a single diagnostic scale: 1 point for age > or =75, 2 points for atherosclerosis (ICD code 440), 3 points for cardiomyopathy (ICD code 425), 4 points for iron-deficiency anemia (ICD code 280), and 5 points for cerebral degeneration (ICD code 331). This scale has moderate discriminative power (ROC = 0.67). On average, each point increase on this scale is associated with a 1.58-times increase in mortality risk, with score of 6 on the scale carrying a mortality risk >5%.

CONCLUSIONS

This new 5-item scale, based on patient age and past medical history, correlates moderately with the rate of in-hospital death after CEA. This clinical index may serve to identify high-risk patients. Future improvements to this diagnostic scale should focus on the diagnostic values of additional laboratory and demographic data.