The Usefulness of Computer-Aided Detection of Brain Metastases on Contrast-Enhanced Computed Tomography Using Single-Shot Multibox Detector: Observer Performance Study

OBJECTIVE

This study aimed to test the usefulness of computer-aided detection (CAD) for the detection of brain metastasis (BM) on contrast-enhanced computed tomography.

METHODS

The test data set included whole-brain axial contrast-enhanced computed tomography images of 25 cases with 62 BMs and 5 cases without BM. Six radiologists from 3 institutions with 2 to 4 years of experience independently reviewed the cases, both in conditions with and without CAD assistance. Sensitivity, positive predictive value, number of false positives, and reading time were compared between the conditions using paired t tests. Subanalysis was also performed for groups of lesions divided according to size. A P value <0.05 was considered statistically significant.

RESULTS

With CAD, sensitivity significantly increased from 80.4% to 83.9% (P = 0.04), whereas positive predictive value significantly decreased from 88.7% to 84.8% (P = 0.03). Reading time with and without CAD was 112 and 107 seconds, respectively (P = 0.38), and the number of false positives was 10.5 with CAD and 7.0 without CAD (P = 0.053). Sensitivity significantly improved for 6- to 12-mm lesions, from 71.2% without CAD to 80.3% with CAD (P = 0.02). The sensitivity of the CAD (95.2%) was significantly higher than that of any reader (with CAD: P = 0.01; without CAD: P = 0.005).

CONCLUSIONS

Computer-aided detection significantly improved BM detection sensitivity without prolonging reading time while marginally increased the false positives.

Impact of Vessel Suppressed-CT on Diagnostic Accuracy in Detection of Pulmonary Metastasis and Reading Time

RATIONALE AND OBJECTIVES

To assess if vessel suppression (VS) improves nodule detection rate, interreader agreement, and reduces reading time in oncologic chest computed tomography (CT).

MATERIAL AND METHODS

One-hundred consecutive oncologic patients (65 male; median age 60y) who underwent contrast-enhanced chest CT were retrospectively included. For all exams, additional VS series (ClearRead CT, Riverrain Technologies, Miamisburg) were reconstructed. Two groups of three radiologists each with matched experience were defined. Each group evaluated the SD-CT as well as VS-CT. Each reader marked the presence, size, and position of pulmonary nodules and documented reading time. In addition, for the VS-CT the presence of false positive nodules had to be stated. Cohen's Kappa (k) was used to calculate the interreader-agreement between groups. Reading time was compared using paired t test.

RESULTS

Nodule detection rate was significantly higher in VS-CT compared to the SD-CT (+21%; p <0.001). Interreader-agreement was higher in the VS-CT (k = 0.431, moderate agreement) compared to SD-CT (k = 0.209, fair agreement). Almost all VS-CT series had false positive findings (97-99 out of 100). Average reading time was significantly shorter in the VS-CT compared to the SD-CT (154 ± 134vs. 194 ± 126; 21%, p<0.001).

CONCLUSIONS

Vessel suppression increases nodule detection rate, improves interreader agreement, and reduces reading time in chest CT of oncologic patients. Due to false positive results a consensus reading with the SD-CT is essential.

Performance and reading time of lung nodule identification on multidetector CT with or without an artificial intelligence-powered computer-aided detection system

AIM

To compare the performance and reading time of different readers using automatic artificial intelligence (AI)-powered computer-aided detection (CAD) to detect lung nodules in different reading modes.

MATERIALS AND METHODS

One hundred and fifty multidetector computed tomography (CT) datasets containing 340 nodules ≤10 mm in diameter were collected retrospectively. A CAD with vessel-suppressed function was used to interpret the images. Three junior and three senior readers were assigned to read (1) CT images without CAD, (2) second-read using CAD in which CAD was applied only after initial unassisted assessment, and (3) a concurrent read with CAD in which CAD was applied at the start of assessment. Diagnostic performances and reading times were compared using analysis of variance.

RESULTS

For all readers, the mean sensitivity improved from 64% (95% confidence interval [CI]: 62%, 66%) for the without-CAD mode to 82% (95% CI: 80%, 84%) for the second-reading mode and to 80% (95% CI: 79%, 82%) for the concurrent-reading mode (p<0.001). There was no significant difference between the two modes in terms of the mean sensitivity, specificity, and area under the receiver operating characteristic curve (AUC) for both junior and senior readers and all readers (p>0.05). The reading time of all readers was significantly shorter for the concurrent-reading mode (124 ± 25 seconds) compared to without CAD (156 ± 34 seconds; p<0.001) and the second-reading mode (197 ± 46 seconds; p<0.001).

CONCLUSION

In CAD for lung nodules at CT, the second-reading mode and concurrent-reading mode may improve detection performance for all readers in both screening and clinical routine practice. Concurrent use of CAD is more efficient for both junior and senior readers.

A Bayesian Latent Class Model to Predict Kidney Obstruction in the Absence of Gold Standard

Kidney obstruction, if untreated in a timely manner, can lead to irreversible loss of renal function. A widely used technology for evaluations of kidneys with suspected obstruction is diuresis renography. However, it is generally very challenging for radiologists who typically interpret renography data in practice to build high level of competency due to the low volume of renography studies and insufficient training. Another challenge is that there is currently no gold standard for detection of kidney obstruction. Seeking to develop a computer-aided diagnostic (CAD) tool that can assist practicing radiologists to reduce errors in the interpretation of kidney obstruction, a recent study collected data from diuresis renography, interpretations on the renography data from highly experienced nuclear medicine experts as well as clinical data. To achieve the objective, we develop a statistical model that can be used as a CAD tool for assisting radiologists in kidney interpretation. We use a Bayesian latent class modeling approach for predicting kidney obstruction through the integrative analysis of time-series renogram data, expert ratings, and clinical variables. A nonparametric Bayesian latent factor regression approach is adopted for modeling renogram curves in which the coefficients of the basis functions are parameterized via the factor loadings dependent on the latent disease status and the extended latent factors that can also adjust for clinical variables. A hierarchical probit model is used for expert ratings, allowing for training with rating data from multiple experts while predicting with at most one expert, which makes the proposed model operable in practice. An efficient MCMC algorithm is developed to train the model and predict kidney obstruction with associated uncertainty. We demonstrate the superiority of the proposed method over several existing methods through extensive simulations. Analysis of the renal study also lends support to the usefulness of our model as a CAD tool to assist less experienced radiologists in the field.

Computer-aided detection of brain metastasis on 3D MR imaging: Observer performance study

PURPOSE

To assess the effect of computer-aided detection (CAD) of brain metastasis (BM) on radiologists' diagnostic performance in interpreting three-dimensional brain magnetic resonance (MR) imaging using follow-up imaging and consensus as the reference standard.

MATERIALS AND METHODS

The institutional review board approved this retrospective study. The study cohort consisted of 110 consecutive patients with BM and 30 patients without BM. The training data set included MR images of 80 patients with 450 BM nodules. The test set included MR images of 30 patients with 134 BM nodules and 30 patients without BM. We developed a CAD system for BM detection using template-matching and K-means clustering algorithms for candidate detection and an artificial neural network for false-positive reduction. Four reviewers (two neuroradiologists and two radiology residents) interpreted the test set images before and after the use of CAD in a sequential manner. The sensitivity, false positive (FP) per case, and reading time were analyzed. A jackknife free-response receiver operating characteristic (JAFROC) method was used to determine the improvement in the diagnostic accuracy.

RESULTS

The sensitivity of CAD was 87.3% with an FP per case of 302.4. CAD significantly improved the diagnostic performance of the four reviewers with a figure-of-merit (FOM) of 0.874 (without CAD) vs. 0.898 (with CAD) according to JAFROC analysis (p < 0.01). Statistically significant improvement was noted only for less-experienced reviewers (FOM without vs. with CAD, 0.834 vs. 0.877, p < 0.01). The additional time required to review the CAD results was approximately 72 sec (40% of the total review time).

CONCLUSION

CAD as a second reader helps radiologists improve their diagnostic performance in the detection of BM on MR imaging, particularly for less-experienced reviewers.

Progress in Fully Automated Abdominal CT Interpretation

OBJECTIVE

Automated analysis of abdominal CT has advanced markedly over just the last few years. Fully automated assessment of organs, lymph nodes, adipose tissue, muscle, bowel, spine, and tumors are some examples where tremendous progress has been made. Computer-aided detection of lesions has also improved dramatically.

CONCLUSION

This article reviews the progress and provides insights into what is in store in the near future for automated analysis for abdominal CT, ultimately leading to fully automated interpretation.

Developments in Screening Tests and Strategies for Colorectal Cancer

BACKGROUND

Worldwide, colorectal cancer (CRC) is the third most common cancer in men and second most common in women. It is the fourth most common cause of cancer mortality. In the United States, CRC is the third most common cause of cancer and second most common cause of cancer mortality. Incidence and mortality rates have steadily fallen, primarily due to widespread screening.

METHODS

We conducted keyword searches on PubMed in four categories of CRC screening: stool, endoscopic, radiologic, and serum, as well as news searches in Medscape and Google News.

RESULTS

Colonoscopy is the gold standard for CRC screening and the most common method in the United States. Technological improvements continue to be made, including the promising "third-eye retroscope." Fecal occult blood remains widely used, particularly outside the United States. The first at-home screen, a fecal DNA screen, has also recently been approved. Radiological methods are effective but seldom used due to cost and other factors. Serum tests are largely experimental, although at least one is moving closer to market.

CONCLUSIONS

Colonoscopy is likely to remain the most popular screening modality for the immediate future, although its shortcomings will continue to spur innovation in a variety of modalities.

Assessment of the Incremental Benefit of Computer-Aided Detection (CAD) for Interpretation of CT Colonography by Experienced and Inexperienced Readers

OBJECTIVES

To quantify the incremental benefit of computer-assisted-detection (CAD) for polyps, for inexperienced readers versus experienced readers of CT colonography.

METHODS

10 inexperienced and 16 experienced radiologists interpreted 102 colonography studies unassisted and with CAD utilised in a concurrent paradigm. They indicated any polyps detected on a study sheet. Readers' interpretations were compared against a ground-truth reference standard: 46 studies were normal and 56 had at least one polyp (132 polyps in total). The primary study outcome was the difference in CAD net benefit (a combination of change in sensitivity and change in specificity with CAD, weighted towards sensitivity) for detection of patients with polyps.

RESULTS

Inexperienced readers' per-patient sensitivity rose from 39.1% to 53.2% with CAD and specificity fell from 94.1% to 88.0%, both statistically significant. Experienced readers' sensitivity rose from 57.5% to 62.1% and specificity fell from 91.0% to 88.3%, both non-significant. Net benefit with CAD assistance was significant for inexperienced readers but not for experienced readers: 11.2% (95%CI 3.1% to 18.9%) versus 3.2% (95%CI -1.9% to 8.3%) respectively.

CONCLUSIONS

Concurrent CAD resulted in a significant net benefit when used by inexperienced readers to identify patients with polyps by CT colonography. The net benefit was nearly four times the magnitude of that observed for experienced readers. Experienced readers did not benefit significantly from concurrent CAD.

History, evolution, and current status of radiologic imaging tests for colorectal cancer screening

Colorectal cancer screening is thought to be an effective tool with which to reduce the mortality from colorectal cancer through early detection and removal of colonic adenomas and early colon cancers. In this article, we review the history, evolution, and current status of imaging tests of the colon-including single-contrast barium enema, double-contrast barium enema, computed tomographic (CT) colonography, and magnetic resonance (MR) colonography-for colorectal cancer screening. Despite its documented value in the detection of colonic polyps, the double-contrast barium enema has largely disappeared as a screening test because it is widely perceived as a labor-intensive, time-consuming, and technically demanding procedure. In the past decade, the barium enema has been supplanted by CT colonography as the major imaging test in colorectal cancer screening in the United States, with MR colonography emerging as another viable option in Europe. Although MR colonography does not require ionizing radiation, the radiation dose for CT colonography has decreased substantially, and regular screening with this technique has a high benefit-to-risk ratio. In recent years, CT colonography has been validated as an effective tool for use in colorectal cancer screening that is increasingly being disseminated.

Population screening for colorectal cancer by flexible sigmoidoscopy or CT colonography: study protocol for a multicenter randomized trial

BACKGROUND

Colorectal cancer (CRC) is the second most prevalent type of cancer in Europe. A single flexible sigmoidoscopy (FS) screening at around the age of 60 years prevents about one-third of CRC cases. However, FS screens only the distal colon, and thus mortality from proximal CRC is unaffected. Computed tomography colonography (CTC) is a highly accurate examination that allows assessment of the entire colon. However, the benefit of CTC testing as a CRC screening test is uncertain. We designed a randomized trial to compare participation rate, detection rates, and costs between CTC (with computer-aided detection) and FS as primary tests for population-based screening.

METHODS/DESIGN

An invitation letter to participate in a randomized screening trial comparing CTC versus FS will be mailed to a sample of 20,000 people aged 58 or 60 years, living in the Piedmont region and the Verona district of Italy. Individuals with a history of CRC, adenomas, inflammatory bowel disease, or recent colonoscopy, or with two first-degree relatives with CRC will be excluded from the study by their general practitioners. Individuals responding positively to the invitation letter will be then randomized to the intervention group (CTC) or control group (FS), and scheduled for the screening procedure. The primary outcome parameter of this part of the trial is the difference in advanced neoplasia detection between the two screening tests. Secondary outcomes are cost-effectiveness analysis, referral rates for colonoscopy induced by CTC versus FS, and the expected and perceived burden of the procedures. To compare participation rates for CTC versus FS, 2,000 additional eligible subjects will be randomly assigned to receive an invitation for screening with CTC or FS. In the CTC arm, non-responders will be offered fecal occult blood test (FOBT) as alternative screening test, while in the FS arm, non-responders will receive an invitation letter to undergo screening with either FOBT or CTC. Data on reasons for participation and non-participation will also be collected.

DISCUSSION

This study will provide reliable information concerning benefits and risks of the adoption of CTC as a mass screening intervention in comparison with FS. The trial will also evaluate the role of computer-aided detection in a screening setting.

TRIAL REGISTRATION

ClinicalTrials.gov Identifier: NCT01739608.

Evaluation of computer-aided detection and diagnosis systems

Computer-aided detection and diagnosis (CAD) systems are increasingly being used as an aid by clinicians for detection and interpretation of diseases. Computer-aided detection systems mark regions of an image that may reveal specific abnormalities and are used to alert clinicians to these regions during image interpretation. Computer-aided diagnosis systems provide an assessment of a disease using image-based information alone or in combination with other relevant diagnostic data and are used by clinicians as a decision support in developing their diagnoses. While CAD systems are commercially available, standardized approaches for evaluating and reporting their performance have not yet been fully formalized in the literature or in a standardization effort. This deficiency has led to difficulty in the comparison of CAD devices and in understanding how the reported performance might translate into clinical practice. To address these important issues, the American Association of Physicists in Medicine (AAPM) formed the Computer Aided Detection in Diagnostic Imaging Subcommittee (CADSC), in part, to develop recommendations on approaches for assessing CAD system performance. The purpose of this paper is to convey the opinions of the AAPM CADSC members and to stimulate the development of consensus approaches and "best practices" for evaluating CAD systems. Both the assessment of a standalone CAD system and the evaluation of the impact of CAD on end-users are discussed. It is hoped that awareness of these important evaluation elements and the CADSC recommendations will lead to further development of structured guidelines for CAD performance assessment. Proper assessment of CAD system performance is expected to increase the understanding of a CAD system's effectiveness and limitations, which is expected to stimulate further research and development efforts on CAD technologies, reduce problems due to improper use, and eventually improve the utility and efficacy of CAD in clinical practice.

Multiple sclerosis: identification of temporal changes in brain lesions with computer-assisted detection software

Multiple sclerosis (MS) is a chronic disease with a progressing and evolving course. Serial imaging with MRI is the mainstay in monitoring and managing MS patients. In this work we demonstrate the performance of a locally developed computer-assisted detection (CAD) software used to track temporal changes in brain MS lesions. CAD tracks changes in T2-bright MS lesions between two time points on a 3D high-resolution isotropic FLAIR MR sequence of the brain acquired at 3 Tesla. The program consists of an image-processing pipeline, and displays scrollable difference maps used as an aid to the neuroradiologist for assessing lesional change. To assess the value of the software we have compared diagnostic accuracy and duration of interpretation of the CAD-assisted and routine clinical interpretations in 98 randomly chosen, paired MR examinations from 88 patients (68 women, 20 men, mean age 43.5, age range 21-75) with a diagnosis of definite MS. The ground truth was determined by a three-expert panel. In case-wise analysis, CAD interpretation showed higher sensitivity than a clinical report (87% vs 77%, respectively). Lesion-wise analysis demonstrated improved sensitivity of CAD over a routine clinical interpretation of 40%-48%. Mean software-assisted interpretation time was 2.7 min. Our study demonstrates the potential of including CAD software in the workflow of neuroradiology practice for the detection of MS lesional change. Automated quantification of temporal change in MS lesion load may also be used in clinical research, e.g., in drug trials.

Efficacy of computer-aided detection as a second reader for 6-9-mm lesions at CT colonography: multicenter prospective trial

PURPOSE

To assess the effect of computer-aided detection (CAD) as a second reader on the sensitivity and specificity of computed tomographic (CT) colonography in detecting 6-9-mm colorectal cancer (CRC) lesions.

MATERIALS AND METHODS

Individuals with clinical indications for colonoscopy--either for symptoms or as part of participating in a surveillance program or CRC screening--were prospectively enrolled at one of 10 academic centers between July 2007 and May 2009. Institutional review board approval was obtained at each clinical site, and all participants provided written informed consent. All participants underwent CT colonography and colonoscopy on the same day. Experienced readers interpreted the CT colonography images unassisted and then reviewed all colorectal lesion-like structures pinpointed by the CAD algorithm. Segmental unblinding of CT colonoscopy findings at colonoscopy was utilized. The sensitivity and specificity of unassisted and CAD-assisted reading in identifying individuals with 6-9-mm lesions were calculated and compared by means of pairwise analysis.

RESULTS

A total of 618 participants (mean age, 57.9 years; 54.5% male) were included in the final analysis. Of these participants, 464 (75.1%) had no lesions 6 mm or larger, and 52 (8.4%) had 6-9-mm lesions. The sensitivity of CT colonography with unassisted reading and that with CAD-assisted reading in identifying individuals with 6-9-mm lesions was 65.4% (95% confidence interval [CI]: 50.9%, 78.0%) and 76.9% (95% CI: 63.2%, 87.5%; P = .016), respectively. No significant change in specificity was observed: The specificity of CT colonography with unassisted and that with CAD-assisted reading was 91.8% (95% CI: 88.9%, 94.1%) and 90.9% (95% CI: 88.0%, 93.4%; P = .063), respectively. Evaluation of CAD candidates required an additional 1.6 minutes (25th-75th percentile: 1.0 minute to 3.4 minutes).

CONCLUSION

The addition of CAD to reading performed by experienced readers resulted in a significant benefit in the detection of 6-9-mm polyps at CT colonography in this cohort.

SUPPLEMENTAL MATERIAL

http://radiology.rsna.org/lookup/suppl/doi:10.1148/radiol.12120376/-/DC1.

The second ESGAR consensus statement on CT colonography

Objective

To update quality standards for CT colonography based on consensus among opinion leaders within the European Society of Gastrointestinal and Abdominal Radiology (ESGAR).

Material and methods

A multinational European panel of nine members of the ESGAR CT colonography Working Group (representing six EU countries) used a modified Delphi process to rate their level of agreement on a variety of statements pertaining to the acquisition, interpretation and implementation of CT colonography. Four Delphi rounds were conducted, each at 2 months interval.

Results

The panel elaborated 86 statements.

In the final round the panelists achieved complete consensus in 71 of 86 statements (82 %). Categories including the highest proportion of statements with excellent Cronbach's internal reliability were colon distension, scan parameters, use of intravenous contrast agents, general guidelines on patient preparation, role of CAD and lesion measurement.

Lower internal reliability was achieved for the use of a rectal tube, spasmolytics, decubitus positioning and number of CT data acquisitions, faecal tagging, 2D vs. 3D reading, and reporting.

Conclusion

The recommendations of the consensus should be useful for both the radiologist who is starting a CTC service and for those who have already implemented the technique but whose practice may need updating.

Key Points

• Computed tomographic colonography is the optimal radiological method of assessing the colon

• This article reviews ESGAR quality standards for CT colonography

• This article is aimed to provide CT-colonography guidelines for practising radiologists

• The recommendations should help radiologists who are starting/updating their CTC services

CAD-associated reader error in CT colonography

RATIONALE AND OBJECTIVES

Computed tomographic colonographic interpretation with computer-aided detection (CAD) may be superior to unaided viewing, although polyp characteristics may influence accuracy. Reader error due to polyp characteristics was evaluated in a multiple-case, multiple-reader trial of computed tomographic colonography with CAD.

MATERIALS AND METHODS

Two experts retrospectively reviewed 52 positive cases (74 polyps) and categorized them as hard, moderate, or easy to detect. Each case was evaluated without and with CAD. Features that may influence a reader's ability to detect a polyp or to accept or reject a CAD mark were tabulated. The association between polyp characteristics and detection rates in the trial was assessed. The difference in detection rates (CAD vs unassisted) was calculated, and regression analysis was performed.

RESULTS

Of 64 polyps found by CAD, experts categorized 20 as hard, 28 as moderate, and 16 as easy to detect. Reader characterization errors predominated (47.3%) over other errors. Factors associated with lower detection rates included small size, flat morphology, and resemblance to a thickened fold. CAD was superior for polyps resembling lipomas compared to those that did not resemble lipomas (average increase in detection rate with CAD, 12.8% vs 5.5%; P < .05).

CONCLUSIONS

Polyp characteristic may impair computed tomographic colonographic interpretation augmented by CAD. Readers can avoid errors of measurement by evaluating diminutive polyp candidates with sample measurements. Caution should be taken when evaluating focally thick folds and when using visual impression to dismiss a polyp candidate as a lipoma when it is submerged in densely tagged fluid.

Multiple Sclerosis Lesions in the Brain: Computer-Assisted Assessment of Lesion Load Dynamics on 3D FLAIR MR Images

The detection and monitoring of brain lesions caused by multiple sclerosis is commonly performed with the use of magnetic resonance imaging. Analysis of a large number of images is a time-consuming challenge to the neuroradiologist, that can be accelerated with the assistance of computer-detection software. In 98 baseline and follow-up brain magnetic resonance studies from 88 patients with a diagnosis of multiple sclerosis, we employed locally developed lesion-detection software to assess temporal change in the load of brain lesions and compared its results to routine clinical reports. Analyzing the differences between the follow-up study and the baseline study, the software displays the results in the form of a scrollable axial volume, with the changed lesions highlighted in different colors and superimposed on the baseline reference scan. Disagreements between the software and the clinical readers in the detection of changed lesions were observed only in 11 (11.2%) cases, and the difference did not reach statistical significance (p=0.07). The mean interpretation time with assistance of the software was 2.7±2.2 minutes. We conclude that the performance of the software-assisted interpretation in the analysis of change over time in multiple sclerosis brain lesions is comparable to the performance of clinical readers, with a possibly shorter assessment time. Our study demonstrates the potential of including lesion-detection software in the workflow of neuroradiology practice.

Distributed human intelligence for colonic polyp classification in computer-aided detection for CT colonography

PURPOSE

To assess the diagnostic performance of distributed human intelligence for the classification of polyp candidates identified with computer-aided detection (CAD) for computed tomographic (CT) colonography.

MATERIALS AND METHODS

This study was approved by the institutional Office of Human Subjects Research. The requirement for informed consent was waived for this HIPAA-compliant study. CT images from 24 patients, each with at least one polyp of 6 mm or larger, were analyzed by using CAD software to identify 268 polyp candidates. Twenty knowledge workers (KWs) from a crowdsourcing platform labeled each polyp candidate as a true or false polyp. Two trials involving 228 KWs were conducted to assess reproducibility. Performance was assessed by comparing the area under the receiver operating characteristic curve (AUC) of KWs with the AUC of CAD for polyp classification.

RESULTS

The detection-level AUC for KWs was 0.845 ± 0.045 (standard error) in trial 1 and 0.855 ± 0.044 in trial 2. These were not significantly different from the AUC for CAD, which was 0.859 ± 0.043. When polyp candidates were stratified by difficulty, KWs performed better than CAD on easy detections; AUCs were 0.951 ± 0.032 in trial 1, 0.966 ± 0.027 in trial 2, and 0.877 ± 0.048 for CAD (P = .039 for trial 2). KWs who participated in both trials showed a significant improvement in performance going from trial 1 to trial 2; AUCs were 0.759 ± 0.052 in trial 1 and 0.839 ± 0.046 in trial 2 (P = .041).

CONCLUSION

The performance of distributed human intelligence is not significantly different from that of CAD for colonic polyp classification.

Role of CT colonography in symptomatic assessment, surveillance and screening

INTRODUCTION

When 'whole colonic imaging' is indicated, clinicians must decide between optical colonoscopy, barium enema and CT colonography (CTC). CTC is a relatively new technique which has become increasingly accessible in the UK over the past 5 years. As radiologists have gained experience and scanning parameters have standardised, there have been substantial improvements in both the accuracy and safety of CTC.

METHODS

We review evidence from observational studies and randomised trials, and draw on expert opinion, to provide a comprehensive discussion of the current role of CTC in both symptomatic and asymptomatic individuals.

CONCLUSIONS

The emergence of CTC could soon entirely obviate the need for barium enema. CTC now has a complementary role alongside colonoscopy in symptomatic patients and a possible future role in colorectal cancer screening in the UK.

Computer-aided diagnosis of renal obstruction: utility of log-linear modeling versus standard ROC and kappa analysis

Background

The accuracy of computer-aided diagnosis (CAD) software is best evaluated by comparison to a gold standard which represents the true status of disease. In many settings, however, knowledge of the true status of disease is not possible and accuracy is evaluated against the interpretations of an expert panel. Common statistical approaches to evaluate accuracy include receiver operating characteristic (ROC) and kappa analysis but both of these methods have significant limitations and cannot answer the question of equivalence: Is the CAD performance equivalent to that of an expert? The goal of this study is to show the strength of log-linear analysis over standard ROC and kappa statistics in evaluating the accuracy of computer-aided diagnosis of renal obstruction compared to the diagnosis provided by expert readers.

Methods

Log-linear modeling was utilized to analyze a previously published database that used ROC and kappa statistics to compare diuresis renography scan interpretations (non-obstructed, equivocal, or obstructed) generated by a renal expert system (RENEX) in 185 kidneys (95 patients) with the independent and consensus scan interpretations of three experts who were blinded to clinical information and prospectively and independently graded each kidney as obstructed, equivocal, or non-obstructed.

Results

Log-linear modeling showed that RENEX and the expert consensus had beyond-chance agreement in both non-obstructed and obstructed readings (both p < 0.0001). Moreover, pairwise agreement between experts and pairwise agreement between each expert and RENEX were not significantly different (p = 0.41, 0.95, 0.81 for the non-obstructed, equivocal, and obstructed categories, respectively). Similarly, the three-way agreement of the three experts and three-way agreement of two experts and RENEX was not significantly different for non-obstructed (p = 0.79) and obstructed (p = 0.49) categories.

Conclusion

Log-linear modeling showed that RENEX was equivalent to any expert in rating kidneys, particularly in the obstructed and non-obstructed categories. This conclusion, which could not be derived from the original ROC and kappa analysis, emphasizes and illustrates the role and importance of log-linear modeling in the absence of a gold standard. The log-linear analysis also provides additional evidence that RENEX has the potential to assist in the interpretation of diuresis renography studies.

Computed tomographic colonography: ready for prime time?

Computed tomographic (CT) colonography is a noninvasive method to evaluate the colon and has received considerable attention in the last decade as a colon-imaging tool. The technique has also been proposed as a potential primary colon cancer-screening method in the United States. The accuracy of the technique for the detection of large lesions seems to be high, perhaps in the range of colonoscopy. Overall, the field is rapidly evolving. Available data suggest that CT colonography, although a viable colon cancer screening modality in the United States, is not ready for widespread implementation, largely because of the lack of standards for training and reading and the limited number of skilled readers.

CT colonography: computer-assisted detection of colorectal cancer

OBJECTIVES

Computer-aided detection (CAD) for CT colonography (CTC) has been developed to detect benign polyps in asymptomatic patients. We aimed to determine whether such a CAD system can also detect cancer in symptomatic patients.

METHODS

CTC data from 137 symptomatic patients subsequently proven to have colorectal cancer were analysed by a CAD system at 4 different sphericity settings: 0, 50, 75 and 100. CAD prompts were classified by an observer as either true-positive if overlapping a cancer or false-positive if elsewhere. Colonoscopic data were used to aid matching.

RESULTS

Of 137 cancers, CAD identified 124 (90.5%), 122 (89.1%), 119 (86.9%) and 102 (74.5%) at a sphericity of 0, 50, 75 and 100, respectively. A substantial proportion of cancers were detected on either the prone or supine acquisition alone. Of 125 patients with prone and supine acquisitions, 39.3%, 38.3%, 43.2% and 50.5% of cancers were detected on a single acquisition at a sphericity of 0, 50, 75 and 100, respectively. CAD detected three cancers missed by radiologists at the original clinical interpretation. False-positive prompts decreased with increasing sphericity value (median 65, 57, 45, 24 per patient at values of 0, 50, 75, 100, respectively) but many patients were poorly prepared.

CONCLUSION

CAD can detect symptomatic colorectal cancer but must be applied to both prone and supine acquisitions for best performance.

CT colonography computer-aided polyp detection: Effect on radiologist observers of polyp identification by CAD on both the supine and prone scans

RATIONALE AND OBJECTIVES

To determine whether the display of computer-aided detection (CAD) marks on individual polyps on both the supine and prone scans leads to improved polyp detection by radiologists compared to the display of CAD marks on individual polyps on either the supine or the prone scan, but not both.

MATERIALS AND METHODS

The acquisition of patient data for this study was approved by the Institutional Review Board and was Health Insurance Portability and Accountability Act-compliant. Subsequently, the use of the data was declared exempt from further institutional review board review. Four radiologists interpreted 33 computed tomography colonography cases, 21 of which had one adenoma 6-9 mm in size, with the assistance of a CAD system in the first reader mode (ie, the radiologists reviewed only the CAD marks). The radiologists were shown each case twice, with different sets of CAD marks for each of the two readings. In one reading, a true-positive CAD mark for the same polyp was displayed on both the supine and prone scans (a double-mark reading). In the other reading, a true-positive CAD mark was displayed either on the supine or prone scan, but not both (a single-mark reading). True-positive marks were randomized between readings and there was at least a 1-month delay between readings to minimize recall bias. Sensitivity and specificity were determined and receiver operating characteristic (ROC) and multiple-reader multiple-case analyses were performed.

RESULTS

The average per polyp sensitivities were 60% (38%-81%) versus 71% (52%-91%) (P = .03) for single-mark and double-mark readings, respectively. The areas (95% confidence intervals) under the ROC curves were 0.76 (0.62-0.88) and 0.79 (0.58-0.96), respectively (P = NS). Specificities were similar for the single-mark compared with the double-mark readings.

CONCLUSION

The display of CAD marks on a polyp on both the supine and prone scans led to more frequent detection of polyps by radiologists without adversely affecting specificity for detecting 6-9 mm adenomas.

Effect of computer-aided detection for CT colonography in a multireader, multicase trial

PURPOSE

To assess the effect of using computer-aided detection (CAD) in second-read mode on readers' accuracy in interpreting computed tomographic (CT) colonographic images.

MATERIALS AND METHODS

The contributing institutions performed the examinations under approval of their local institutional review board, with waiver of informed consent, for this HIPAA-compliant study. A cohort of 100 colonoscopy-proved cases was used: In 52 patients with findings positive for polyps, 74 polyps of 6 mm or larger were observed in 65 colonic segments; in 48 patients with findings negative for polyps, no polyps were found. Nineteen blinded readers interpreted each case at two different times, with and without the assistance of a commercial CAD system. The effect of CAD was assessed in segment-level and patient-level receiver operating characteristic (ROC) curve analyses.

RESULTS

Thirteen (68%) of 19 readers demonstrated higher accuracy with CAD, as measured with the segment-level area under the ROC curve (AUC). The readers' average segment-level AUC with CAD (0.758) was significantly greater (P = .015) than the average AUC in the unassisted read (0.737). Readers' per-segment, per-patient, and per-polyp sensitivity for all polyps of 6 mm or larger was higher (P < .011, .007, .005, respectively) for readings with CAD compared with unassisted readings (0.517 versus 0.465, 0.521 versus 0.466, and 0.477 versus 0.422, respectively). Sensitivity for patients with at least one large polyp of 10 mm or larger was also higher (P < .047) with CAD than without (0.777 versus 0.743). Average reader sensitivity also improved with CAD by more than 0.08 for small adenomas. Use of CAD reduced specificity of readers by 0.025 (P = .05).

CONCLUSION

Use of CAD resulted in a significant improvement in overall reader performance. CAD improves reader sensitivity when measured per segment, per patient, and per polyp for small polyps and adenomas and also reduces specificity by a small amount.

Colorectal polyps: stand-alone performance of computer-aided detection in a large asymptomatic screening population

PURPOSE

To evaluate stand-alone performance of computer-aided detection (CAD) for colorectal polyps of 6 mm or larger at computed tomographic (CT) colonography in a large asymptomatic screening cohort.

MATERIALS AND METHODS

In this retrospective, institutional review board-approved, HIPAA-compliant study, a CAD software system was applied to screening CT colonography in 1638 women and 1408 men (mean age, 56.9 years) evaluated at a single medical center between March 2006 and December 2008. All participants underwent cathartic preparation with stool tagging; electronic cleansing was not used. The reference standard consisted of interpretation by experienced radiologists in all cases. This interpretation was further refined for the subset of cases with positive findings by using subsequent colonoscopic or CT colonographic confirmation, as well as retrospective expert localization of polyps with CT colonography. This test set was not involved in training the CAD system. The Fisher exact test was used to evaluate significance; 95% confidence intervals (CIs) were obtained by using the exact method.

RESULTS

Per-patient CAD sensitivities were 93.8% (350 of 373; 95% CI: 90.9%, 96.1%) and 96.5% (137 of 142; 95% CI: 92.0%, 98.8%) at 6- and 10-mm threshold sizes, respectively. Per-polyp CAD sensitivities for all polyps, regardless of histologic features, were 90.1% (547 of 607; 95% CI: 88.0%, 92.8%) and 96.0% (168 of 175; 95% CI: 91.9%, 98.4%) at 6- and 10-mm threshold sizes, respectively; CAD sensitivities for advanced neoplasia and cancer were 97.0% (128 of 132; 95% CI: 92.4%, 99.2%) and 100% (13 of 13; 95% CI: 79.4%, 100%), respectively. The mean and median false-positive rates were 4.7 and 3 per series, respectively (9.4 and 6 per patient). Among 373 patients with a positive finding at CT colonography, CAD marked an additional 15 polyps of 6 mm or larger, including four large polyps, that were missed at the prospective three-dimensional reading by an expert but were found at subsequent colonoscopy.

CONCLUSION

Stand-alone CAD demonstrated excellent performance for polyp detection in a large screening population, with high sensitivity and an acceptable number of false-positive results.

Computer-aided Detection in Computed Tomography Colonography with Full Fecal Tagging: Comparison of Standalone Performance of 3 Automated Polyp Detection Systems

Purpose

We sought to compare the performance of 3 computer-aided detection (CAD) polyp algorithms in computed tomography colonography (CTC) with fecal tagging.

Methods

CTC data sets of 33 patients were retrospectively analysed by 3 different CAD systems: system 1, MedicSight; system 2, Colon CAD; and system 3, Polyp Enhanced View. The polyp database comprised 53 lesions, including 6 cases of colorectal cancer, and was established by consensus reading and comparison with colonoscopy. Lesions ranged from 6-40 mm, with 25 lesions larger than 10 mm in size. Detection and false-positive (FP) rates were calculated.

Results

CAD systems 1 and 2 could be set to have varying sensitivities with higher FP rates for higher sensitivity levels. Sensitivities for system 1 ranged from 73%–94% for all lesions (78%–100% for lesions ≥10 mm) and, for system 2, from 64%–94% (78%–100% for lesions ≥10 mm). System 3 reached an overall sensitivity of 76% (100% for lesions ≥10 mm). The mean FP rate per patient ranged from 8–32 for system 1, from 1–8 for system 2, and was 5 for system 3. At the highest sensitivity level for all polyps (94%), system 2 showed a statistically significant lower FP rate compared with system 1 ( P = .001). When analysing lesions ≥10 mm, system 3 had significantly fewer FPs than systems 1 and 2 ( P < .012).

Conclusions

Standalone CTC-CAD analysis in the selected patient collective showed the 3 systems tested to have a variable but overall promising performance with respect to sensitivity and the FP rate.

Improving the accuracy of CTC interpretation: computer-aided detection

Computer-aided polyp detection aims to improve the accuracy of the colonography interpretation. The computer searches the colonic wall to look for polyplike protrusions and presents a list of suspicious areas to a physician for further analysis. Computer-aided polyp detection has developed rapidly in the past decade in the laboratory setting and has sensitivities comparable with those of experts. Computer-aided polyp detection tends to help inexperienced readers more than experienced ones and may also lead to small reductions in specificity. In its currently proposed use as an adjunct to standard image interpretation, computer-aided polyp detection serves as a spellchecker rather than an efficiency enhancer.

Reader performance in radiographic diagnosis of signs of mitral regurgitation in cavalier King Charles spaniels

OBJECTIVES

To measure accuracy and variability of diagnosis by radiography of heart enlargement (HE) and heart failure (HF) in mitral regurgitation (MR).

METHODS

Sixteen readers representing four levels of experience evaluated 50 sets of radiographs with varying severity of MR for presence or absence of HE, left atrial enlargement (LAE) and HF. The performance of the readers was compared with a reference standard, using area under the curve (AUC) of receiver operating characteristic (ROC) curves. The interreader agreement value kappa (K) was calculated. A subset of difficult cases of HF was analysed before and after removing an outlying reader from each group.

RESULTS

AUC for HE was 0.89, for LAE it was 0.93 and for HF it was 0.92. Experience increased certainty of diagnosis but not accuracy. K ranges were HE, 0.53 to 0.67; LAE, 0.61 to 0.69 and HF, 0.49 to 0.58. When only difficult cases of HF were read, accuracy decreased and experienced readers performed better than inexperienced. When outlying readers were excluded, the differences between experienced and inexperienced readers increased.

CLINICAL SIGNIFICANCE

LAE, not HE, should be used to evaluate the heart size and indirectly the severity of MR on radiographs. For HF, agreement among individual readers was only moderate. Studies of reader accuracy should consider the effects of interreader variability.

Effect of CAD on radiologists' detection of lung nodules on thoracic CT scans: analysis of an observer performance study by nodule size

RATIONALE AND OBJECTIVES

To retrospectively investigate the effect of a computer-aided detection (CAD) system on radiologists' performance for detecting small pulmonary nodules in computed tomography (CT) examinations, with a panel of expert radiologists serving as the reference standard.

MATERIALS AND METHODS

Institutional review board approval was obtained. Our dataset contained 52 CT examinations collected by the Lung Image Database Consortium, and 33 from our institution. All CTs were read by multiple expert thoracic radiologists to identify the reference standard for detection. Six other thoracic radiologists read the CT examinations first without and then with CAD. Performance was evaluated using free-response receiver operating characteristics (FROC) and the jackknife FROC analysis methods (JAFROC) for nodules above different diameter thresholds.

RESULTS

A total of 241 nodules, ranging in size from 3.0 to 18.6 mm (mean, 5.3 mm) were identified as the reference standard. At diameter thresholds of 3, 4, 5, and 6 mm, the CAD system had a sensitivity of 54%, 64%, 68%, and 76%, respectively, with an average of 5.6 false positives (FPs) per scan. Without CAD, the average figures of merit (FOMs) for the six radiologists, obtained from JAFROC analysis, were 0.661, 0.729, 0.793, and 0.838 for the same nodule diameter thresholds, respectively. With CAD, the corresponding average FOMs improved to 0.705, 0.763, 0.810, and 0.862, respectively. The improvement achieved statistical significance for nodules at the 3 and 4 mm thresholds (P = .002 and .020, respectively), and did not achieve significance at 5 and 6 mm (P = .18 and .13, respectively). At a nodule diameter threshold of 3 mm, the radiologists' average sensitivity and FP rate were 0.56 and 0.67, respectively, without CAD, and 0.67 and 0.78 with CAD.

CONCLUSION

CAD improves thoracic radiologists' performance for detecting pulmonary nodules smaller than 5 mm on CT examinations, which are often overlooked by visual inspection alone.

Polyp characteristics correctly annotated by computer-aided detection software but ignored by reporting radiologists during CT colonography

PURPOSE

To retrospectively describe the characteristics of polyps incorrectly dismissed by radiologists despite appropriate computer-aided detection (CAD) prompting during computed tomographic (CT) colonography.

MATERIALS AND METHODS

Ethics committee approval and patient informed consent were obtained from institutions that provided the data sets used in this HIPAA-compliant study. A total of 111 polyps that had a diameter of at least 6 mm and were detected with CAD were collated from three previous studies in which researchers investigated radiologist performance with and without CAD (total, 25 readers). Two new observers graded each polyp with predefined criteria, including polyp size, morphology, and location; data set quality; ease of visualization; tagging use and polyp coating; colonic curvature; CAD mark obscuration; and number of false-positive findings. The 86 polyps that were missed before CAD (those that were unreported by one or more original readers) were divided into those that remained unreported after CAD (no CAD gain, n = 36) and those that were reported correctly by at least one additional reader (CAD gain, n = 50). Logistic-regression analysis and the Fisher exact and Mann-Whitney tests were used to compare the results of both groups with each other and with a control group of 25 polyps, all of which were detected by readers without CAD.

RESULTS

Before CAD, polyps 10 mm in diameter or larger, those that were rated easy to visualize, and those that were uncoated by tagged fluid were less likely to be missed (72%, 76%, and 80% of control polyps vs 43%, 43%, and 59% of missed polyps, respectively; P < .001, P < .01, and P < .03, respectively). After CAD, the odds of CAD gain decreased with increasing polyp size (odds ratio, 0.92; 95% confidence interval: 0.85, 1.00; P = .04) and irregular morphology (odds ratio, 0.28; 95% confidence interval: 0.08, 0.92; P = .04).

CONCLUSION

Larger irregular polyps are a common source of incorrect radiologist dismissal, despite correct CAD prompting.

Computer-aided detection (CAD) as a second reader using perspective filet view at CT colonography: effect on performance of inexperienced readers

AIM

To evaluate whether computer-aided detection (CAD) as a second reader using perspective filet view [three-dimensional (3D) filet] improves the performance of inexperienced readers at computed tomography colonography (CTC) compared with unassisted 3D filet and unassisted two-dimensional (2D) CTC.

MATERIAL AND METHODS

Fifty symptomatic patients underwent CTC and same-day colonoscopy with segmental unblinding. Two inexperienced readers read the CTC studies on 3D filet and 2D several weeks apart. Four months later, readers re-read the cases only evaluating CAD marks using 3D filet. Suspicious CAD marks not previously described on 3D filet were recorded. Jackknife free-response receiver operating characteristic (JAFROC-1) analysis was used to compare the observers' performances in detecting lesions with 3D filet, 2D and 3D filet with CAD.

RESULTS

One hundred and three lesions > or =3mm were detected at colonoscopy with segmental unblinding. CAD alone had a sensitivity of 73% (75/103) at a mean false-positive rate per patient of 12.8 in supine and 11.4 in prone. For inexperienced readers sensitivities with 3D filet with CAD were 58% (60/103) and 48% (50/103) with an improvement of 14-16 percentage points (p<0.05) compared with 2D and of 10-11 percentage points (p<0.05) compared with 3D filet. For inexperienced readers, the false-positive rate was 25-41% and 71-200% higher with 3D filet with CAD compared with 3D filet and 2D, respectively. JAFROC-1 analysis showed no significant differences in per-lesion overall performance among reading modes (p=0.8).

CONCLUSION

CAD applied as a second reader using 3D filet increased both sensitivity and the number of false positives by inexperienced readers compared with 3D filet and 2D, thus not improving overall performance, i.e., the ability to distinguish between lesions and non-lesions.

Computed tomographic colonography: current perspectives and future directions

Computed tomographic (CT) colonography, also known as virtual colonoscopy or CT colography, is capable of detecting colon polyps and cancers. It is emerging rapidly and has gained great momentum over the past several years, to the point where it has been proposed to be a viable primary colon cancer screening option. Despite the current publicity, many issues concerning CT colonography remain. As of 2009, the following topics are of paramount importance: (1) accuracy, including both sensitivity and specificity, (2) bowel preparation, (3) safety, (4) extracolonic findings, (5) patient acceptability, (6) training and standardization, and (7) implementation of CT colonography. Although much about CT colonography has already been learned, more remains to be learned in the future.

CT colonography polyp matching: differences between experienced readers

The purpose of this study was to investigate if experienced readers differ when matching polyps shown by both CT colonography (CTC) and optical colonoscopy (OC) and to explore the reasons for discrepancy. Twenty-eight CTC cases with corresponding OC were presented to eight experienced CTC readers. Cases represented a broad spectrum of findings, not completely fulfilling typical matching criteria. In 21 cases there was a single polyp on CTC and OC; in seven there were multiple polyps. Agreement between readers for matching was analyzed. For the 21 single-polyp cases, the number of correct matches per reader varied from 13 to 19. Almost complete agreement between readers was observed in 15 cases (71%), but substantial discrepancy was found for the remaining six (29%) probably due to large perceived differences in polyp size between CT and OC. Readers were able to match between 27 (71%) and 35 (92%) of the 38 CTC detected polyps in the seven cases with multiple polyps. Experienced CTC readers agree to a considerable extent when matching polyps between CTC and subsequent OC, but non-negligible disagreement exists.

Computed tomographic colonography

PURPOSE OF REVIEW

Computed tomographic colonography is a new and noninvasive method to evaluate the colon. The goal of this review is to discuss the latest data and define outstanding issues related to computed tomographic colonography.

RECENT FINDINGS

Computed tomographic colonography is gaining momentum as a potential primary colon cancer screening method in the USA. Although not as accurate as colonoscopy, the accuracy of computed tomographic colonography for detection of large lesions appears to be in the 80-90% range. The field is rapidly evolving, not only in terms of technology but also in a variety of other practical areas.

SUMMARY

Current data suggest that computed tomographic colonography is a viable colon cancer screening modality in the USA. However, it is not ready for widespread implementation, largely because of lack of standards for training and reading and the fact that the number of skilled readers is limited.

Does a computer-aided detection algorithm in a second read paradigm enhance the performance of experienced computed tomography colonography readers in a population of increased risk?

We prospectively determined whether computer-aided detection (CAD) could improve the performance characteristics of computed tomography colonography (CTC) in a population of increased risk for colorectal cancer. Therefore, we included 170 consecutive patients that underwent both CTC and colonoscopy. All findings >or=6 mm were evaluated at colonoscopy by segmental unblinding. We determined per-patient sensitivity and specificity for polyps >or=6 mm and >or=10 mm without and with computer-aided detection (CAD). The McNemar test was used for comparison the results without and with CAD. Unblinded colonoscopy detected 50 patients with lesions >or=6 mm and 25 patients with lesions >or=10 mm. Sensitivity of CTC without CAD for these size categories was 80% (40/50, 95% CI: 69-81%) and 64% (16/25, 95% CI: 45-83%), respectively. CTC with CAD detected one additional patient with a lesion >or=6 mm and two with a lesion >or=10 mm, resulting in a sensitivity of 82% (41/50, 95% CI: 71-93%) (p = 0.50) and 72% (18/25, 95% CI: 54-90%) (p = 1.0), respectively. Specificity without CAD for polyps >or=6 mm and >or=10 mm was 84% (101/120, 95% CI: 78-91%) and 94% (136/145, 95% CI: 90-98%), respectively. With CAD, the specificity remained (nearly) unchanged: 83% (99/120, 95% CI: 76-89%) and 94% (136/145, 95% CI: 90-98%), respectively. Thus, although CTC with CAD detected a few more patients than CTC without CAD, it had no statistically significant positive influence on CTC performance.

Imaging and cancer: a review

Multiple biomedical imaging techniques are used in all phases of cancer management. Imaging forms an essential part of cancer clinical protocols and is able to furnish morphological, structural, metabolic and functional information. Integration with other diagnostic tools such as in vitro tissue and fluids analysis assists in clinical decision-making. Hybrid imaging techniques are able to supply complementary information for improved staging and therapy planning. Image guided and targeted minimally invasive therapy has the promise to improve outcome and reduce collateral effects. Early detection of cancer through screening based on imaging is probably the major contributor to a reduction in mortality for certain cancers. Targeted imaging of receptors, gene therapy expression and cancer stem cells are research activities that will translate into clinical use in the next decade. Technological developments will increase imaging speed to match that of physiological processes. Targeted imaging and therapeutic agents will be developed in tandem through close collaboration between academia and biotechnology, information technology and pharmaceutical industries.