Soft-copy mammographic readings with different computer-assisted detection cuing environments: preliminary findings

PURPOSE

To assess the performance of radiologists in the detection of masses and microcalcification clusters on digitized mammograms by using different computer-assisted detection (CAD) cuing environments.

MATERIALS AND METHODS

Two hundred nine digitized mammograms depicting 57 verified masses and 38 microcalcification clusters in 85 positive and 35 negative cases were interpreted independently by seven radiologists using five display modes. Except for the first mode, for which no CAD results were provided, suspicious regions identified with a CAD scheme were cued in all the other modes by using a combination of two cuing sensitivities (90% and 50%) and two false-positive rates (0.5 and 2.0 per image). A receiver operating characteristic study was performed by using soft-copy images.

RESULTS

CAD cuing at 90% sensitivity and a rate of 0.5 false-positive region per image improved observer performance levels significantly (P < .01). As accuracy of CAD cuing decreased so did observer performances (P < .01). Cuing specificity affected mass detection more significantly, while cuing sensitivity affected detection of microcalcification clusters more significantly (P < .01). Reduction of cuing sensitivity and specificity significantly increased false-negative rates in noncued areas (P < .05). Trends were consistent for all observers.

CONCLUSION

CAD systems have the potential to significantly improve diagnostic performance in mammography. However, poorly performing schemes could adversely affect observer performance in both cued and noncued areas.

Statistical test to assess rank-order imaging studies

RATIONALE AND OBJECTIVES

Rank-order experiments often provide a reasonable method of determining whether a large-scale receiver operating characteristic study can be justified. The authors' purpose was to formalize a proposed method for analyzing rank-order imaging experiments and provide methods that can be used in determining sample sizes for both cases and raters.

MATERIALS AND METHODS

Simulations were conducted to determine the adequacy of the normal approximation of a statistic used to test the null hypothesis of random ordering. For a multireader experiment, formulas are presented and guidelines are provided to enable investigators to determine the number of required readers (raters) and cases for a specific study.

RESULTS

When there are at least five ordered images per case, 10 cases are sufficient to test a random rank order. When there are only three or four images for a case, 20 cases are required. The authors constructed tables of statistical power for selected numbers of ordered images, numbers of cases, and degrees of trend, and they also provide an approximation for use in situations that are not tabled.

CONCLUSION

The statistical methods for analyzing rank-order experiments and estimating sample sizes for study planning are relatively simple to implement. The derived formulas for sample size estimation, when applied to typical imaging experiments, indicate that modest numbers of cases and readers are required for rank-order studies.

Multipoint rank-order study methodology: observer issues

RATIONALE AND OBJECTIVES

We performed a multipoint rank-order experiment to evaluate variability in observers' sensitivity to small differences in image presentation and to assess observers' performance as a function of the type and number of tasks included.

METHODS

Five experienced observers were presented with four sets of chest images that had been compressed at five different levels. Each set contained six images ranging from noncompressed to approximately 60:1-compressed images. Observers were asked to review all images of each case side by side and rank-order the "quality" of each to enable determination of the presence or absence of interstitial disease and/or pneumothoraces.

RESULTS

Observers varied significantly in their ability to detect very small differences among the images (P < 0.001). Those who performed well did so regardless of whether they ranked a specific abnormality in a multidisease or a single-disease setting.

CONCLUSIONS

Selected observers can reliably detect very small differences among similar images. These readers could be used to confirm or rule out the need for objective observer-performance-type studies.

Empiric assessment of parameters that affect the design of multireader receiver operating characteristic studies

RATIONALE AND OBJECTIVES

The authors attempted to assess experimentally the magnitude of reader variability and the correlations and interactions among cases, readers, and modalities during observer performance studies and their possible effects on study design and sample size.

MATERIALS AND METHODS

Published data from 32 selected receiver operating characteristic (ROC) studies were reviewed to compare the magnitude of the variance component from readers with the variance component from modality. Estimates of correlation and interactions among cases, readers, and modalities were also computed directly from ROC data ascertained during two large studies performed in our laboratory. Each of these two studies included 529 cases and six readers, but one study used eight modalities and the other nine.

RESULTS

Published results indicate that reader variability is task dependent and larger (P < .05) than modality variability in detection of interstitial disease. Measured correlations between modalities for the same reader were task dependent and ranged from 0.35 to 0.59. Modality-by-reader and modality-by-case interactions often are not important factors. The random error term was greater than the modality-by-reader interaction in 11 of 20 comparisons and greater than the modality-by-case interaction in eight of 20 comparisons.

CONCLUSION

Use of the same cases interpreted with different modes is justifiable in many situations because of the high variability from readers. This comprehensive review of existing ROC studies resulted in parameter assessments that can be used to better estimate sample-size requirements in multireader ROC studies.

Greater tuberosity changes as revealed by radiography: lack of clinical usefulness in patients with rotator cuff disease

OBJECTIVE

Studies linking greater tuberosity findings on radiographs with rotator cuff disease have largely been uncontrolled and biased toward more severe disease. We correlated greater tuberosity changes seen on radiography with rotator cuff disease seen on MR images in a broadly symptomatic patient population.

MATERIALS AND METHODS

Both radiography and MR imaging were performed in 108 shoulders. Unaware of the MR imaging findings, three radiologists independently reviewed the radiographs for cortical thickening, subcortical sclerosis, and cystlike lesions in the humeral greater tuberosity. Interobserver agreement was analyzed using kappa statistics. We correlated the radiographic findings with MR imaging evidence of rotator cuff tears and tendonopathy. The positive predictive value of each finding for rotator cuff disease was also calculated.

RESULTS

Interobserver agreement for the three radiographic findings was poor to fair: Kappa values ranged from .06 to .41. Cortical thickening and subcortical sclerosis were not seen more frequently in shoulders with rotator cuff disease than in normal shoulders. Cystlike lesions were more prevalent in shoulders with rotator cuff disease, but the association reached statistical significance (p < .05) for one observer only. Positive predictive values for each finding were low (14-48% for predicting full-thickness rotator cuff tears).

CONCLUSION

Cortical thickening of the greater tuberosity and subcortical sclerosis are not associated with rotator cuff disease. For some observers, identifying cystlike lesions is associated with rotator cuff disease, but the clinical usefulness of the observation is limited by high interobserver variability and poor positive predictive value.

MR imaging of knees having isolated and combined ligament injuries

OBJECTIVE

Although clinical evaluation and MR imaging both accurately reveal injuries in knees with isolated ligament tears, physical examination becomes progressively less reliable when multiple lesions exist. We investigated the accuracy of MR imaging of knees having varying degrees and numbers of ligament injuries.

SUBJECTS AND METHODS

We prospectively interpreted the MR images of 340 consecutive injured knees and compared these interpretations with the results of subsequent arthroscopy or open surgery, which served as the gold standard. Our interpretations of MR images focused on five soft-tissue supporting structures (the two cruciate ligaments, the two collateral ligaments, and the patellar tendon) and the two menisci. Patients were divided into three groups: no ligament injuries, single ligament injuries, and multiple ligament injuries.

RESULTS

Using MR imaging, we found overall sensitivity and specificity for diagnosing ligament tears to be 94% and 99%, respectively, when no or one ligament was torn and 88% and 84%, respectively, when two or more supporting structures were torn. The difference in specificity was statistically significant (p < .0001). Sensitivity for diagnosing meniscal tears decreased as the number of injured structures increased, but the relationship achieved statistical significance (p = .001) only for the medial meniscus. For all categories of injury, MR imaging was more accurate than clinical evaluation, statistics for which were taken from the orthopedic literature.

CONCLUSION

In knees with multiple ligament injuries, the diagnostic specificity of MR imaging for ligament tears decreases, as does the sensitivity for medial meniscal tears.

Anterolateral soft-tissue impingement in the ankle: diagnosis using MR imaging

OBJECTIVE

This study was performed to elucidate the MR imaging findings and pitfalls for the diagnosis of anterolateral soft-tissue impingement in the ankle, a cause of chronic ankle pain that can be relieved by arthroscopic resection.

MATERIALS AND METHODS

We retrospectively reviewed MR imaging examinations of 18 patients with arthroscopically confirmed anterolateral ankle impingement. The MR images of 18 additional subjects with symptoms that could mimic anterolateral impingement, but who had a surgically confirmed alternate diagnosis (instability, peroneal tendon injury, osteochondral defect, normal arthroscopy) and no evidence of impingement at arthroscopy, served as controls.

RESULTS

On the MR imaging studies, nine patients had an ankle effusion, eight of whom showed an abnormal soft-tissue structure in the anterolateral gutter, 2-15 mm in maximal diameter. No soft-tissue mass was seen in the patients without joint fluid. Four control subjects with instability had a similar soft-tissue structure in the anterolateral gutter, but in the control subjects the finding represented a portion of the torn anterior talofibular ligament.

CONCLUSION

Anterolateral soft-tissue impingement of the ankle can be suggested by MR imaging when fluid in the lateral gutter outlines an abnormal soft-tissue structure separate from the anterior talofibular ligament.

Are MR imaging signs of meniscocapsular separation valid?

PURPOSE

To define the positive predictive value (PPV) for the magnetic resonance (MR) imaging diagnosis of meniscocapsular separation in the knee.

MATERIALS AND METHODS

The MR reports of 52 patients aged 16-75 years who had a prospective MR diagnosis of meniscocapsular injury were correlated with arthroscopic results. MR images obtained in 50 of these patients were retrospectively reviewed for specific signs of meniscocapsular separation, which included meniscal displacement, peripheral meniscal corner tears, increased perimeniscal signal intensity, fluid deep to the medial collateral ligament, and abnormal lateral fascicles. MR findings were correlated with arthroscopic findings.

RESULTS

For the prospective MR interpretations, the PPV for meniscocapsular separation was 9% medially and 13% laterally. Meniscal displacement (measured from the meniscal edge to the tibia) was as great as 10 mm medially or 13 mm laterally without meniscocapsular tears at arthroscopy. Meniscal displacement did not correlate with effusion. Meniscal corner tears had a PPV of 0% medially and 50% laterally. Fluid at the meniscocapsular border and fluid deep to the medial collateral ligament were poor predictors of meniscocapsular abnormalities. Abnormal-appearing meniscal fascicles had a PPV of 8% for lateral meniscocapsular separation.

CONCLUSION

The PPV for the MR diagnosis of meniscocapsular separation is low; the reported MR signs correlate poorly with arthroscopic findings.

Do pulmonary radiographic findings at presentation predict mortality in patients with community-acquired pneumonia?

BACKGROUND

Previous studies have reported conflicting results on whether pulmonary radiographic findings predict mortality for patients with community-acquired pneumonia (CAP).

OBJECTIVE

To determine whether pulmonary radiographic findings at presentation are independently associated with 30-day mortality in patients with suspected CAP.

METHODS

This study was conducted as part of the Pneumonia Patient Outcomes Research Team multicenter, prospective cohort study of ambulatory and hospitalized patients with clinical and radiographic evidence of CAP. For each patient with CAP, a structured data form was completed by a panel of radiologists to evaluate the radiographic pattern of infiltrate, number of lobes involved, presence of pleural effusion, and 8 other radiographic characteristics. Cox proportional hazards models were used to evaluate the independent association between radiographic findings and 30-day mortality, while controlling for patient demographic, clinical, and laboratory characteristics with a known association with this outcome.

RESULTS

Of 2287 patients enrolled in the Pneumonia Patient Outcomes Research Team cohort study, 1906 patients (83.3%) had a pulmonary radiographic infiltrate confirmed by the radiology panel. Overall, 30-day mortality in this cohort was 4.9%. Univariate regression analyses demonstrated the following radiographic characteristics to be significantly associated with 30-day mortality: (1) bilateral pleural effusions (risk ratio [RR], 7.0; 95% confidence interval [CI], 3.9-12.6); (2) a pleural effusion of moderate or greater size (RR, 3.4; 95% CI, 1.4-8.4); (3) 2 or more lobes involved with infiltrate (RR, 2.5; 95% CI, 1.6-3.8); (4) bilateral infiltrate (RR, 2.8; 95% CI, 1.9-4.2); (5) bronchopneumonia (RR, 1.6; 95% CI, 1.0-2.7); and (6) air bronchograms (RR, 0.5; 95% CI, 0.2-0.9). Multivariate analysis of radiographic features and other clinical characteristics showed the presence of bilateral pleural effusions (RR, 2.8; 95% CI, 1.4-5.8) was independently associated with mortality.

CONCLUSIONS

In patients with CAP, the presence of bilateral pleural effusions is an independent predictor of short-term mortality. This finding, which is available at presentation, can help guide physicians' assessment of prognosis in CAP.

Interobserver reliability of the chest radiograph in community-acquired pneumonia. PORT Investigators

OBJECTIVE

To evaluate the interobserver reliability of pulmonary radiographic findings in patients with community-acquired pneumonia (CAP).

DESIGN

A prospective, multicenter study.

SETTING

Physician offices, medical walk-in clinics, emergency departments, and inpatient wards affiliated with three university hospitals, one community hospital, and one staff model health maintenance organization in three geographic areas.

METHODS

Copies of the initial chest radiograph of patients suspected of having CAP were independently read by two staff radiologists at the coordinating university hospital. Interobserver reliability for the interpretation for radiographic findings was assessed by calculation of agreement rates and the kappa statistic.

PARTICIPANTS

Adults (age > or = 18 years) with symptoms or signs of CAP and a pulmonary radiographic infiltrate documented by a local study site radiologist.

RESULTS

Among the 282 patients whose initial pulmonary radiographs were evaluated, there was agreement between the two staff radiologists on the presence of infiltrate in 79.4% and on the absence of an infiltrate in 6.0% (kappa = 0.37; 95% confidence interval [CI] = 0.22 to 0.52). For the 224 patients with an infiltrate identified by both radiologists, there was further agreement that the infiltrate was unilobar in 41.5% and multilobar in 33.9% (kappa = 0.51; 95% CI = 0.28 to 0.62), pleural effusion was present in 10.7% and absent in 73.2% (kappa = 0.46; 95% CI = 0.33 to 0.50), and the infiltrate was alveolar in 96.3% of patients and interstitial in no patients (kappa = -0.01; 95% CI = -0.03 to 0.00). Among the 210 patients with an alveolar infiltrate, both radiologists classified the infiltrate as lobar in 74.6% and bronchopneumonia in 2.4% (kappa = 0.09; 95% CI = -0.04 to 0.22), and agreed on the presence of air bronchograms in 7.6% and their absence in 52.9% (kappa = 0.01; 95% CI = -0.13 to 0.15).

CONCLUSION

In patients with CAP, two university radiologists identified the presence of infiltrate, multilobar disease, and pleural effusion with fair to good interobserver reliability. However, interobserver reliability for the pattern of infiltrate and the presence of air bronchograms was poor.

MR imaging of the foot: utility of complex oblique imaging planes

Successful MR imaging of the foot presents special challenges to the radiologist. Accurate and confident diagnosis presupposes the ability to produce high-resolution images of obliquely oriented, relatively small structures. Orienting the foot within an appropriate local coil to bring such structures into an orthogonal imaging plane, or even into a conventional oblique plane, may be impossible or intolerably uncomfortable for the patient. The frequent result is motion artifacts, which are accentuated when using a small field of view. However, when patients are comfortably positioned, the anatomy of interest often lies in a plane that is not orthogonal to any of the conventional imaging planes. Fortunately, commercially available MR imaging equipment can produce images in complex oblique planes with relative ease. In this pictorial essay, we discuss the technical considerations for expedient diagnostic MR imaging of the complex anatomy of the foot and illustrate our experiences with this technique.

Subjective quality assessment of computed radiography hand images

To evaluate the sensitivity of a non-receiver-operating characteristic (ROC) study in assessing small differences of perceived image quality of hand images acquired by computed radiography (CR) and conventional screen-film systems, hand images were acquired on 12 patients with both conventional screen-film and CR. Each CR image was then processed with three different edge-enhancement algorithms. One conventional film and four CR images were then viewed side by side by five radiologists. Observers rated perceived image quality of each radiograph using a 10-category discrete scale. The study was repeated after 6 weeks using a different block randomization scheme. Despite the small sample size, significant differences (P < .05) in assigned image quality were detected among CR images acquired at low, medium, and high resolutions. Image processing routines did not fully compensate for differences in quality between conventional film and CR-acquired images. The quality rating of the reference conventional image was found to be dependent on the quality of images with which it was compared. Small, highly sensitive study designs can be used to identify radiologists' perceived differences in image quality. "Reference" or "gold standard" quality are important in such studies. Edge-enhancement schemes cannot fully compensate for perceived image quality degradations because of reduced image resolution.

The pronator quadratus: a distinct forearm space?

The pronator quadratus sign is associated with fractures of the distal radius and ulna and is believed to be due to accumulation of fluid within the pronator quadratus muscle. This anatomic study based on dye injection and x-ray film examination shows that the pronator quadratus occupies a distinct forearm space without intramuscular communication.

Selection of processing algorithms for digital image compression: a rank-order study

RATIONALE AND OBJECTIVES

We investigated non-receiver operating characteristic (non-ROC) methods for the selection of processing algorithms for digital image compression.

METHODS

We performed a multipoint, rank-order study with 20 posteroanterior chest images, each processed using four different algorithms. Seven radiologists reviewed these alongside the digitized noncompressed image. Observers were forced to rank order the similarity and/or difference of the processed images to the nonprocessed image in each case.

RESULTS

A two-way analysis of variance of the rankings was statistically significant (p = .025), indicating that one processing scheme yielded images that were clearly perceived as the most similar to the nonprocessed images. The selected processing scheme was not the one that yielded the lowest quantitative difference from the nonprocessed images as measured by root mean square error.

CONCLUSION

Non-ROC study designs that are highly sensitive to small differences among similar images can be used to select processing algorithms.

A cadaveric and radiologic assessment of catheter placement for the measurement of forearm compartment pressures

The diagnosis of compartment syndrome is essentially a clinical one. In recent years, much interest and research has focused on the development of intracompartmental pressure monitoring devices to help in the diagnosis of this condition. Proper placement of the catheter is essential for accurate monitoring, because an incorrect result is potentially more dangerous than no reading at all. In the forearm, measurement in the flexor digitorum profundus is thought to be a more sensitive way of assessing rising pressure. One of the risks of catheter placement is inadvertent damage to a neurovascular structure, particularly when monitoring pressures in deeper muscle bellies. Standard anatomic texts and outpatient magnetic resonance scans of the forearm were studied to define a safe pathway for the introduction of a catheter into the volar forearm. An approach from the midline to the ulna, between the tendons of the flexor carpi radialis and palmaris longus, seemed the safest. Ten cadaveric forearms had 100 cc of gastrograffin injected into the deep forearm through a dorsal approach. The median and ulnar nerves were cannulated with a fine wire from the elbow to the wrist. Two cannulae were passed, using the suggested approach, at 2 locations between the wrist and the mid forearm. All limbs then were scanned with computed tomography, and the images were reviewed. In all forearms, the cannulae passed between the median and ulnar nerves into the belly of the flexor digitorum profundus and posed no risk to the neurovascular structures.

Subjective and objective assessment of image quality--a comparison

Forced-choice just noticeable difference (JND) studies are extremely sensitive to image quality variations that are below the threshold at which the differences are apparent to or definable by the observer. Paired comparisons of 4K and 2K laser-printed posteroanterior chest images consistently demonstrated that although images are viewed as comparable by radiologists, when forced to choose the better ("sharper") image, they actually select the higher-resolution images in 83% of the paired observations. We conclude that small differences in image quality may be detectable even in image sets which are considered to be comparable by subjective assessments.

Digital radiography and conventional imaging of the chest: a comparison of observer performance

OBJECTIVE

The purpose of this study was to compare observer performance in interpreting high-quality, digitally acquired computed radiographs of the chest displayed on either laser-printed radiographs or a workstation with observer performance in interpreting conventional chest radiographs.

SUBJECTS AND METHODS

We performed a receiver-operating-characteristic study in which the presence or absence of five abnormalities was determined by nine experienced radiologists on 310 posteroanterior radiographs of the chest displayed in three forms: conventional radiographs, laser-printed films of digital radiographs, and digital radiographs on a high-resolution workstation.

RESULTS

The results of our study suggest that observer performance with laser-printed films of digital radiographs obtained with high-resolution (4K x 5K), high-contrast sensitivity (12 bits) and appropriate exposure is comparable to observer performance with conventional radiographs. Observer performance with digital radiographs displayed on the workstation was found to be significantly lower for abnormalities that contained high-frequency and low-contrast information (e.g., interstitial disease and pneumothorax).

CONCLUSION

Computed radiography technology can produce image quality that is adequate for interpreting posteroanterior radiographs of the chest. Observer performance is not as good when radiographs displayed on workstations are used to diagnose specific abnormalities.

Avoiding neurologic and vascular injuries with screw fixation of the tibial component in total knee arthroplasty

This investigation determined the position of neurologic and vascular structures at risk during screw fixation of an uncemented tibial component in total knee arthroplasty relative to the cut surface of the tibial plateau. Sixteen anatomic specimens of the lower extremities were studied by systematically placing screws about the tibial plateau followed by computed tomographic scanning with three-dimensional reconstruction and anatomic dissection. The structures at risk during screw preparation and placement were the tibial nerve, popliteal artery, popliteal vein, anterior tibial artery, and common peroneal nerve with its superficial and deep branches. The application of a clock system to the cut surface of the tibial plateau with the six o'clock position defined as directly anterior and the three o'clock position defined as lateral identified the entire region between the 11 o'clock and three o'clock positions as a location where potential injury to vital structures can occur during screw placement. The risk of injury to these structures may be minimized by directing screws radially inward to avoid violation of the tibial cortex. Any penetration of the cortex between the 11 o'clock and three o'clock positions may have potentially devastating consequences because of the close proximity of neurologic and vascular structures.

Practical issues of experimental ROC analysis. Selection of controls

Receiver operating characteristics (ROC) analysis has been used in many medical imaging applications during the past decade. With the recent expansion of the ROC methodology to multi-disease studies, several reports have begun to address both the theoretical and experimental design issues associated with such studies. While the appropriate selection, classification, and verification of actually positive cases is carefully addressed in the literature, similar considerations are rarely given to the selection of actually negative controls for these studies. In this paper, theoretical considerations and experimental data are provided to demonstrate the significance of this very issue.

Carcinoma mimicked by the sternal insertion of the pectoral muscle

A prospective study of 1843 patients was performed to identify the frequency of imaging the medial portion of the greater pectoral muscle on the craniocaudal projection during routine mammography. This artifact was seen in 14 patients. None of these patients had a palpable abnormality; additional craniocaudal views revealed continuity of the density with the pectoral muscle. Four patients were reexamined in 6 months; no change was seen in the size of the density. The density is thought to represent the medial portion of the pectoral muscle, which is included because of vigorous retraction of the breast and slight external rotation during positioning for the craniocaudal view. Familiarity with the appearance of the medial portion of the greater pectoral muscle on craniocaudal mammograms will help prevent inaccurate interpretations and unnecessary biopsies.

The mammographic features of the postlumpectomy, postirradiation breast

Lumpectomy with radiation therapy is becoming widely accepted as an alternative treatment for breast carcinoma in place of radical mastectomy. This article illustrates the spectrum of mammographic changes seen at various intervals after lumpectomy with irradiation to help the mammographer distinguish postoperative scar from a recurrent carcinoma.

Radiation exposure of personnel during digital subtraction angiography

Radiation exposure to the lens of physicians performing intravenous and hand-injected intraarterial digital subtraction angiography (DSA) were monitored with and without a combined face and body shield. Shielding provided nearly a three-fold reduction in dose for both intravenous and intraarterial exams, with the highest doses recorded for intraarterial exams due to longer fluoroscopy and exposure during imaging. When compared with the NCRP guidelines of maximum exposure to the lens, an angiographer could theoretically perform up to two intraarterial and 14 intravenous studies per day with protection or one intraarterial and two intravenous studies per day without protection. The exposure values in this study reflect our equipment and personal technique in carotid DSA and may not apply to other departments, but should encourage other angiographers to monitor exposure in their own angiography suites.