An intranet-based automated system for the surveillance of nosocomial infections: prospective validation compared with physicians' self-reports

OBJECTIVE

To examine the reliability of the data produced by an automated system for the surveillance of nosocomial infections.

SETTING

A 906-bed, tertiary-care teaching hospital.

DESIGN

Three surveillance techniques were concurrently performed in seven high-risk units during an 11-week period: automated surveillance (AS) based on the prospective processing of computerized medical records; laboratory-based ward surveillance (LBWS) based on the retrospective verification by ward clinicians of weekly reports of positive bacteriologic results; and a reference standard (RS) consisting of the infection control team reviewing case records of patients with positive bacteriology results. Bacteremia, urinary tract infections, and catheter-related infections were recorded for all inpatients. The performances (sensitivity, specificity, and time consumption) of AS and LBWS were compared with those of RS.

RESULTS

Of 548 positive bacteriology samples included during the study period, 229 (42%) were classified as nosocomial infections. The overall sensitivity was 91% and 59% for AS and LBWS, respectively. The two methods had the same overall specificity value (91%). Kappa measures of agreement were 0.81 and 0.54 for AS and LBWS, respectively. AS required less time to collect data (54 seconds per week per unit) compared with LBWS (7 minutes and 43 seconds per week per unit) and RS (37 minutes and 15 seconds per week per unit).

CONCLUSION

Our results confirm that the retrospective review of charts and laboratory data by physicians lacks sensitivity for the surveillance of nosocomial infections. The intranet-based automated method developed for this purpose was more accurate and less time-consuming than the weekly, retrospective LBWS method.

Diagnostic tests: distinguishing good tests from bad and even ugly ones

This article focuses on the selection and interpretation of diagnostic tests, emphasizing the importance of understanding how their mathematical parameters affect the information they provide in various settings. The utility and limitations of sensitivity, specificity, predictive value, and receiver operating characteristic (ROC) curves are discussed using catheter-related bloodstream infections as an example. ROC curves have been used for selecting optimal cutoff values for a positive result and for selecting among several alternative diagnostic tests. For example, 16 different tests have been proposed for diagnosis of catheter-related bloodstream infection; ROC analysis provides an effective way to determine which test offers the best overall performance.

Evaluation of two retrospective active surveillance methods for the detection of nosocomial infection in surgical patients

OBJECTIVE

To compare the sensitivity and specificity of two retrospective active surveillance methods based on review of the medical record and review of the discharge form in identfying nosocomial infection, taking the prospective surveillance method as the reference standard.

DESIGN

Blind comparison of three active nosocomial infection surveillance methods.

SETTING

Department of General Surgery of a tertiary-care hospital with a referral population of 266,000 people.

METHODS

All operated patients admitted to the Department of Surgery for more than 24 hours and discharged from January 1, 1994, to December 31, 1994, were included. Prospective surveillance consisted of daily review of the patient's record during hospitalization. Retrospective surveillance consisted of review of the medical record and the discharge form. Sensitivity and specificity of both retrospective methods were calculated.

RESULTS

Of the 1,514 patients included in the study, 1,476 (97.5%) were reviewed by means of the retrospective surveillance system. A total of 20, 8, and 4 hours per week was needed for the active prospective system, review of the medical record, and review of the hospital discharge form, respectively. The documented cumulative incidence of nosocomial infection was 21.8% for the prospective system, 19.6% for review of the medical record, and 12.6% for review of the discharge form. The overall sensitivity of review of the medical record was 88% and of the discharge form 56%, with a specificity of 99%. For review of the medical record, the highest sensitivity was 93%, for urinary tract infections; for review of the discharge form, the highest was 57%, for surgical-wound infection.

CONCLUSIONS

The retrospective method of review of the medical record was the most efficient active surveillance strategy in detecting nosocomial infection in surgical patients.

The merits of confidence intervals relative to hypothesis testing

A variety of statistical tests of a null hypothesis commonly are used in biomedical studies. While these tests are the mainstay for justifying inferences drawn from data, they have important limitations. This report discusses the relative merits of two different approaches to data analysis and display, and recommends the use of confidence intervals rather than classic hypothesis testing.

Formulae for a confidence interval surrounding the point estimate of an average value take the form: d= ±zσ/√n, where “d” represents the average difference between central and extreme values, “z” is derived from the density function of a known distribution, and “a/-∨n” represents the magnitude of sampling variability. Transposition of terms yields the familiar formula for hypothesis testing of normally distributed data (without applying the finite population correction factor): z = d/(σ/√n).