Resource requirements for evaluating ambulatory health care

Three decades of research on computer applications in health care: medical informatics support at the Agency for Healthcare Research and Quality

The Agency for Healthcare Research and Quality and its predecessor organizations-collectively referred to here as AHRQ-have a productive history of funding research and development in the field of medical informatics, with grant investments since 1968 totaling $107 million. Many computerized interventions that are commonplace today, such as drug interaction alerts, had their genesis in early AHRQ initiatives. This review provides a historical perspective on AHRQ investment in medical informatics research. It shows that grants provided by AHRQ resulted in achievements that include advancing automation in the clinical laboratory and radiology, assisting in technology development (computer languages, software, and hardware), evaluating the effectiveness of computer-based medical information systems, facilitating the evolution of computer-aided decision making, promoting computer-initiated quality assurance programs, backing the formation and application of comprehensive data banks, enhancing the management of specific conditions such as HIV infection, and supporting health data coding and standards initiatives. Other federal agencies and private organizations have also supported research in medical informatics, some earlier and to a greater degree than AHRQ. The results and relative roles of these related efforts are beyond the scope of this review.

The Ambulatory Care Medical Audit Demonstration Project. Research design

The authors describe the design of and statistical analyses involved in the Ambulatory Care Medical Audit Demonstration Project, which tested feasibility, cost, and effectiveness of cycles that met quality assurance requirements in eight pediatric and eight general medicine group practices at four teaching hospitals and six health centers. The authors used a concurrent crossover design using randomized cycles of quality assurance so that a practice was a control site for one guideline and an experimental site for another. For 12 months before and 18 months during and after quality assurance experimental interventions, the authors measured practitioner conformance to review criteria for patient-care guidelines believed to improve outcomes, including four internal medicine patient-care guidelines (ie, follow-up of low hematocrit, cancer screening for women, follow-up of high serum glucose, and monitoring of patients treated with digoxin) and four pediatric patient-care guidelines (ie, follow-up of positive urine cultures, screening for disease and immunizing infants, management of acute gastroenteritis, and management of acute ear infection). The authors distinguished review criteria whose performance depended on personal efforts of practitioners from those that concerned performance dependent on the practice's system for reporting test results and calling patients to return for care.

What makes quality assurance effective? Results from a randomized, controlled trial in 16 primary care group practices

OBJECTIVES

The authors estimate separately contributions of each component intervention to overall effectiveness of quality assurance cycles used to improve practice performance.

METHODS

In a randomized, controlled trial, experimental cycles of quality assurance were conducted for eight patient-care guidelines, with two experimental cycles assigned to each of 16 group practices. For three separate interventions per cycle, practitioners: (1) were notified of the name of the experimental guideline, (2) discussed criteria of conformance to the guideline, and (3) received feedback on performance. Actions taken in response to interventions were documented. Using medical records data for a baseline year and for 3 months after each intervention and an additional 9 months, the authors scored each practice for conformance to two experimental guidelines and to control guidelines.

RESULTS

For all patient-care guidelines combined, and for four of five guidelines showing improvement, knowledge of guidelines and review criteria alone produced no change. After feedback, performance improved and improvement persisted for at least 9 months. The number of corrective actions implemented contributed significantly to effectiveness of quality assurance.

CONCLUSIONS

Feedback to providers of data on their performance is a more powerful stimulus for quality improvement than is knowledge of guidelines or discussion of review criteria.