Quality improvement among primary care practitioners: an overall appraisal of results of the Ambulatory Care Medical Audit Demonstration Project

Leading improvement

PURPOSE

To provide research-informed guidance to leaders of quality and safety improvement and evidence-based materials for education programmes for leaders

DESIGN/METHODOLOGY/APPROACH

Search of databases and hard copy literature since 1985 into managers' and leaders' role in quality and safety improvement. Classification into "eA": Empirical research strong evidence, "eB": Empirical research, weak evidence and, "eC": Conceptual discussion, not based on systematic empirical research. Summary, and synthesis of the best available evidence for a guidance checklist for leaders.

FINDINGS

Although most literature emphasises the importance of committed leadership for successful quality and safety improvement, research evidence supporting this is scarce and often scientifically limited. The research shows evidence of the limitations and scope of leader actions for improving health care provision, the need to engage clinicians in this work and ways to do so, as well as the leadership role played by others apart from senior leaders. The ability of managers and other leaders skilfully to tailor Q&SI to the situation may be important but descriptions of how leaders do this and evidence supporting this proposition are lacking.

RESEARCH LIMITATIONS/IMPLICATIONS

More research is needed about whether or how the leader role is different according to the stage of quality and safety development of the organisation, the type of organisation, the type of context, the level and type of leader and the type of improvement and improvement method.

PRACTICAL IMPLICATIONS

Implications for leaders' actions are provided in a "best evidence guidance" checklist. This provides more tangible and research-informed guidance than the inspirational literature or studies from single organisations.

ORIGINALITY/VALUE

This paper provides the first overview and synthesis of a wide range of studies which can be used as a basis for future research and materials for educational programmes. It provides the first detailed guidance for leaders about specific actions which research suggests they need to take to improve quality and safety.

Inconsistent report cards: assessing the comparability of various measures of the quality of ambulatory care

BACKGROUND

Report cards based on various performance measures have become increasingly common for rating hospitals and health care plans. However, little has been done to create report cards at the ambulatory clinic level, nor has there been much comparison of the potential components of report cards.

OBJECTIVES

To create a report card for ambulatory clinics based on different data collection methods and to assess the correlations of clinic scores across various domains of quality.

RESEARCH DESIGN

Cross-sectional chart review (n = 3614), patient (n = 2180), and physician surveys (n = 169).

SUBJECTS

Sample of outpatients ages 20 to 75 and their primary care providers in 11 ambulatory clinic sites in the Boston-area from May 1996 to June 1997.

MEASURES

Performance on various quality indicators for each site.

RESULTS

Report card scores for five quality domains (performance on HEDIS-like measures, clinic function, patient satisfaction, diabetes guideline compliance, asthma guideline compliance) were created for each site. None of the five domain scores were significantly correlated with any of the other domains. In addition, there was substantial intraclinic variation in domain scores when compared with the corresponding mean domain score across all clinics. Additional clinic domain scores were created by limiting measures to those found on chart review or survey alone. The chart review and survey domain scores for each clinic were also not significantly correlated.

CONCLUSIONS

Report cards that emphasize only one domain of quality or use limited data collection methods may provide incomplete or inconsistent information to health care consumers about the overall quality of an outpatient clinic.

Quality improvement interventions: what works?

Quality improvement efforts in healthcare focus on the use of interventions and tools to change identified processes. What interventions and tools are effective? A survey of the literature yields information on interventions and tools that have been used to successfully create quality improvement. Effective methods for changing clinical practice include face-to-face education outreach visits, involvement of local opinion leaders, reminder systems, repeated feedback from the senior medical staff, patient-mediated interventions, and a combination of interventions deployed simultaneously. Participation in an organized continuous quality improvement process is beneficial in conjunction with additional interventions.

Methodologic challenges in developing and implementing measures of quality for child health care

OBJECTIVE

To review the major building blocks in measurement of quality for child health care, with recommendations for future research.

METHODS

We describe a framework of building blocks for quality measurement and discuss how an investigator's choices for each component are constrained by the special features of child health care.

RESULTS

Methodologic challenges for children's health care include developmental change and dependency on others, fragmentary care and inadequate health care data, unusual care settings, potential for long-term consequences, proxy reporting of outcomes and patient experience, small sample sizes, and lack of evidence that links processes and outcomes of care and of methods for risk adjustment. We cite examples of child-specific measures of quality that illustrate solutions to these challenges.

CONCLUSIONS

Children are different from adults, and measures of health care quality for children must differ from those for adults. We suggest future research on measures of quality directed toward overcoming the methodologic problems specific to child health care.

Lessons from experienced guideline implementers: attend to many factors and use multiple strategies

BACKGROUND

Studies of clinical guideline implementation have focused almost entirely on changing individual clinician behavior with single intervention strategies and without much attention to the situational context. The goal of this project was to learn from clinic leaders, seasoned in the guideline implementation process, what contextual variables they viewed as important and whether implementation success could be expected if only a single implementation strategy was used.

METHODS

In 1998, 12 people with extensive experience in leading clinical guideline implementation were identified who were thought to have particularly keen insight into the process. They were interviewed to generate variables they considered important, as well as strategies they considered effective when used appropriately. A modified nominal group/Delphi process was then used for rating these variables and strategies, and the reactions of international experts were obtained to add perspective to this information.

RESULTS

Eighty-seven variables and 25 strategies were identified, clustering in 6 categories (ranked in order of importance by the panel): organizational capabilities for change, infrastructure for implementation, implementation strategies, medical group characteristics, guideline characteristics, and external environment. All six categories were considered to be important, key, or essential by the experienced implementers, although variables within a medical group that directly affect its ability to undertake planned change were rated as much more important than either guideline characteristics or the external environment.

DISCUSSION

Although the opinions of those experienced in the process of guideline implementation are primarily of value for generating hypotheses, panel members believe that implementation efforts focusing on the individual physician with a single strategy are unlikely to be successful. Rather, implementation efforts must use multiple strategies that take account of multiple characteristics of the guideline, practice organization, and external environment.

External audit on the clinical practice and medical decision-making at the departments of radiotherapy in Budapest and Vienna

PURPOSE

To present an example of how to study and analyze the clinical practice and the quality of medical decision-making under daily routine working conditions in a radiotherapy department, with the aims of detecting deficiencies and improving the quality of patient care.

METHODS

Two departments, each with a divisional organization structure and an established internal audit system, the University Clinic of Radiotherapy and Radiobiology in Vienna (Austria), and the Department of Radiotherapy at the National Institute of Oncology in Budapest (Hungary), conducted common external audits. The descriptive parameters of the external audit provided information on the auditing (auditor and serial number of the audit), the cohorts (diagnosis, referring institution, serial number and intention of radiotherapy) and the staff responsible for the treatment (division and physician). During the ongoing external audits, the qualifying parameters were (1) the sound foundation of the indication of radiotherapy, (2) conformity to the institution protocol (3), the adequacy of the choice of radiation equipment, (4) the appropriateness of the treatment plan, and the correspondence of the latter with (5) the simulation and (6) verification films. Various degrees of deviation from the treatment principles were defined and scored on the basis of the concept of Horiot et al. (Horiot JC, Schueren van der E. Johansson KA, Bernier J, Bartelink H. The program of quality assurance of the EORTC radiotherapy group. A historical overview. Radiother. Oncol. 1993,29:81-84), with some modifications. The action was regarded as adequate (score 1) in the event of no deviation or only a small deviation with presumably no alteration of the desired end-result of the treatment. A deviation adversely influencing the result of the therapy was considered a major deviation (score 3). Cases involving a minor deviation (score 2) were those only slightly affecting the therapeutic end-results, with effects between those of cases with scores 1 and 3. Non-performance of the necessary radiotherapeutic procedures was penalized by the highest score of 4. Statistical evaluation was performed with the BMDP software package, using variance analysis.

RESULTS

Bimonthly audits (six with a duration of 4-6 h in each institution) were carried out by three auditors from the evaluating departments; they reviewed a total of 452 cases in Department A, and 265 cases in Department B. Despite the comparable staffing and instrumental conditions, a markedly higher number (1.5 times) of new cases were treated in Department A, but with a lower quality of radiotherapy, as adequate values of qualifying parameters (1-6) were more frequent for the cases treated in Department B (85.3%, 94%, 83.4%, 28.3%, 41.9% and 81.1%) than for those in Department A (67%, 83.4%, 87.8%, 26.1%, 33.2% and 17.7%). The responsible division (including staff and instrumentation), the responsible physician and the type of the disease each exerted a highly significant effect on the quality level of the treatment. Statistical analysis revealed a positive influence of the curative (relative to the palliative/symptomatic) intention of the treatment on the level of quality, but the effect of the first radiotherapy (relative to the second or further one) was statistically significant in only one department. At the same time, the quality parameters did not vary with the referring institution, the auditing person or the serial number of the audit.

CONCLUSION

The external audit relating to the provision of radiotherapeutic care proved feasible with the basic conformity and compliance of the staff and resulted in valuable information to take correction measures.

[Methodology for the assessment of activities in primary health care]

A case study assessed health care activities in a primary health care facility to help develop methodologies of care work assessment and of the proposal of leading categories for the said processes, based on the work process. This study was developed at the Centro de Saúde Escola of the Faculdade de Medicina of the Universidade de São Paulo, using as empirical material the assessment of the care activities aimed at the emergency care and programmed care of 'chronic' patients (patients suffering from hypertension and diabetes). The analysis of these assessments which considered different moments of the work process (working objects, activity of agents, analysis of the working means and result of the work) suggest the need of creating analytical categories that will make workable the understanding of this process, inserting each moment to the work as a whole. The analytical categories proposed are: 1) workability as a means of making effective the proposed actions; 2) strategic effectiveness as a measure of the scope of the results that have been observed among patients and/or the population and that can be the result of the actions performed.

Developing a CQI program in a family medicine department

BACKGROUND

Efforts to implement continuous quality improvement (CQI) principles in ambulatory or primary care settings still lag behind efforts in the hospital setting. Many physicians view the concept of CQI with unconcealed skepticism; the process of ambulatory care is very different from that of hospital-based care; and the data necessary to guide CQI efforts are often either missing or inaccurate in the outpatient setting. Since fall 1995, the Department of Family Medicine (DFM) at the University of Michigan (Ann Arbor), including approximately 35 faculty members at seven family practice sites, has been engaged in CQI projects.

PLANNING AND IMPLEMENTATION

The CQI committee had a six-month deadline to lay out a plan for educating all faculty and staff in the importance of the CQI approach to problems; design methods for all faculty and staff to buy in to the concepts; and develop a plan to address basic clinical CQI activities, administrative systems change and work environment improvement, and larger ad hoc projects in clinical care, educational programs, and research programs.

IMPLEMENTATION

CQI activities were incorporated into the routine monthly business agendas at each clinical site, each of which had a functioning local committee and had begun development of at least one CQI project.

PROJECTING INTO THE FUTURE AND CONCLUSIONS

Cost cutting has further moved CQI from the sideline to center stage in the DFM's activities. An effective CQI program can be a major asset in the current competitive health care market, but designing and implementing an outpatient CQI program is a difficult and complex process. Three major problems--the ongoing resistance to change, the slow pace of adding CQI projects to already overburdened work schedules, and the need to conduct the program with ever-decreasing resources available-persist.

Continuous quality improvement in primary care: what's happening?

OBJECTIVES

Despite much health care interest in quality and Continuous Quality Improvement, there is little quantitative information about it. The purpose of this study was to measure the attitudes, activities, and organizational cultures concerning Continuous Quality Improvement in a group of Midwestern primary care clinics.

METHODS

Three surveys of the clinicians, nurses, and other staff in 44 primary care clinics in the metropolitan area of Minneapolis and St. Paul were conducted. These surveys assessed: (1) attitudes about quality improvement, (2) previous efforts in these clinics to use process improvement teams, and (3) the extent to which the clinics' organizational cultures were perceived as supporting quality. The Provider Attitude Survey was completed by clinicians and nurses; the Process Improvement Progress was completed by members of the best Continuous Quality Improvement teams, if any; and the Quality Systems Inventory was completed by all personnel.

RESULTS

Most of the clinical personnel reported support for various Continuous Quality Improvement concepts, but their understanding and experience were limited. Only 20 (45%) clinics had had at least one Continuous Quality Improvement team in the past, only five of the 12 teams with adequate information had completed an improvement cycle, and only seven reported improving a process with it. The mean clinic scores for quality culture were no better than those in other types of organizations.

CONCLUSIONS

Despite relatively favorable attitudes and some Continuous Quality Improvement activities, there appears to be a need to help clinics build skill and experience for the required care improvements.

Patient satisfaction with the general practitioner: a two-level analysis

OBJECTIVES

The authors examine how patient satisfaction with health care providers relates to either the individual characteristics of respondents or the characteristics of health care providers and the structural setting in which they work.

METHODS

Measures of three dimensions of patient satisfaction with the general practitioner (GP)--accessibility, interpersonal relationship, information given--were derived from an existing data set. Patients were nested with GPs. Multilevel analysis was used as the analyzing technique.

RESULTS

Between 90% and 95% of the variance in patient satisfaction scores is at the patient level, whereas the remaining 5% to 10% is at the GP or practice level. At the patient level, in addition to the usual predictor variables such as age and morbidity, which explain approximately 5% of the variance at this level, previous experiences with the general practitioner in the form of misunderstandings or incidents may play an important role in the emergence of dissatisfaction among patients.

CONCLUSIONS

This study demonstrated the usefulness of multilevel analysis in studying patient satisfaction scores. Findings indicate that the effectiveness of strategies directed at health care providers or services and aiming to improve the quality of care through the patient's eyes can be questioned when these strategies are based on general satisfaction scores only. More attention should be paid to the interaction process between patient and GP.