Defining and evaluating quality for ambulatory care educational programs

Stakeholder Participation and Cross-Sectoral Cooperation in a Quality Circle on Community-Based Teaching: Results of a Qualitative Interview Study

Introduction

Community-based learning in family medicine practices is an increasingly important part of the medical curriculum internationally. It is widely regarded as one solution to healthcare system needs, such as training and retaining a workforce willing to work in primary care. However, the perspectives of community-based medical educators and representatives from university-based medical education are rarely integrated. To improve teaching quality and promote exchange between those two sectors of medical education, the Institute for General Practice and Interprofessional Care at Tübingen University started a quality circle in family medicine teaching involving stakeholders from both sectors in 2018. The study aims to describe how the participants of this specific QC describe the cross-sectoral cooperation and participation of stakeholders in the quality management of community and university medical education.

Methods

After an observed meeting of the quality circle, semi-structured interviews were conducted with n=12 participants of the quality circle. Interview transcripts were analyzed using grounded theory.

Results

According to the participants, the quality circle provides a dynamic continuity which allows participants to navigate known barriers to transsectoral collaboration in the quality management of community-based medical education. The quality circle is perceived as an instrument for quality improvement that offered continuity and direction. At the same time, it allows for enough freedom and flexibility for the involved stakeholders to creatively work together on quality management and be inspired by their experiences.

Discussion

The quality circle has the potential to facilitate collaboration between the two teaching settings, form a creative community, and give medical students an active role in educational quality management.

Do quality indicators for general practice teaching practices predict good outcomes for students?

Keele medical students spend 113 days in general practices over our five-year programme. We collect practice data thought to indicate good quality teaching. We explored the relationships between these data and two outcomes for students; Objective Structured Clinical Examination (OSCE) scores and feedback regarding the placements. Though both are surrogate markers of good teaching, they are widely used. We collated practice and outcome data for one academic year. Two separate statistical analyses were carried out: (1) to determine how much of the variation seen in the OSCE scores was due to the effect of the practice and how much to the individual student. (2) to identify practice characteristics with a relationship to student feedback scores. (1) OSCE performance: 268 students in 90 practices: six quality indicators independently influenced the OSCE score, though without linear relationships and not to statistical significance. (2) Student satisfaction: 144 students in 69 practices: student feedback scores are not influenced by practice characteristics. The relationships between the quality indicators we collect for practices and outcomes for students are not clear. It may be that neither the quality indicators nor the outcome measures are reliable enough to inform decisions about practices' suitability for teaching.

Course Offerings in the Fourth Year of Medical School: How U.S. Medical Schools Are Preparing Students for Internship

The fourth year of medical school remains controversial, despite efforts to reform it. A committee from the Clerkship Directors in Internal Medicine and the Association of Program Directors in Internal Medicine examined transitions from medical school to internship with the goal of better academic advising for students. In 2013 and 2014, the committee examined published literature and the Web sites of 136 Liaison Committee on Medical Education-accredited schools for information on current course offerings for the fourth year of medical school. The authors summarized temporal trends and outcomes when available.Subinternships were required by 122 (90%) of the 136 schools and allow students to experience the intern's role. Capstone courses are increasingly used to fill curricular gaps. Revisiting basic sciences in fourth-year rotations helps to reinforce concepts from earlier years. Many schools require rotations in specific settings, like emergency departments, intensive care units, or ambulatory clinics. A growing number of schools require participation in research, including during the fourth year. Students traditionally take fourth-year clinical electives to improve skills, both within their chosen specialties and in other disciplines. Some students work with underserved populations or seek experiences that will be henceforth unavailable, whereas others use electives to "audition" at desired residency sites. Fourth-year requirements vary considerably among medical schools, reflecting different missions and varied student needs. Few objective outcomes data exist to guide students' choices. Nevertheless, both medical students and educators value the fourth year of medical school and feel it can fill diverse functions in preparing for residency.

Moving forward in GME reform: a 4 + 1 model of resident ambulatory training

BACKGROUND

Traditional ambulatory training models have limitations in important domains, including opportunities for residents to learn, fragmentation of care delivery experience, and satisfaction with ambulatory experiences. New models of ambulatory training are needed.

AIM

To compare the impact of a traditional ambulatory training model with a templated 4 + 1 model.

SETTING

A large university-based internal medicine residency using three different training sites: a patient-centered medical home, a hospital-based ambulatory clinic, and community private practices.

PARTICIPANTS

Residents, faculty, and administrative staff.

PROGRAM DESCRIPTION

Development of a templated 4 + 1 model of residency where trainees do not attend to inpatient and outpatient responsibilities simultaneously.

PROGRAM EVALUATION

A mixed-methods analysis of survey and nominal group data measuring three primary outcomes: 1) Perception of learning opportunities and quality of faculty teaching; 2) Reported fragmentation of care delivery experience; 3) Satisfaction with ambulatory experiences. Self-reported empanelment was a secondary outcome. Residents' learning opportunities increased (p = 0.007) but quality of faculty teaching was unchanged. Participants reported less fragmentation in the care residents provide patients in the inpatient and outpatient setting (p < 0.0001). Satisfaction with ambulatory training improved (p < 0.0001). Self-reported empanelment also increased (p < 0.0001). Results held true for residents, faculty, and staff at all three ambulatory training sites (p < 0.0001).

DISCUSSION

A 4 + 1 model increased resident time in ambulatory continuity clinic, enhanced learning opportunities, reduced fragmentation of care residents provide, and improved satisfaction with ambulatory experiences. More studies of similar models are needed to evaluate effects on additional trainee and patient outcomes.

A scoping review of undergraduate ambulatory care education

BACKGROUND

Since a disproportionate amount of medical education still occurs in hospitals, there are concerns that medical school graduates are not fully prepared to deliver efficient and effective care in ambulatory settings to increasingly complex patients.

AIMS

To understand the current extent of scholarship in this area.

METHOD

A scoping review was conducted by searching electronic databases and grey literature sources for articles published between 2001 and 2011 that identified key challenges and models of practice for undergraduate teaching of ambulatory care. Relevant articles were charted and assigned key descriptors, which were mapped onto Canadian recommendations for the future of undergraduate medical education.

RESULTS

Most of the relevant articles originated in the United States, Australia, or the United Kingdom. Recommendations related to faculty development, learning contexts and addressing community needs had numerous areas of scholarly activity while scholarly activity was lacking for recommendations related to inter-professional practice, the use of technology, preventive medicine, and medical leadership.

CONCLUSIONS

Systems should be established to support education and research collaboration between medical schools to develop best practices and build capacity for change. This method of scoping the field can be applied using best practices and recommendations in other countries.

Pharmacy preceptors' views on the value and optimal frequency of quality assurance visits to advanced pharmacy practice experience sites

OBJECTIVE

To determine volunteer preceptors' perceived value and desired frequency of quality assurance visits by experiential education faculty members.

METHODS

An electronic survey instrument was sent to 235 volunteer preceptors.

RESULTS

A 71.5% response rate was achieved. Nearly 90% of respondents indicated that onsite visits met their needs. Approximately 50% of respondents preferred monthly onsite visits, 17% preferred every other month, and 32% preferred once per year.

CONCLUSIONS

A quality assurance program for preceptors and experiential sites that includes onsite visits from experiential education faculty members meets multiple needs of the college and the preceptors. More research is needed to determine the impact of this method of quality assurance on experiential education.

Continuity clinic satisfaction and valuation in residency training

BACKGROUND

Internal Medicine residency training in ambulatory care has been judged inadequate, yet how trainees value continuity clinic and which aspects of clinic affect attitudes are unknown.

OBJECTIVES

To determine the value that Internal Medicine residents place on continuity clinic and how clinic precepting, operations, and patient panels affect its valuation.

DESIGN AND MEASUREMENTS

A survey on ambulatory care was developed, including questions on career choice and the value of clinical training experiences. Independent variables were Likert-scale ratings (1 = disagree strongly/no value; 3 = neutral; 5 = agree strongly/high value) on preceptors, patients, operations, and resident characteristics. Odds ratios and stepwise multivariate logistic regression with clustering were used to evaluate associations between clinic valuation and independent variables.

SUBJECTS

Internal medicine residents at 3 residency programs.

RESULTS

218 of 260 residents (83.8%) completed the survey. Resident ratings were highest on diversity of illness seen (4.1), medical record systems used (4.1), and contact with preceptors who were receptive to questions (4.8). Resident ratings were lowest on economic diversity of patients (2.7), interruptions from inpatient wards (3.1), and contact with preceptors who taught history and physical exam skills (3.5). High ratings on all precepting issues and nearly all operational issues were associated with valuing clinic. With multivariate analysis, high ratings of preceptors as role models were most strongly associated with valuing clinic (corrected relative risk 3.44). A planned career in general Internal Medicine was not associated with valuing clinic.

CONCLUSIONS

Satisfaction with preceptors, particularly as role models, and clinic operations correlate with the value residents place on continuity clinic.

An evaluation of the Rural Medical Education Program of the State University Of New York Upstate Medical University, 1990-2003

PURPOSE

The Rural Medical Education Program (RMED) of the State University of New York (SUNY) Upstate Medical University is a 36-week clinical experience in rural communities for medical students that began in 1989. The authors sought to assess RMED's success in providing a valuable educational experience for students that assists rural communities recruit physicians.

METHOD

In 2004, the authors used the Physician Masterfiles of the American Medical Association to compare practice locations of SUNY Upstate graduates who completed RMED with those who did not; surveyed former RMED students to assess their satisfaction with their practice location and the importance of RMED in helping them choose a location; interviewed hospital administrators in communities that have hosted RMED students to understand the impact of RMED on host communities; and compared United States Medical Licensing Examination Step 2 scores of RMED students with those of non-RMED students to evaluate educational attainment.

RESULTS

A greater percentage of former RMED students practiced in rural locations [22/86 (26%)] than did non-RMED students [95/1,307 (7%)]. Ninety-one percent (69/76) of former RMED students were satisfied with their location, and 84% (64/76) believed that RMED was important in helping them choose a location. Hospital administrators viewed the program highly because it helped them recruit physicians and benefitted their medical staff. RMED students had higher adjusted mean Step 2 scores than did non-RMED students (212.3 versus 199.1).

CONCLUSION

The RMED program has successfully met its goals of providing a valuable educational experience for medical students and assisting rural communities recruit physicians.

Medical Students' and Residents' preferred site characteristics and preceptor behaviours for learning in the ambulatory setting: a cross-sectional survey

BACKGROUND

Medical training is increasingly occurring in the ambulatory setting for final year medical students and residents. This study looks to identify if gender, school, level of training, or specialty affects learner's (final year medical students and residents) preferred site characteristics and preceptor behaviours for learning in the ambulatory setting.

METHODS

All final year medical students and residents at the five medical schools in Ontario (N = 3471) were surveyed about the site characteristics and preceptor behaviours most enhancing their learning in the ambulatory setting. Preferred site characteristics and preceptor behaviours were rank ordered. Factor analysis grouped the site characteristics and preceptor behaviours into themes which were then correlated with gender, school, level of training, and specialty.

RESULTS

Having an adequate number and variety of patients while being supervised by enthusiastic preceptors who give feedback and are willing to discuss their reasoning processes and delegate responsibility are site characteristics and preceptor behaviours valued by almost all learners. Some teaching strategies recently suggested to improve efficiency in the ambulatory teaching setting, such as structuring the interview for the student and teaching and reviewing the case in front of the patient, were found not to be valued by learners. There was a striking degree of similarity in what was valued by all learners but there were also some educationally significant differences, particularly between learners at different levels and in different specialties. Key findings between the different levels include preceptor interaction being most important for medical students as opposed to residents who most value issues pertaining to patient logistics. Learning resources are less valued early and late in training. Teaching and having the case reviewed in front of the patient becomes increasingly less valued as learners advance in their training. As one approaches the end of ones' training office management instruction becomes increasingly valued. Differences between specialties pertain most to the type of practice residents will ultimately end up in (ie: office based specialties particularly valuing instruction in office management and health care system interaction).

CONCLUSIONS

Preceptors need to be aware of, and make efforts to provide, teaching strategies such as feedback and discussing clinical reasoning, that learners have identified as being helpful for learning. If strategies identified as not being valued for learning, such as teaching in front of the patient, must continue it will be important to explore the barriers they present to learning. Although what all learners want from their preceptors and clinic settings to enhance their learning is remarkably similar, being aware of the educationally significant differences, particularly for learners at different levels and in different specialties, will enhance teaching in the ambulatory setting.

Evaluation of self-directed clinical education: validation of an instrument

AIM

To explore the evaluation of self-directed, integrated clinical education.

METHODS

We delivered a quantitative and qualitative, self-report questionnaire to students through their web-based learning management system. The questionnaire was distributed 4 times over 1 year, each time in 2 parts. A generic part evaluated boundary conditions for learning, teaching activities and "real patient learning". Factor analysis with varimax rotation was used to validate the constructs that made up the scale and to stimulate hypotheses about how they interrelated. A module-specific part evaluated real patient learning of the subject matter in the curriculum.

RESULTS

A total of 101 students gave 380 of a possible 404 responses (94%). The generic data loaded onto 4 factors, corresponding to: firm quality; hospital-based teaching and learning; community and out-patient learning, and problem-based learning (PBL). A 5-item quality index had content, construct and criterion validity. Quality differed greatly between firms. Self-evaluation of module-specific, real patient learning was also valid. It was strongly influenced by the specialty interests of hospital firms.

CONCLUSIONS

Quality is a multidimensional construct. Self-report evaluation of real patient learning is feasible, and could be capitalised on to promote reflective self-direction. The social and material context of learning is an important dimension of educational quality.

Teaching the teachers: national survey of faculty development in departments of medicine of U.S. teaching hospitals

OBJECTIVE

To determine the prevalence, topics, methods, and intensity of ongoing faculty development (FD) in teaching skills.

DESIGN

Mailed survey.

PARTICIPANTS

Two hundred and seventy-seven of the 386 (72%) U.S. teaching hospitals with internal medicine residency programs.

MEASUREMENTS

Prevalence and characteristics of ongoing FD.

RESULTS

One hundred and eight teaching hospitals (39%) reported ongoing FD. Hospitals with a primary medical school affiliation (university hospitals) were more likely to have ongoing FD than non-university hospitals. For non-university hospitals, funding from the Health Resources Services Administration and >50 house staff were associated with ongoing FD. For university hospitals, >100 department of medicine faculty was associated. Ongoing programs included a mean of 10.4 topics (standard deviation, 5.4). Most offered half-day workshops (80%), but 22% offered > or =1-month programs. Evaluations were predominantly limited to postcourse evaluations forms. Only 14% of the hospitals with ongoing FD (5% of all hospitals) had "advanced" programs, defined as offering > or =10 topics, lasting >2 days, and using > or =3 experiential teaching methods. These were significantly more likely to be university hospitals and to offer salary support and/or protected time to their FD instructors. Generalists and hospital-based faculty were more likely to receive training than subspecialist and community-based faculty. Factors facilitating participation in FD activities were supervisor attitudes, FD expertise, and institutional culture.

CONCLUSIONS

A minority of U.S. teaching hospitals offer ongoing faculty development in teaching skills. Continued progress will likely require increased institutional commitment, improved evaluations, and adequate resources, particularly FD instructors and funding.

Assessing quality and costs of education in the ambulatory setting: a review of the literature

PURPOSE

Time-pressured interactions with little direct observation or feedback characterize teaching in ambulatory settings. The authors report findings from the literature on teaching and learning in the ambulatory setting and propose opportunities for further research that addresses these barriers.

METHOD

The authors searched 1995-1999 databases for all empirical studies that focused on research conducted in ambulatory settings. Publications were reviewed for evidence of inclusion criteria. Findings were sorted into categories previously described for defining and evaluating quality of ambulatory care educational programs.

RESULTS

Most studies were conducted in departments of internal medicine (40%), focused on medical students (43%), and took place in a single program (77%), making generalizations difficult. Students and residents are learning in ambulatory environments, and the types of patients they encounter are likely to prepare them for practice. Patient care outcomes have emerged as a measure of learning. Teachers may be the single most important factor, yet they lack self-confidence as teachers. Community-based preceptors teach because of enjoyment of teaching and the opportunity to stay current. However, none of the studies addressed the impact of the Medicare documentation requirements on satisfaction with teaching. Teaching settings cost about one third more than non-teaching settings to operate.

CONCLUSION

This review identifies many gaps in our knowledge of effective clinical teaching practices, and of learning environments in which that teaching takes place. The predominance of single-institution studies limits generalizability of current findings. A prioritized research agenda should be established and funded, focusing on improving the efficiency and effectiveness of teaching and learning in ambulatory settings.

Fieldwork education: The future of occupational therapy depends on it

Fieldwork continues to be the cornerstone of preparation for entry-level occupational therapy clinicians. During the past five years the Canadian healthcare system has experienced decreases in federal funding, organizational changes such as the movement to programme management, and increased focus on community-based needs. Two fieldwork projects were tested at a large health sciences facility, to meet the challenge of providing effective fieldwork experiences that build on current educational methodologies while still responding to changing realities in healthcare practice. The academic programme with which the health sciences facility is affiliated has developed a strong focus on self-directed learning, using problem-based and small group learning formats. The development, implementation and evaluation of the two fieldwork projects is described. Discussion focuses on the processes needed to facilitate innovative, flexible fieldwork models.

Measuring instructional quality in community-orientated medical education: looking into the black box

OBJECTIVES

Decentralizing medical education to community settings has raised issues of instructional quality. The need to evaluate community-based instruction accents the need to adopt a systems perspective, moving beyond factors known to comprise general clinical teaching effectiveness to include factors that focus on instruction as a process. Application of evaluation models using traditional input-output analysis can be flawed. This approach--dubbed the 'black box'--typically examines inputs and outputs, but often ignores throughputs.

DESIGN

In this article we open the black box, using theory to examine the underlying processes that define community-based medical education. We first describe the components and processes of an instructional model that is framed by the philosophy of quality and grounded in experiential learning theory. Without examining the critical processes at work inside the black box - i.e. how students come to acquire clinical knowledge and how behaviours are influenced - it is difficult to assess which programme features contribute to success. Tensions created by the absence of consensus on the outcomes of instruction and the challenge of developing adequate measures are highlighted.

SETTING

State University of New York at Buffalo.

SUBJECTS

Clinician-teachers, learners and patients in the environment.

RESULTS AND CONCLUSIONS

We conclude with describing a tool for evaluating community-based instruction that is guided by the context of our model.