Development and psychometric evaluation of an instrument to measure knowledge, skills, and attitudes towards quality improvement in health professions education: The Beliefs, Attitudes, Skills, and Confidence in Quality Improvement (BASiC-QI) Scale

Twelve tips for creating a longitudinal quality improvement and safety education for early health professions students

Despite the numerous calls for integrating quality improvement and patient safety (QIPS) curricula into health professions education, there are limited examples of effective implementation for early learners. Typically, pre-clinical QIPS experiences involve lectures or lessons that are disconnected from the practice of medicine. Consequently, students often prioritize other content they consider more important. As a result, they may enter clinical settings without essential QIPS skills and struggle to incorporate these concepts into their early professional identity formation. In this paper, we present twelve tips aimed at assisting educators in developing QIPS education early in the curricula of health professions students. These tips address various key issues, including aligning incentives, providing longitudinal experiences, incorporating real-world care outcomes, optimizing learning environments, communicating successes, and continually enhancing education and care delivery processes.

The TeamSTEPPS for Improving Diagnosis Team Assessment Tool: Scale Development and Psychometric Evaluation

INTRODUCTION

One in three patients is affected by diagnosis-related communication failures. Only a few valid and reliable instruments that measure teamwork and communication exist, and none of those focus on improving diagnosis. The authors developed, refined, and psychometrically evaluated the TeamSTEPPSⓇ for Improving Diagnosis Team Assessment Tool (TAT), which assesses diagnostic teamwork and communication in five critical teamwork domains and can be used to identify strengths and opportunities for improvement and monitor performance.

METHODS

The TAT was administered as a cross-sectional survey to 360 health professionals across nine diverse US health systems. Content and construct validity were evaluated through pilot implementation and subject matter expert review. Reliability and internal consistency were assessed with Cronbach's alpha. To understand sources of variation in TAT scores and assess the tool's consistency across diverse health care organizations, generalizability theory (G-theory) was used. Best practices in screening for careless responding identified participants with random or nonvarying responses.

RESULTS

Analyses indicated strong support for the tool. Content validity findings indicated that the TAT encompassed relevant diagnostic improvement teamwork and communication content. Construct validity, evaluated through pilot implementations, demonstrated the tool's effectiveness in assessing teamwork categories. Reliability analyses confirmed the TAT's internal consistency, with an overall Cronbach's alpha of 0.97. Each dimension of the TAT exhibited good reliability coefficients, ranging from 0.83 to 0.95. G-theory analysis showed that variations in TAT scores were primarily attributed to respondents (28.0%) and scale dimensions (59.6%); both are desirable facets of variation. Further, examination of careless respondents ensured the accuracy and quality of the results, enhancing the TAT's credibility as a valuable diagnostic improvement tool.

CONCLUSION

Psychometric evaluation demonstrated that the TAT is a reliable and valid instrument for assessing teamwork and communication among and across diagnostic teams. The TAT adds a novel, evidence-based, psychometrically sound measurement tool to help advance diagnostic teamwork and communication to improve patient care and outcomes.

From Theory to Practice: Development and Evaluation of a Quality Improvement Curriculum for Psychiatry Residents

OBJECTIVES

Quality improvement (QI) is a systematic approach used to analyze and address problems in healthcare. Evidence of its success has led some national regulatory bodies to require QI education in residency training programs. However, limited work to date has demonstrated how residency programs can integrate best practices in QI education to design their own curriculum. This study describes the implementation and evaluation of a new QI curriculum, grounded in a theoretical model of how QI education works, for Canadian psychiatry residents.

METHODS

PGY-2 and PGY-4 psychiatry residents received a 2.5-h mixed didactic and simulation-based QI workshop as a part of the 2021-2022 academic curriculum. Their knowledge and attitudes toward QI were assessed using the QI Knowledge Application Tool Revised (QIKAT-R) and the Beliefs and Attitudes subscale of the Beliefs, Attitudes, Skills, and Confidence in QI (BASiC-QI).

RESULTS

Eleven of 12 residents (92%) who completed the curriculum participated in the study. Average QIKAT-R scores improved from 4.45 to 7.00. Average BASiC-QI Beliefs and Attitudes subscale scores increased by 5.55 points. Residents reported enjoying QI and an increased desire to participate in future QI projects.

CONCLUSION

This study demonstrates how a programme theory of QI education can be used to develop an effective, locally-tailored curriculum. This approach can be replicated by other educators to develop or improve QI curricula.

A theoretical and practical approach to quality improvement education

Quality Improvement (QI) knowledge and skills are required at all levels of physician training. System improvement efforts need to include understanding of health disparities and design of interventions to reduce those disparities, thus health equity needs to be integrated into QI education. Payors, accreditation bodies and health systems' emphases on QI result in the need for QI curricula that meet the needs of diverse learners. This article presents a theoretical background and practical tools for designing, implementing, and evaluating a QI educational program across the spectrum of physician training with an emphasis on competency-based education and a goal of continuous practice improvement. Practice-based learning and improvement and systems-based practice are two core domains of competencies for readiness to practice. These competencies can be met through the health systems science framework for studying improvement in patient care and health care delivery coupled with QI science. Curricula should incorporate interactive learning of theory and principles of QI as well as mentored, experiential QI project work with multidisciplinary teams. QI projects often develop ideas and implement changes but are often inconsistent in studying intervention impacts or reaching the level of patient outcomes. Curriculum design should incorporate adult learning principles, competency based medical education, environmental and audience factors, and formats for content delivery. Key QI topics and how they fit into the clinical environment and teaching resources are provided, as well as options for faculty development. Approaches to evaluation are presented, along with tools for assessing learner's beliefs and attitudes, knowledge and application of QI principles, project evaluation, competency and curriculum evaluation. If the goal is to empower the next generation of change agents, there remains a need for development of scientific methodology and scholarly work, as well as faculty development and support by institutions.

A Scoring Rubric for the Knowledge Section of the Systems Quality Improvement Training and Assessment Tool

INTRODUCTION

Quality improvement (QI) competencies for health professions trainees were developed to address health care quality. Strategies to integrate QI into curricula exist, but methods for assessing interdisciplinary learners' competency are less developed. We refined the Knowledge section scoring rubric of the Systems Quality Improvement Training and Assessment Tool (SQI TAT) and examined its validity evidence.

METHODS

In 2017, the SQI TAT Knowledge section was expanded to cover seven core QI concepts, and the scoring rubric was refined. Three coders independently scored 35 SQI TAT Knowledge sections (18 pretests, 17 posttests). Interrater reliability was assessed by percent agreement and Cohen's kappa for individual variables and by Lin's concordance correlation for total scores for knowledge and application. Concurrent validity was assessed by comparing responses from two groups with different QI exposure and evaluating whether differences in exposure were measured.

RESULTS

Total-score interrater reliability average measures of concordance were .89 for all coders and >.70 for six of seven concept scores. The total score discriminated the two groups (p <. 05), and five of seven concept scores were higher for the group with more QI experience. Total scores were significantly higher posttest than pretest (p < .001), with improvement in posttest knowledge scores.

DISCUSSION

The SQI TAT Knowledge section provides a comprehensive assessment of QI knowledge. The scoring rubric was able to discriminate QI knowledge along a continuum. The SQI TAT Knowledge section is not linked to a clinical context, making it useful for assessing interprofessional learners and varying education levels.

Development and Testing of the Quality Improvement Self-efficacy Inventory

Quality improvement is paramount for patient safety. Leading change for quality improvement requires nurses with knowledge and skills beyond the clinical management of patients. In this study, staff nurses working in hospitals throughout Alabama were asked via an online survey to rate their quality improvement knowledge and skills using the new 10-item Quality Improvement Self-Efficacy Inventory (QISEI) and their perceptions of the nursing work environment using the Practice Environment Scale of the Nursing Work Index. Nurses (N = 886) rated the basic quality improvement items higher than the more advanced items. Several nurse characteristics and the nursing work environment were associated with nurses' ratings of their quality improvement knowledge and skills. Educators and administrators in health care organizations can use QISEI to gauge their nurses' knowledge and skills and then develop continuous professional development opportunities aimed at improving quality and safety competencies.

Baseline Quality Improvement Capacity of 33 Endocrinology Centers Participating in the T1D Exchange Quality Improvement Collaborative

This article describes the evolution of the Type 1 Diabetes Exchange Quality Improvement Collaborative (T1DX-QI) and provides insight into the development and growth of a successful type 1 diabetes quality improvement (QI) program. Since its inception 8 years ago, the collaborative has expanded to include centers across the United States with varying levels of QI experience, while simultaneously achieving many tangible improvements in type 1 diabetes care. These successes underscore the importance of learning health systems, data-sharing, benchmarking, and peer collaboration as drivers for continuous QI. Future efforts will include recruiting additional small- to medium-sized centers focused on adult care and underserved communities to further the goal of improving care and outcomes for all people living with type 1 diabetes.

A Tale of Four Programs: How Residents Learn About Quality Improvement during Postgraduate Medical Education at the University of Calgary

Phenomenon: Quality Improvement (QI) has become increasingly incorporated into competency frameworks for physician learners over the past two decades. As contemporary medical education adapts competency-based models of training, learners will be required to demonstrate competency in QI. There is a need to explore how various residency programs are currently teaching QI, including how residents might participate in experiential QI activities, and the various outcomes of these strategies. Approach: A collective case study examined how residents in four postgraduate programs at a single academic institution have learned about QI since the formal integration of QI as a cross-cutting competency. Data from surveys, interviews, observations, and archival records were collected in order to develop a comprehensive understanding of each case in its real-life context and explore current and historical trends and patterns within and across the four programs. Findings: Teaching and resident involvement in QI projects increased across all four programs since its formal integration into the national physician competency framework. Two programs had a longitudinal, hybrid QI curriculum involving didactic and experiential components. Two programs had didactic-alone QI curricula, with minimal resident engagement in applied QI activities. Between-program differences were quantified with regards to learning climate, safety climate, QI knowledge, skills, and attitudes, attitudes toward research during residency, and quality of mentorship for scholarly activities. Residents in programs with experiential learning reported higher knowledge, skills, and attitudes toward QI were motivated to lead improvement efforts in their future practice. Residents in programs with didactic-only QI teaching perceived that the historical operationalization of the scholarly project as research was a barrier to their involvement in QI, as it was not valued or legitimized in their academic and clinical contexts. Common barriers and facilitators to engagement with QI across all programs included time, mentorship, motivation, and competing demands such as feeling pressure to conduct research in order to obtain competitive fellowship positions or employment. Common across all programs was the perception of the residency scholarly project requirement as a "checkbox." Associations were quantified between the constructs of learning climate with safety culture, and safety culture with QI knowledge, skills, and attitudes. Insights: While hybrid QI curricula with experiential learning remains an effective curriculum strategy, tensions between research and QI may be a critical barrier to learner engagement in experiential activities. In addition to providing learners with support, time, mentorship, and explicitly communicating the value QI by the program, the local safety culture may impact QI learning beyond the core curriculum. Reconceptualization of the scholarly project requirements to normalize QI activities and recognizing the potential influence of the local organizational culture on QI learning and how trainees can positively or negatively shape these cultures warrants consideration.

A realist synthesis of quality improvement curricula in undergraduate and postgraduate medical education: what works, for whom, and in what contexts?

BACKGROUND

With the integration of quality improvement (QI) into competency-based models of physician training, there is an increasing requirement for medical students and residents to demonstrate competence in QI. There may be factors that commonly facilitate or inhibit the desired outcomes of QI curricula in undergraduate and postgraduate medical education. The purpose of this review was to synthesise attributes of QI curricula in undergraduate and postgraduate medical education associated with curricular outcomes.

METHODS

A realist synthesis of peer-reviewed and grey literature was conducted to identify the common contexts, mechanisms, and outcomes of QI curricula in undergraduate and postgraduate medical education in order to develop a programme theory to articulate what works, for whom, and in what contexts.

RESULTS

18854 records underwent title and abstract screening, full texts of 609 records were appraised for eligibility, data were extracted from 358 studies, and 218 studies were included in the development and refinement of the final programme theory. Contexts included curricular strategies, levels of training, clinical settings, and organisational culture. Mechanisms were identified within the overall QI curricula itself (eg, clear expectations and deliverables, and protected time), in the didactic components (ie, content delivery strategies), and within the experiential components (eg, topic selection strategies, working with others, and mentorship). Mechanisms were often associated with certain contexts to promote educational and clinical outcomes.

CONCLUSION

This research describes the various pedagogical strategies for teaching QI to medical learners and highlights the contexts and mechanisms that could potentially account for differences in educational and clinical outcomes of QI curricula. Educators may benefit from considering these contexts and mechanisms in the design and implementation of QI curricula to optimise the outcomes of training in this competency area.

Closing the gap: a transatlantic collaboration to foster quality improvement training in graduate entry medical students using applications of QI methodologies to medical education

The alarming prevalence of medical error and adverse events in the health system raises a call to action to ensure that doctors in training receive adequate training in quality improvement (QI). Training medical students in QI remains a challenge given time constraints, lack of clinical exposure, and already saturated curricula. In some instances, QI training may be delivered during clerkship through didactic, and in some instances, and experiential learning. Preclinical years of medical school remain focused on introducing students to scientific and clinical concepts, rarely do they learn about QI. The Program for Innovation in Scholarship and Medicine (PRISM) is a programme that introduces first-year medical students to the fundamentals of QI using their experience as a medical student as the context. PRISM is a condensed QI curriculum that is delivered through an international partnership, based on a previously piloted programme at a Canadian medical school. Following an introductory workshop, medical students work in teams to develop QI proposals (project charters) which detail how QI principles and tools can generate small-scale improvements within their educational programme. Project charters are assessed by a team of faculty and upper year students, who have previously participated. On completion of the programme, students demonstrated increased knowledge, skills, and attitudes towards QI. Programme participants were satisfied with the structure and expectations of PRISM and expressed a newfound interest in QI. Nearly all participants would recommend PRISM to another medical student. In conclusion, PRISM serves as a resourceful, efficient educational approach for preclerkship students that provides an introduction to the concepts of QI in order for early trainees to build on baseline knowledge and skills throughout their training.