Instruments to Evaluate Nurse Residency Programs: A Review of the Literature

Beyond competence and retention: Developing a comprehensive evaluation framework for a rural primary care nurse practitioner residency program

ABSTRACT

With the rise in nurse practitioner (NP) residency programs, evaluations have largely focused on retention and competency completion for residents. There is a need for expanded evaluation to ensure the sustainability of NP residency programs, to ensure timely adaptations to address resident satisfaction, and to solidify a long-term pathway of NPs well prepared for rural practice. We created a family nurse practitioner (FNP) residency program with a comprehensive evaluation framework to prepare residents for practice in rural settings. The evaluation framework was developed through collaborative engagement of an external evaluation team, program leadership, and clinical site representatives. The evaluation framework of the FNP residency program combined resident assessment and holistic program evaluation, using a rapid continuous quality improvement (QI) approach. The evaluation considered three distinct perspectives: the resident, the peer coach, and the clinical site. The rapid continuous QI approach allowed program leadership to respond swiftly to programmatic challenges, improve the residency program in response to residents' reported experiences, and emphasize sustainability for continued program impact, while assessing residents' learning and performance. The program's data-driven evaluation approach has demonstrated its success in meeting the goals of the Health Resources and Services Administration funding by increasing the number of primary care providers in rural settings. The program's expansion and continued success have further validated the efficacy of this evaluation framework in assessing, improving, and ensuring the sustainability of APRN residency programs. This article calls for the adoption of similar evaluation strategies in future residency programs to promote their long-term success and impact in rural health care settings.

Assessing Nurse Competency in the Veterans Health Administration Registered Nurse Transition-to-Practice Residency Program: Item Pool Content Validation

BACKGROUND

New graduate registered nurse (RN) competencies are complex and difficult to measure. Psychometrically sound tools are needed to evaluate competencies for nurses participating in nurse residencies.

PURPOSE

Project aims were to develop an item pool for a tool to measure new graduate RN competencies for the Veterans Health Administration RN Transition-to-Practice Residency Program; validate item pool content; and use consensus methods to improve item pool content validity.

METHODS

A sequential, mixed-methods design was used. Item pool creation, content validation, and revisions included a collaborative process with the evaluation team, operational stakeholders, and subject matter experts (SMEs).

RESULTS

Inclusion of SMEs in item development enhanced item pool content validity to measure nurse competency. Stakeholder feedback ensured programmatic logistical and evaluation concerns were met.

CONCLUSIONS

Engaging SMEs in conceptualization, item development, and aligning existing standards enhanced item pool content validity to measure nurse competencies for new graduate RNs.

A Statewide Nurse Residency Program Evaluation

Across the nation, hospitals are answering the call to action for funding and requiring new-to-practice registered nurses to complete a nurse residency program (NRP). In response, all Maryland acute care hospitals successfully funded, adopted, and implemented a 12-month standardized, commercially available (vendored) NRP. However, although the hospitals use the same curriculum, their administrative practices, content, and deliverables vary. A statewide program evaluation was conducted by the Maryland Organization of Nurse Leaders, Inc./Maryland Nurse Residency Collaborative to understand the NRP structure and process variability, including implementation challenges, and what worked and what did not. Findings from 26 semi-structured interviews with Maryland acute care hospital NRP coordinators showed that leadership support and resources, differences in patient populations, number of nurse residents, and organizational requirements best explained programmatic adaptations. Using lessons learned, best practices are shared, including methods to garner leadership support and reduce adaptations. Although program customization is expected, program replicability and scalability are essential to achieving program outcomes. [J Contin Educ Nurs. 2023;54(9):421-429.].

Meeting Theory-to-Practice Gaps? Evaluation of New Graduates

For decades, nursing graduates have exhibited a gap between theory and practice. Healthcare organizations have implemented transition programs to support successful entry into practice. Nurse educators can restructure residency programs to help graduates develop the confidence and competence needed for practice by identifying the gaps between theory and practice for nursing graduates. This study's findings support continual process improvement through evaluation and contribute to the literature about the challenges new nurses encounter.

Measuring Nurse Residency Program Outcomes: Selecting the Right Tool

Nursing professional development practitioners are critical to nurse residency program (NRP) development, implementation, and evaluation. Conducting a program evaluation provides an opportunity to collect and analyze data on identified goals and outcome measures and determine if the program components need to modified or changed. Health care organizations need a systematic process for evaluating NRPs to ensure effectiveness of program delivery, provide evidence to demonstrate program value to the organization, report achievement of program outcomes, and for continuous quality improvement. With the growing importance of offering high-quality NRPs, nurses in professional development must critically consider the type of data required to demonstrate program value and use the best tools to collect outcome data. This article discusses a variety of data collection tools that can be selected to measure NRP outcomes. [J Contin Educ Nurs. 2022;53(9):389-392.].

Improving Practice Readiness Among Nurse Residents

Background Recent evidence indicates a mere 8% of new graduate RNs are prepared to make entry-level clinical judgments. Residency programs, designed to overcome the academic-practice gap, lack universal guidelines for content, delivery, structure, and consistent assessment of practice readiness prior to independent practice. Method Virtual simulation to overcome practice gaps, with pre- and post-assessment of practice readiness prior to independent practice, was implemented in an existing nurse residency program. Results A 373% improvement in virtual clinical performance scores, as well as reductions in medication errors, sentinel events, and failures to rescue, was noted. These improvements readily translated to practice, as noted in a corresponding decrease in the rate of practice errors during actual clinical performance. Conclusion Transformed pedagogical technologies and learning methodologies are providing promising interventions to overcome the practice readiness deficit and set the stage for safer practice among new graduate RNs. [J Contin Educ Nurs. 2022;53(9):411-416.].

Developing the Graduate Nurse Residency: An Oral History With Dr. Colleen Goode and Dr. Mary Krugman

Health care organizations face the challenge of needing newly licensed nurses to fill positions and facilitate competent care for patients. Wide variation in graduate nurse orientation programs, a growing complexity of care, and high graduate nurse turnover rates led to the development of nurse residency programs. The historical perspectives of two nursing pioneers involved in the development of a national model for nurse residency programs provide context to the importance of creating a vision, providing leadership, and applying an evidence-based rationale to structure a series of learning and work experiences designed to support graduate nurses as they transition into their first professional nursing position. [J Contin Educ Nurs. 2022;53(4):171-177.].

Establishing competency-based measures for Department of Veterans Affairs post-graduate nurse practitioner residencies

BACKGROUND

In the past decade, numerous nurse residency models have been created and implemented nationwide; however, validated specialty-specific competency standards have not been established to evaluate Nurse Practitioner (NP) resident core competencies.

PURPOSE

To report the specialty-specific competency assessment tool devised to assess Department of Veterans Affairs (VA) NP residents' competencies and discuss the VA NP residency program's effectiveness in expanding new graduate NP knowledge and skills in the veteran-centric care setting.

METHODS

The VA Nursing Academic Partnership NP residency faculty established and piloted a web-based Nurse Practitioner Resident Competency Assessment (NPRCA) instrument for the comprehensive, specialty-specific assessment of individual NP resident's skill competencies across 24 areas.

RESULTS

The VA specialty-specific competency assessment instrument demonstrates strong internal consistency. The robust VA NP residency program enhances new graduate NP competencies.

CONCLUSIONS

The VA NP residency model can further the goal of standardizing clinical competencies in NP residency programs.

External Evaluation of the Clinical Nurse Transition Program: The Next Step

Evaluating Nurse Transition programs has undergone an evolutionary process to more accurately report outcomes. Based on a previous work, this project used the Clinical Nurse Transition Program-Evaluation Tool to externally evaluate the effectiveness of nurse transition programs within the U.S. Army Nurse Corps. The results support the belief that a new graduate nurse performs better at 24 weeks within the Clinical Nurse Transition Program than at 12 weeks. In addition, new graduates perform above standards 6 months postgraduation.

A Psychometric Evaluation of the Casey-Fink Graduate Nurse Experience Survey

OBJECTIVE

The purpose of this study was to evaluate the psychometric properties of the Casey-Fink Graduate Nurse Experience Survey (CFGNES).

BACKGROUND

Transitioning from the student role to professional nurse is challenging and stressful. Accurate measurement of role transition is important because of concerns regarding retention.

METHODS

A secondary analysis of 71 919 graduate nurse responses to the CFGNES, collected 6 months into the Vizient/AACN residency program, was conducted. Psychometric testing included exploratory and confirmatory factor analysis. Internal consistency reliability was measured using Cronbach's α.

RESULTS

Analysis revealed a consistent 5-factor solution. Factors were labeled job satisfaction, support, organize/prioritize care, role confidence, and professional socialization. Results demonstrated the CFGNES is a reliable and valid instrument for measuring perceptions of role transition.

CONCLUSIONS

This study was needed to inform survey item revisions. The CFGNES continues to be valuable in providing voice to the needs of graduate nurses and evidence for improving nurse residency program outcomes.

The Journey of a Home Care Nurse Residency Program

UnityPoint at Home launched a Nurse Residency Program to attract newly licensed registered nurses to begin their career in home healthcare. The program broadened recruitment options by providing new graduates with a dedicated program to ease transition into the workforce, and allowed us to cultivate relationships with regional nursing education programs. Since its inception 3 years ago, the residency program has expanded to several organizational regions and added a hospice track. The 1-year retention rate is 93%, and for those who entered the program 2 or more years ago, the retention rate is 89%. The purpose of this article is to describe the journey of UnityPoint at Home's Nurse Residency Program, from development of the pilot to integration in several organizational regions.

Evaluation of Clinical Nurse Transition Program at US Army Hospitals

Introduction

The purpose of this evaluation was to determine the effectiveness of the clinical nurse transition program (CNTP) for newly commissioned Army nurses. This is one of the first evaluations conducted to document the outcomes of the Army Nurse Corps (ANC) transition-to practice residency program.

Materials and Methods

This project was a prospective pre-test/post-test design evaluation of newly licensed registered nurses (RNs) entering the military health system from Fall 2017 to Fall 2018 using Casey-Fink Readiness for Practice Survey and the New Graduate RN Transition Program Competency Assessment Tool. Data were analyzed using SAS 9.4. All tests were two-tailed as applicable and p-values less than 0.05 were considered significant. Missing data were treated as missing for individual items; their values were not imputed. No participant was excluded due to missing items. This evaluation was determined to be exempt from human subject protection regulations by the Human Protections Administrator at Womack Army Medical Center.

Results

A total of 92 pre-test surveys were sent to the nine CNTP training sites which actively trained RNs during the study timeframe. Due to a shipping error, 6 of the 92 completed pre-test surveys from one CNTP site were not received, for an overall response rate of 93.4% (86/92), from eight CNTP sites. Of the 86 participants from whom pre-tests were received, all 86 successfully returned their post-test survey, for a 100% response rate pre- to post- survey. The participants were predominately white (73.3%) and female (70.9%) with a median age of 23 years (IQR: 22–24 years) and an undergraduate GPA of 3.5 (IQR: 3.4–3.7, Table II). Quantitative data analysis revealed that after the training, all participants reported a significant improvement in providing safe (p < 0.0001) and quality care (p < 0.0001) for an average workload of in-patient patients. It also showed a significant improvement in team leading (p < 0.0001), effective decision making (p < 0.0001), incorporating evidence into practice (p < 0.0001), and demonstrated professionalism (p < 0.0001). The overall clinical competence and readiness to practice (p < 0.0001) also improved. Data shows that participants perceive significant improvement in all the objectives set by the ANC for the transition program.

Conclusion

The results of this program evaluation provide evidence that the CNTP has been a success in meeting the intended objectives. Participants showed a significant improvement in their perceived readiness and level of competency in clinical, technical, and leadership skills performance at the end of the program, as well as improved communication and teamwork. The CNTP experience provides an effective means to facilitate the development of newly licensed Army nurses’ clinical competence and confidence in practice. The study limitation includes the Casey-Fink Readiness for Practice Survey had relatively low reliability, but it was chosen as it was better aligned to the specified CNTP objectives. And also, with any self-reported evaluation, there is potential bias regarding the accuracy of participants’ self-perception. Follow-up study may include examining the demographics of the incoming nurses, such as their source of commission. To determine if and how the comfort and confidence self-reported by the new graduate nurses is reflective of actual ability, nurse preceptors may be asked to complete skills assessments or simulations with objective measurements may be used.