Quality, methods, and recommendations of systematic reviews on measures of evidence-based practice: an umbrella review

Perspectives on how evidence-based practice changes over time: A qualitative exploration of occupational therapy and physical therapy graduates

RATIONALE

The integration of evidence-based practice (EBP) into rehabilitation education programs has been widespread, but little is known about how graduates' EBP competencies evolve over time.

AIMS AND OBJECTIVES

To explore how and why the use of EBP by occupational therapists (OTs) and physical therapists (PTs) evolves during the first 3 years of clinical practice.

METHOD

We used an interpretive description methodology. We conducted semi-structured interviews with OTs and PTs who participated in a minimum of three out of four time points in a previous longitudinal pan-Canadian mixed methods study. Data analysis was guided by Braun and Clarke's approach to thematic analysis.

RESULTS

Seventeen clinicians (13 OTs and 4 PTs) participated in the study. Our analysis identified six overarching themes: (1) evolution of "what EBP is and what it means"; (2) over time, evidence takes a back seat; (3) patients and colleagues have a vital and perennial role in clinical decision making; (4) continuing professional development plays a vital role in EBP; (5) personal attitudes and attributes influence EBP; and (6) organizational factors influence EBP.

CONCLUSION

Our study highlights the dynamic nature of EBP use among OTs and PTs in the first 3 years of clinical practice, emphasizing the need for contextualized approaches and ongoing support to promote evidence-informed healthcare in rehabilitation.

Effect of Citation Numbers and Team Members on the Possibility of and Time Needed to Complete Screening for Systematic and Scoping Reviews

Covidence is a tool that facilitates systematic review screening. Librarians obtained Covidence review data from a large research institution to identify the effect that the total number of citations, team members, and librarian collaboration has on the possibility of completion and time needed for screening. The fewer the citations and the fewer the citations per collaborator, the more likely the team is to complete title and abstract and full text screening, and the faster they will finish the screening process. This relationship was stronger for the number of citations per collaborator than the number of citations alone.

Measuring evidence-based practice knowledge, skills, attitudes, and behavior in students of manual therapy education programs: A scoping review of instruments and their measurement properties

OBJECTIVE

The purpose of this review was to identify assessment instruments and their measurement properties for assessing evidence-based practice (EBP) knowledge, skills, attitudes, and behavior among students of manual therapy education programs.

METHODS

7 electronic databases were systematically searched from inception to May 19, 2023. (MEDLINE, EMBASE, CINAHL, ERIC, EBSCO Discovery, LISA, Google Scholar.) Search terms were subject headings specific to each database (MeSH in MEDLINE) and relevant to evidence-based practice, assessment tools/instruments, and manual therapy healthcare professions. Eligible studies included students of manual therapy education programs (chiropractic, physiotherapy, occupational therapy, osteopathy) and provided evidence supporting instrument measurement properties (reliability, validity). Titles and abstracts were screened by 2 reviewers. Data on each instrument and its properties were extracted and tabulated by 2 reviewers. Instruments were compared using the Classification Rubric for EBP Assessment Tools in Education (CREATE) framework including the 5 steps in the EBP model. Joanna Briggs Institute methodology and Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews checklist were followed.

RESULTS

5 studies were identified, (3 physiotherapy, 2 chiropractic) Two studies used a physiotherapy-focused modification of the Fresno test. One study presented the Knowledge of Research Evidence Competencies instrument. Two studies presented original instruments. Instruments focused on the knowledge domain and did not assess all 5 EBP model steps.

CONCLUSION

The current literature does not address all 5 steps of the EBP model. The identified instruments have the potential to ensure chiropractic institutions are graduating chiropractors who are highly skilled in evidence-based practice.

Refining and adapting the measurement properties of evidence-based practice measures for physiotherapy students

OBJECTIVE

There is a lack of reliable and valid evidence-based practice (EBP) measures for physiotherapy students. This study validated EBP-student (EBP-S) measures for physiotherapy students.

METHODS

EBP measures developed from previous research were cross-culturally validated for use by physiotherapy students. The adapted EBP-S consisted of six measures: use of EBP, EBP activities, EBP knowledge, self-efficacy for EBP, attitudes towards EBP, and perceptions of the teaching and assessment of EBP in the curriculum. The final version was completed by physiotherapy students (n = 335). The psychometric properties for each EBP-S measure were estimated, including construct validity using Rasch model, internal consistency reliability using person separation index (PSI), test-retest reliability using intraclass correlation coefficient (ICC), and differential item functioning (DIF).

RESULTS

Two formative measures (use of EBP and EBP activities) were only linguistically modified for use with students. A Rasch model was applied to the other four reflective measures. For knowledge, 55% (6/11) items fit the Rasch model with chi-square fit statistic (χ2) = 34.46, p = 0.08; PSI = 0.85. For self-efficacy, 89% (8/9) items fit the Rasch model with χ2 = 25.11, p = 0.80; PSI = 0.89. For attitudes, 62% (8/13) items fit the Rasch model with χ2 = 61.49, p = 0.00; PSI = 0.71. For perception of the teaching and assessment of EBP in the curriculum, 62% (8/13) items fit the Rasch model with χ2 = 80.99, p = 0.45; PSI = 0.92. perception of the teaching and assessment of EBP in the curriculum showed DIF in three items. The ICCs ranged between 0.80 and 0.98.

CONCLUSIONS

The EBP-S measures were validated for physiotherapy students, including the testing of psychometric properties, which were not tested in the original studies. Further refinements should be considered for the use of the EBP-S with other groups of students or if changes are applied to the current curriculum.

Occupational Therapy Students' Evidence-Based Practice Skills as Reported in a Mobile App: Cross-Sectional Study

BACKGROUND

Evidence-based practice (EBP) is an important aspect of the health care education curriculum. EBP involves following the 5 EBP steps: ask, assess, appraise, apply, and audit. These 5 steps reflect the suggested core competencies covered in teaching and learning programs to support future health care professionals applying EBP. When implementing EBP teaching, assessing outcomes by documenting the student's performance and skills is relevant. This can be done using mobile devices.

OBJECTIVE

The aim of this study was to assess occupational therapy students' EBP skills as reported in a mobile app.

METHODS

We applied a cross-sectional design. Descriptive statistics were used to present frequencies, percentages, means, and ranges of data regarding EBP skills found in the EBPsteps app. Associations between students' ability to formulate the Population, Intervention, Comparison, and Outcome/Population, Interest, and Context (PICO/PICo) elements and identifying relevant research evidence were analyzed with the chi-square test.

RESULTS

Of 4 cohorts with 150 students, 119 (79.3%) students used the app and produced 240 critically appraised topics (CATs) in the app. The EBP steps "ask," "assess," and "appraise" were often correctly performed. The clinical question was formulated correctly in 53.3% (128/240) of the CATs, and students identified research evidence in 81.2% (195/240) of the CATs. Critical appraisal checklists were used in 81.2% (195/240) of the CATs, and most of these checklists were assessed as relevant for the type of research evidence identified (165/195, 84.6%). The least frequently correctly reported steps were "apply" and "audit." In 39.6% (95/240) of the CATs, it was reported that research evidence was applied. Only 61% (58/95) of these CATs described how the research was applied to clinical practice. Evaluation of practice changes was reported in 38.8% (93/240) of the CATs. However, details about practice changes were lacking in all these CATs. A positive association was found between correctly reporting the "population" and "interventions/interest" elements of the PICO/PICo and identifying research evidence (P<.001).

CONCLUSIONS

We assessed the students' EBP skills based on how they documented following the EBP steps in the EBPsteps app, and our results showed variations in how well the students mastered the steps. "Apply" and "audit" were the most difficult EBP steps for the students to perform, and this finding has implications and gives directions for further development of the app and educational instruction in EBP. The EBPsteps app is a new and relevant app for students to learn and practice EBP, and it can be used to assess students' EBP skills objectively.

Challenges and future directions in the measurement of evidence-based practice: Qualitative analysis of umbrella review findings

RATIONALE, AIMS AND OBJECTIVES: An important aspect of scholarly discussions about evidence-based practice (EBP) is how EBP is measured. Given the conceptual and empirical developments in the study of EBP over the last 3 decades, there is a need to better understand how to best measure EBP in educational and clinical contexts. The aim of this study was to identify and describe the main challenges, recommendations for practice, and areas of future research in the measurement of EBP across the health professions as reported by systematic reviews (SRs).

METHODS

We conducted a secondary analysis of qualitative data obtained in the context of a previously published umbrella review that aimed to compare SRs on EBP measures. Two reviewers independently extracted excerpts from the results and discussion/conclusion sections of the 10 included SRs that aligned with the three research aims. An iterative six-phase reflexive thematic analysis according to Braun and Clarke was conducted.

RESULTS

Our thematic analysis produced five themes describing the main challenges associated with measuring EBP, four themes outlining main recommendations for practice, and four themes representing areas of future research. Challenges include limited psychometric testing and validity evidence for existing EBP measures; limitations with the self-report format; lack of construct clarity of EBP measures; inability to capture the complexity of the EBP process and outcomes; and the context-specific nature of EBP measures. Reported recommendations for practice include acknowledging the multidimensionality of EBP; adapting EBP measures to the context and re-examining the validity argument; and considering the feasibility and acceptability of measures. Areas of future research included the development of comprehensive, multidimensional EBP measures and the need for expert consensus on the operationalization of EBP.

CONCLUSIONS

This study suggests that existing measures may be insufficient in capturing the multidimensional, contextual and dynamic nature of EBP. There is a need for a clear operationalization of EBP and an improved understanding and application of validity theory.