Teaching critical appraisal skills in health care settings

Teaching evidence-based medicine with electronic databases for preclinical education

To evaluate the efficacy of an educational module on evidence-based medicine (EBM) assisted with electronic medical databases (EMDs) for preclinical education, medical students (n = 111) were matriculated in a program consisted of 16 2-h sessions on EBM plus hands-on experience on EMDs in a problem-based learning-type format. Students were required to make an oral presentation on designated clinical scenarios before and after the sessions, without prior notice, as an indicator of performance. In addition, questionnaires focusing on behavioral changes, awareness, and confidence of mastering EBM were administered before and after the sessions to assess the attitudinal and behavioral impact of the intervention on the participants. We found evidence of better postprogram performance in utilizing EBM-relevant concepts and resources when the enrolled medical students were giving oral presentations. Moreover, the participants reported increased awareness of EBM and, behaviorally, increased utilization of EBM-relevant resources provided by libraries. Also, they reported improvement on appropriately using EBM-relevant resources, and 99% of the participants reported strong confidence in practicing EBM. In conclusion, modules on EBM implemented with EMDs benefitted medical students in scenario-oriented PBL tutorials. Improvements in awareness, behavior, confidence, and performance in mastering EBM were noted.

Portuguese Medical Students' Interest for Science and Research Declines after Freshman Year

The integration of scientific research into medical curricula remains insufficient despite its advantages for medical students' professional development and the advancement of medicine. This study aimed to evaluate the impact of first-year medical course attendance on medical students' attitudes and perceptions towards scientific research and clinical practice, while also assessing the contribution of sociodemographic and academic factors. Two hundred and thirteen medical students self-administrated a questionnaire at the beginning and at the end of the first school year. Their responses were compared and two regression models were calculated to assess factors influencing students' attitudes and perceptions. After freshman year, students displayed significantly lower positive attitudes towards science and research. Their motivation to perform research and to integrate it into the curriculum also decreased, while the importance attributed to research skills for clinical practice increased. Motivation to perform research and negative attitudes were positively and negatively associated with grade point average (GPA), respectively. Female students and those who attended public secondary schools attributed greater importance to communication skills. This study reinforces the need to early develop research skills and positive attitudes in medical students, motivating them to become physician-scientists. Additional follow-up studies may offer further contributions to the integration of research into medical curricula.

Teaching Critical Appraisal Through the Lens of Study Design in Journal Club

Training in endocrinology presents a major challenge: How to both develop a fellow’s medical knowledge and teach that fellow to independently increase that fund of knowledge as a future endocrinologist. Journal clubs hold a traditional role in teaching the most recent modern advances in medicine but frequently struggle to provide trainees with the skills needed to assess the quality of current literature. Using journal clubs to instill the skills needed for critical appraisal overcomes this difficulty and substantially enhances the value of journal clubs to trainees. Here, our journal club is reviewed, in which we teach critical appraisal by focusing on understanding study design as the key to assessing an article’s strengths and weaknesses. This journal club starts the year by reviewing study designs in a structured sequence, with sessions built to foster group participation and involvement of both fellows and faculty. Later, fellows put this learning into practice as they independently assess articles alongside their faculty. Generalizing this practice allows for implementation in other programs, including even those without faculty with a background in research methodology.

Development and pilot testing of a tool to assess evidence-based practice skills among French general practitioners

BACKGROUND

There is currently an absence of valid and relevant instruments to evaluate how Evidence-based Practice (EBP) training improves, beyond knowledge, physicians' skills. Our aim was to develop and test a tool to assess physicians' EBP skills.

METHODS

The tool we developed includes four parts to assess the necessary skills for applying EBP steps: clinical question formulation; literature search; critical appraisal of literature; synthesis and decision making. We evaluated content and face validity, then tested applicability of the tool and whether external observers could reliably use it to assess acquired skills. We estimated Kappa coefficients to measure concordance between raters.

RESULTS

Twelve general practice (GP) residents and eleven GP teachers from the University of Bordeaux, France, were asked to: formulate four clinical questions (diagnostic, prognosis, treatment, and aetiology) from a proposed clinical vignette, find articles or guidelines to answer four relevant provided questions, analyse an original article answering one of these questions, synthesize knowledge from provided synopses, and decide about the four clinical questions. Concordance between two external raters was excellent for their assessment of participants' appraisal of the significance of article results (K = 0.83), and good for assessment of the formulation of a diagnostic question (K = 0.76), PubMed/Medline (K = 0.71) or guideline (K = 0.67) search, and of appraisal of methodological validity of articles (K = 0.68).

CONCLUSIONS

Our tool allows an in-depth analysis of EBP skills, thus could supplement existing instruments focused on knowledge or specific EBP step. The actual usefulness of such tools to improve care and population health remains to be evaluated.

Healthcare professionals' behavior, skills, knowledge and attitudes on evidence-based health practice: a protocol of cross-sectional study

BACKGROUND

In Brazil, as in most countries nowadays, there is a pursuit for healthcare quality improvement and sustainability in public and private systems. Healthcare professionals' perceptions, knowledge and attitudes determine evidence-based practice (EBP), which remain uncertain among Brazilian practitioners. A standardised national instrument whose wide use will identify gaps and flaws in establishing an EBP could contribute to an effective resources allocation from health professionals willing to use an EBP.

OBJECTIVES

To present a study protocol on the development and validation of an instrument to measure Brazilian healthcare professionals' behaviour, skills, self-efficacy, knowledge and attitudes towards EBP.

METHODS

This is a validation study with Brazilian healthcare professionals to develop a valid and reliable questionnaire, including selection of domains and formulation of questions. Construct and content validity will be assess by a panel of experts, with data collection and analysis following a Delphi-like methodology. Further, a pilot survey will be accomplished with a representative sample of different healthcare professionals from all main Brazilian regions. An exploratory factor analysis and a confirmatory factor analysis will be conducted afterwards. The ratio of χ² and df (χ²/df), comparative fit index, goodness of fit index and root mean square error of approximation will be used for assessing the model fit. In addition, the reliability of the instrument will be estimated by test-retest reproducibility and Cronbach's alpha coefficient (α).

ETHICS AND DISSEMINATION

This study has received ethical approval from the Pharmaceutical Sciences Faculty of the São Paulo State University (1.425.808). The use among a wide national sample is expected to promote an extensive view of evidence-based decision-making, identifying the knowledge gaps in this area. Study findings will be circulated to healthcare professionals and scientists in the field through the publication in peer-reviewed journals and conference presentations.

The effect of an evidence-based medicine course on medical student critical thinking

OBJECTIVE

Evidence-based medicine (EBM) was designed to foster student critical thinking (CT) while conveying knowledge to them. Chinese medical schools have adopted EBM to a varying degree but studies to examine its intended effect are few. This study evaluates the effect of an EBM course on Chinese medical student CT.

METHODS

A total of 158 medical students at a Chinese medical school took a seven-category Chinese version of the critical thinking disposition inventory (CTDI-CV) before and after taking an EBM course. Two-tailed, paired t-test measured the course effect.

RESULTS

Undergraduate (BS) students showed a significant mean difference (MD) in confidence (MD = 1.43, P = 0.025), inquisitiveness (MD = 1.23, P = 0.041), and overall score (MD = 3.45, P = 0.000). Graduate (MS) students showed a more significant mean difference in confidence (MD = 2.72, P = 0.006). Moreover, BS student course grade was correlated with truth-seeking (r = 0.214, P = 0.029) and open-mindedness (r = 0.246, P = 0.012) while that of MS student was correlated with systematicity (r = 0.295, P = 0.031) and overall score (r = 0.290, P = 0.033).

CONCLUSIONS

The results indicate some positive effect of an EBM course on student CT due to embedded CT elements in delivering the content. The discrepancy between a higher admission test score and weaker confidence shown by MS students reveals a drawback of the test centered Chinese education system.

From dental science to clinical practice: Knowledge translation and evidence-based dentistry principles

It has been claimed that in order to decrease the gap between what we know and what we do, research findings must be translated from knowledge to action. Such practices better enable dentists to make evidence-based decisions instead of personal ideas and judgments. To this end, this literature review aims to revisit the concepts of knowledge translation and evidence-based dentistry (EBD) and depict their role and influence within dental education. It addresses some possible strategies to facilitate knowledge translation (KT), encourage dental students to use EBD principles, and to encourage dental educators to create an environment in which students become self-directed learners. It concludes with a call to develop up-to-date and efficient online platforms that could grant dentists better access to EBD sources in order to more efficiently translate research evidence into the clinic.

The International College of Neuropsychopharmacology (CINP) Treatment Guidelines for Bipolar Disorder in Adults (CINP-BD-2017), Part 1: Background and Methods of the Development of Guidelines

BACKGROUND

This paper includes a short description of the important clinical aspects of Bipolar Disorder with emphasis on issues that are important for the therapeutic considerations, including mixed and psychotic features, predominant polarity, and rapid cycling as well as comorbidity.

METHODS

The workgroup performed a review and critical analysis of the literature concerning grading methods and methods for the development of guidelines.

RESULTS

The workgroup arrived at a consensus to base the development of the guideline on randomized controlled trials and related meta-analyses alone in order to follow a strict evidence-based approach. A critical analysis of the existing methods for the grading of treatment options was followed by the development of a new grading method to arrive at efficacy and recommendation levels after the analysis of 32 distinct scenarios of available data for a given treatment option.

CONCLUSION

The current paper reports details on the design, method, and process for the development of CINP guidelines for the treatment of Bipolar Disorder. The rationale and the method with which all data and opinions are combined in order to produce an evidence-based operationalized but also user-friendly guideline and a specific algorithm are described in detail in this paper.

Development of a critical appraisal tool to assess the quality of cross-sectional studies (AXIS)

OBJECTIVES

The aim of this study was to develop a critical appraisal (CA) tool that addressed study design and reporting quality as well as the risk of bias in cross-sectional studies (CSSs). In addition, the aim was to produce a help document to guide the non-expert user through the tool.

DESIGN

An initial scoping review of the published literature and key epidemiological texts was undertaken prior to the formation of a Delphi panel to establish key components for a CA tool for CSSs. A consensus of 80% was required from the Delphi panel for any component to be included in the final tool.

RESULTS

An initial list of 39 components was identified through examination of existing resources. An international Delphi panel of 18 medical and veterinary experts was established. After 3 rounds of the Delphi process, the Appraisal tool for Cross-Sectional Studies (AXIS tool) was developed by consensus and consisted of 20 components. A detailed explanatory document was also developed with the tool, giving expanded explanation of each question and providing simple interpretations and examples of the epidemiological concepts being examined in each question to aid non-expert users.

CONCLUSIONS

CA of the literature is a vital step in evidence synthesis and therefore evidence-based decision-making in a number of different disciplines. The AXIS tool is therefore unique and was developed in a way that it can be used across disciplines to aid the inclusion of CSSs in systematic reviews, guidelines and clinical decision-making.

Evidence-based Medicine versus the Conventional Approach to Journal Club Sessions: Which One Is More Successful in Teaching Critical Appraisal Skills?

This study aimed to compare evidence-based medicine (EBM) vs. conventional approaches to journal club sessions in teaching critical appraisal skills in reading papers by emergency medicine residents. This double cut off discontinuation regression quasi-experimental study was conducted among emergency medicine residents. EBM vs. the conventional approach were applied to teach critical appraisal skills for half of the residents as an experimental group and another half as a control group respectively. Both groups participated in one hour monthly journal club sessions for six months. Before and after the study, all participants were examined by two tests: the Fresno Test (FT) [to evaluate their knowledge about EBM] and the Critical Appraisal Skills Test (CAST) [to evaluate their competency with critical appraisal skills]. The allocation of the participants into the experimental or control groups was according to their CAST scores before the study. 50 emergency medicine residents participated. After the study, the scores of both groups in the FT and CAST significantly improved (p<0.01), and the promotion of scores of the FT and CAST in the experimental group were more than that of the conventional group (p<0.0001). The current study indicated that an evidence-based medicine approach in journal club sessions was comparatively more advantageous compared to the conventional approach in teaching critical appraisal skills for reading papers among the residents of emergency medicine.

Attitudes of Saudi Arabian Undergraduate Medical Students towards Health Research

OBJECTIVES

This study aimed to evaluate attitudes, perceptions and perceived barriers towards health research among Saudi Arabian undergraduate medical students.

METHODS

This cross-sectional study took place between August and October 2014 and included 520 students from five medical schools across Saudi Arabia. An anonymous online survey with 21 close-ended questions was designed to assess students' attitudes towards research, contribution to research-related activities, awareness of the importance of research, perception of available resources/opportunities for research, appreciation of medical students' research contributions and perceived barriers to research. Responses were scored on a 5-point Likert scale.

RESULTS

A total of 401 students participated in the study (response rate: 77.1%). Of these, 278 (69.3%) were female. A positive attitude towards research was reported by 43.9% of the students. No statistically significant differences were observed between genders with regards to attitudes towards and available resources for research (P = 0.500 and 0.200, respectively). Clinical students had a significantly more positive attitude towards research compared to preclinical students (P = 0.007). Only 26.4% of the respondents believed that they had adequate resources/opportunities for research. According to the students, perceived barriers to undertaking research included time constraints (n = 200; 49.9%), lack of research mentors (n = 95; 23.7%), lack of formal research methodology training (n = 170; 42.4%) and difficulties in conducting literature searches (n = 145; 36.2%).

CONCLUSION

Less than half of the surveyed Saudi Arabian medical students had a positive attitude towards health research. Medical education policies should aim to counteract the barriers identified in this study.

Healthcare Databases in Thailand and Japan: Potential Sources for Health Technology Assessment Research

Background

Health technology assessment (HTA) has been continuously used for value-based healthcare decisions over the last decade. Healthcare databases represent an important source of information for HTA, which has seen a surge in use in Western countries. Although HTA agencies have been established in Asia-Pacific region, application and understanding of healthcare databases for HTA is rather limited. Thus, we reviewed existing databases to assess their potential for HTA in Thailand where HTA has been used officially and Japan where HTA is going to be officially introduced.

Method

Existing healthcare databases in Thailand and Japan were compiled and reviewed. Databases’ characteristics e.g. name of database, host, scope/objective, time/sample size, design, data collection method, population/sample, and variables were described. Databases were assessed for its potential HTA use in terms of safety/efficacy/effectiveness, social/ethical, organization/professional, economic, and epidemiological domains. Request route for each database was also provided.

Results

Forty databases– 20 from Thailand and 20 from Japan—were included. These comprised of national censuses, surveys, registries, administrative data, and claimed databases. All databases were potentially used for epidemiological studies. In addition, data on mortality, morbidity, disability, adverse events, quality of life, service/technology utilization, length of stay, and economics were also found in some databases. However, access to patient-level data was limited since information about the databases was not available on public sources.

Conclusion

Our findings have shown that existing databases provided valuable information for HTA research with limitation on accessibility. Mutual dialogue on healthcare database development and usage for HTA among Asia-Pacific region is needed.

Association between participation and compliance with Continuing Medical Education and care production by physicians: a cross-sectional study

OBJECTIVE

Physician participation in Continuing Medical Education programs may be influenced by a number of factors. To evaluate the factors associated with compliance with the Continuing Medical Education requirements at a private hospital, we investigated whether physicians' activity, measured by volumes of admissions and procedures, was associated with obtaining 40 Continuing Medical Education credits (40 hours of activities) in a 12-month cycle.

METHODS

In an exclusive and non-mandatory Continuing Medical Education program, we collected physicians' numbers of hospital admissions and numbers of surgical procedures performed. We also analyzed data on physicians' time since graduation, age, and gender.

RESULTS

A total of 3,809 credentialed, free-standing, private practice physicians were evaluated. Univariate analysis showed that the Continuing Medical Education requirements were more likely to be achieved by male physicians (odds ratio 1.251; p=0.009) and who had a higher number of hospital admissions (odds ratio 1.022; p<0.001). Multivariate analysis showed that age and number of hospital admissions were associated with achievement of the Continuing Medical Education requirements. Each hospital admission increased the chance of achieving the requirements by 0.4%. Among physicians who performed surgical procedures, multivariate analysis showed that male physicians were 1.3 time more likely to achieve the Continuing Medical Education requirements than female physicians. Each surgical procedure performed increased the chance of achieving the requirements by 1.4%.

CONCLUSION

The numbers of admissions and number of surgical procedures performed by physicians at our hospital were associated with the likelihood of meeting the Continuing Medical Education requirements. These findings help to shed new light on our Continuing Medical Education program.

Comparison of an interactive with a didactic educational intervention for improving the evidence-based practice knowledge of occupational therapists in the public health sector in South Africa: a randomised controlled trial

Background

Despite efforts to identify effective interventions to implement evidence-based practice (EBP), uncertainty remains. Few existing studies involve occupational therapists or resource-constrained contexts. This study aimed to determine whether an interactive educational intervention (IE) was more effective than a didactic educational intervention (DE) in improving EBP knowledge, attitudes and behaviour at 12 weeks.

Methods

A matched pairs design, randomised controlled trial was conducted in the Western Cape of South Africa. Occupational therapists employed by the Department of Health were randomised using matched-pair stratification by type (clinician or manager) and knowledge score. Allocation to an IE or a DE was by coin-tossing. A self-report questionnaire (measuring objective knowledge and subjective attitudes) and audit checklist (measuring objective behaviour) were completed at baseline and 12 weeks. The primary outcome was EBP knowledge at 12 weeks while secondary outcomes were attitudes and behaviour at 12 weeks. Data collection occurred at participants’ places of employment. Audit raters were blinded, but participants and the provider could not be blinded.

Results

Twenty-one of 28 pairs reported outcomes, but due to incomplete data for two participants, 19 pairs were included in the analysis. There was a median increase of 1.0 points (95% CI = -4.0, 1.0) in the IE for the primary outcome (knowledge) compared with the DE, but this difference was not significant (P = 0.098). There were no significant differences on any of the attitude subscale scores. The median 12-week audit score was 8.6 points higher in the IE (95% CI = -7.7, 27.0) but this was not significant (P = 0.196). Within-group analyses showed significant increases in knowledge in both groups (IE: T = 4.0, P <0.001; DE: T = 12.0, P = 0.002) but no significant differences in attitudes or behaviour.

Conclusions

The results suggest that the interventions had similar outcomes at 12 weeks and that the interactive component had little additional effect.

Trial registration

Pan African Controlled Trials Register PACTR201201000346141, registered 31 January 2012. Clinical Trials NCT01512823, registered 1 February 2012. South African National Clinical Trial Register DOH2710093067, registered 27 October 2009. The first participants were randomly assigned on 16 July 2008.

The Adapted Fresno test for speech pathologists, social workers, and dieticians/nutritionists: validation and reliability testing

Purpose

The current versions of the Adapted Fresno test (AFT) are limited to physiotherapists and occupational therapists, and new scenarios and scoring rubrics are required for other allied health disciplines. The aim of this study was to examine the validity, reliability, and internal consistency of the AFT developed for speech pathologists (SPs), social workers (SWs), and dieticians/nutritionists (DNs).

Materials and methods

An expert panel from each discipline was formed to content-validate the AFT. A draft instrument, including clinical scenarios, questionnaire, and scoring rubric, was developed. The new versions were completed by ten SPs, 16 SWs, and 12 DNs, and scored by four raters. Interrater reliability was calculated using intraclass correlation coefficients (2,1) for the individual AFT items and the total score. The internal consistency of the AFT was examined using Cronbach’s α.

Results

Two new clinical scenarios and a revised scoring rubric were developed for each discipline. The reliability among raters was excellent for questions 1, 3, and 6 across all disciplines. Question 7 showed excellent reliability for SPs, but not for SWs and DNs. All other reliability coefficients increased to moderate or excellent levels following training. Cronbach’s α was 0.71 for SPs, 0.68 for SWs, and 0.74 for DNs, indicating that internal consistency was acceptable for all disciplines.

Conclusion

There is preliminary evidence to show that AFT is a valid and reliable tool for the assessment of evidence-based practice knowledge and skills of SPs, SWs, and DNs. Further research is required to establish its sensitivity to detect change in knowledge and skills following an educational program.

What are the effects of teaching evidence-based health care (EBHC)? Overview of systematic reviews

Background

An evidence-based approach to health care is recognized internationally as a key competency for healthcare practitioners. This overview systematically evaluated and organized evidence from systematic reviews on teaching evidence-based health care (EBHC).

Methods/Findings

We searched for systematic reviews evaluating interventions for teaching EBHC to health professionals compared to no intervention or different strategies. Outcomes covered EBHC knowledge, skills, attitudes, practices and health outcomes. Comprehensive searches were conducted in April 2013. Two reviewers independently selected eligible reviews, extracted data and evaluated methodological quality. We included 16 systematic reviews, published between 1993 and 2013. There was considerable overlap across reviews. We found that 171 source studies included in the reviews related to 81 separate studies, of which 37 are in more than one review. Studies used various methodologies to evaluate educational interventions of varying content, format and duration in undergraduates, interns, residents and practicing health professionals. The evidence in the reviews showed that multifaceted, clinically integrated interventions, with assessment, led to improvements in knowledge, skills and attitudes. Interventions improved critical appraisal skills and integration of results into decisions, and improved knowledge, skills, attitudes and behaviour amongst practicing health professionals. Considering single interventions, EBHC knowledge and attitude were similar for lecture-based versus online teaching. Journal clubs appeared to increase clinical epidemiology and biostatistics knowledge and reading behavior, but not appraisal skills. EBHC courses improved appraisal skills and knowledge. Amongst practicing health professionals, interactive online courses with guided critical appraisal showed significant increase in knowledge and appraisal skills. A short workshop using problem-based approaches, compared to no intervention, increased knowledge but not appraisal skills.

Conclusions

EBHC teaching and learning strategies should focus on implementing multifaceted, clinically integrated approaches with assessment. Future rigorous research should evaluate minimum components for multifaceted interventions, assessment of medium to long-term outcomes, and implementation of these interventions.

The cycle of bias in health research: a framework and toolbox for critical appraisal training

Recognizing bias in health research is crucial for evidence-based decision making. We worked with eight community groups to develop materials for nine modular, individualized critical appraisal workshops we conducted with 102 consumers (four workshops), 43 healthcare providers (three workshops), and 33 journalists (two workshops) in California. We presented workshops using a "cycle of bias" framework, and developed a toolbox of presentations, problem-based small group sessions, and skill-building materials to improve participants' ability to evaluate research for financial and other conflicts of interest, bias, validity, and applicability. Participant feedback indicated that the adaptability of the toolbox and our focus on bias were critical elements in the success of our workshops.

Healthcare reform: implications for knowledge translation in primary care

Background

The primary care sector represents the linchpin of many health systems. However, the translation of evidence-based practices into patient care can be difficult, particularly during healthcare reform. This can have significant implications for patients, their communities, and the public purse. This is aptly demonstrated in the area of sexual health. The aim of this paper is to determine what works to facilitate evidence-based sexual healthcare within the primary care sector.

Methods

431 clinicians (214 general practitioners and 217 practice nurses) in New South Wales, Australia, were surveyed about their awareness, their use, the perceived impact, and the factors that hindered the use of six resources to promote sexual healthcare. Descriptive statistics were calculated from the responses to the closed survey items, while responses to open-ended item were thematically analyzed.

Results

All six resources were reported to improve the delivery of evidence-based sexual healthcare. Two resources – both double-sided A4-placards – had the greatest reach and use. Barriers that hindered resource-use included limited time, limited perceived need, and limited access to, or familiarity with the resources. Furthermore, the reorganization of the primary care sector and the removal of particular medical benefits scheme items may have hampered clinician capacity to translate evidence-based practices into patient care.

Conclusions

Findings reveal: (1) the translation of evidence-based practices into patient care is viable despite reform; (2) the potential value of a multi-modal approach; (3) the dissemination of relatively inexpensive resources might influence clinical practices; and (4) reforms to governance and/or funding arrangements may widen the void between evidence-based practices and patient care.

Role of Evidence-Based Reviews in Surgery in teaching critical appraisal skills and in journal clubs

BACKGROUND

Evidence-Based Reviews in Surgery (EBRS) is a program developed to teach critical appraisal skills to general surgeons and residents. The purpose of this study was to assess the use of EBRS by general surgery residents across Canada and to assess residents' opinions regarding EBRS and journal clubs.

METHODS

We surveyed postgraduate year 2-5 residents from 15 general surgery programs. Data are presented as percentages and means.

RESULTS

A total of 231 residents (58%, mean 56% per program, range 0%-100%) responded: 172 (75%) residents indicated that they know about EBRS and that it is used in their programs. More than 75% of residents who use EBRS agreed or strongly agreed that the EBRS clinical and methodological articles and reviews are relevant. Only 55 residents (24%) indicated that they used EBRS online. Most residents (198 [86%]) attend journal clubs. The most common format is a mandatory meeting held at a special time every month with faculty members with epidemiological and clinical expertise. Residents stated that EBRS articles were used exclusively (13%) or in conjunction with other articles (57%) in their journal clubs. Most respondents (176 of 193 [91%]) stated that journal clubs are very or somewhat valuable to their education.

CONCLUSION

The EBRS program is widely used among general surgery residents across Canada. Although most residents who use EBRS rate it highly, a large proportion are unaware of EBRS online features. Thus, future efforts to increase awareness of EBRS online features and increase its accessibility are required.

Evaluation of a national evidence-based health care course via teleconference in a developing country

RATIONALE, AIMS AND OBJECTIVES

Continuing health education is essential but challenged. In 2006, the Brazilian Cochrane Center, in collaboration with the Ministry of Health, launched a mass teaching initiative in evidence-based health care (EBH) for public-sector professionals via teleconferencing. This 152-hour, interactive EBH course has enrolled over 4500 professionals. This study aimed to assess the acquisition EBH knowledge and skills, as well as the attitudes and perceptions of a sample of students enrolled in the 2009 course via teleconferencing.

METHODS

This prospective cohort study analyzed three aspects of this 152-hour EBH course that recruited 1040 volunteer participants, all public health sector employees working in 131 different hospitals or health agencies. Pre- and post-course tests using a modified version of the Berlin questionnaire with 20 multiple-choice questions were used to examine knowledge acquisition in a sample of 297 students. Tests were completed upon registration and at course completion. The research projects submitted by 872 participants were evaluated to assess skill acquisition. Answers to an anonymous survey assessed the attitudes and perceptions of 914 participants.

RESULTS

There was a significant increase in knowledge from baseline to course completion (mean scores 8.2 ± 3.3 versus 13.7 ± 3.0, P < 0.001). Over 90% of the research projects were judged to be of adequate quality (appropriate rationale for the study, well-formulated research question and feasible execution); over 95% of the participants were satisfied with the course.

CONCLUSION

The Brazilian EBH course via teleconference improved the knowledge and skills of public-sector health professionals and was approved by the vast majority of students.

Evidence-based medicine at the intersection of research interests between academic health sciences librarians and medical educators: a review of the literature

OBJECTIVES

In 2008, the Association of Academic Health Sciences Libraries established an Education Research Task Force (ERTF) to plan research addressing research priorities outlined in key Association of American Medical Colleges reports. ERTF members conducted a literature review to describe the state of collaborative research at the intersection of medical education and health sciences librarianship. Analysis of initial results revealed instruction in evidence-based medicine (EBM) was a shared interest and is thus the focus of this review.

METHODS

Searches on EBM teaching programs were conducted, and results were posted to a shared online citation management service. Individual articles were assessed and assigned metadata describing subject matter, scope, and format.

RESULTS

Article analysis identified key themes. Most papers were descriptive narratives of curricular development. Evaluation studies were also prominent and often based on student satisfaction or self-reported competency. A smaller number of controlled studies provide evidence of impacts of librarian involvement in EBM instruction.

CONCLUSIONS

Scholarship of EBM instruction is of common interest between medical educators and health sciences librarians. Coauthorship between the groups and distribution of literature points to a productive collaboration. An emerging literature of controlled studies measuring the impact of cross-disciplinary efforts signals continued progress in the arena of EBM instruction.

Validation of an Evidence-Based Medicine Critically Appraised Topic Presentation Evaluation Tool (EBM C-PET)

BACKGROUND

Competence in evidence-based medicine (EBM) is an important clinical skill. Pediatrics residents are expected to acquire competence in EBM during their education, yet few validated tools exist to assess residents' EBM skills.

OBJECTIVE

We sought to develop a reliable tool to evaluate residents' EBM skills in the critical appraisal of a research article, the development of a written EBM critically appraised topic (CAT) synopsis, and a presentation of the findings to colleagues.

METHODS

Instrument development used a modified Delphi technique. We defined the skills to be assessed while reviewing (1) a written CAT synopsis and (2) a resident's EBM presentation. We defined skill levels for each item using the Dreyfus and Dreyfus model of skill development and created behavioral anchors using a frame-of-reference training technique to describe performance for each skill level. We evaluated the assessment instrument's psychometric properties, including internal consistency and interrater reliability.

RESULTS

The EBM Critically Appraised Topic Presentation Evaluation Tool (EBM C-PET) is composed of 14 items that assess residents' EBM and global presentation skills. Resident presentations (N  =  27) and the corresponding written CAT synopses were evaluated using the EBM C-PET. The EBM C-PET had excellent internal consistency (Cronbach α  =  0.94). Intraclass correlation coefficients were used to assess interrater reliability. Intraclass correlation coefficients for individual items ranged from 0.31 to 0.74; the average intraclass correlation coefficients for the 14 items was 0.67.

CONCLUSIONS

We identified essential components of an assessment tool for an EBM CAT synopsis and presentation with excellent internal consistency and a good level of interrater reliability across 3 different institutions. The EBM C-PET is a reliable tool to document resident competence in higher-level EBM skills.

Critical appraisal training increases understanding and confidence and enhances the use of evidence in diverse categories of learners

BACKGROUND

Training in evidence-based medicine is most commonly offered to physicians, medical students and health-care decision-makers.

SETTING AND PARTICIPANTS

We partnered with community organizations to recruit participants and develop trainings for consumers, non-physician health-care providers and journalists in California.

INTERVENTION

We conducted half-day and one-day workshops in critical appraisal of health evidence. Workshops consisted of didactic presentations, small-group practice sessions and class discussions.

OUTCOME MEASURES

We measured knowledge and confidence immediately before and after the workshops and at follow-up 6 months later. We also asked participants to describe their use of health evidence before the workshops and at follow-up.

RESULTS

At baseline, 41% of the consumers, 45% of the providers and 57% of the journalists correctly answered questions about health evidence. Scores increased by about 20% (P < 0.05) in all groups at the end of the workshops and remained significantly over baseline at follow-up. At baseline, 26% of the participants were confident in their understanding of critical appraisal concepts, significantly increasing to 54% after the workshops and sustained (53%) at follow-up. During discussions, participants' comments often focused on funding and the potential effects of financial conflicts of interest on study findings. Participants did not use evidence more frequently at follow-up but said that they applied workshop skills in evaluating research, communicating with others and making decisions about health care.

CONCLUSION

It is possible to successfully conduct critical appraisal workshops to aid health-related decision making for groups who have previously not had access to this kind of training.

Clinical decision making-a functional medicine perspective

As 21st century health care moves from a disease-based approach to a more patient-centric system that can address biochemical individuality to improve health and function, clinical decision making becomes more complex. Accentuating the problem is the lack of a clear standard for this more complex functional medicine approach. While there is relatively broad agreement in Western medicine for what constitutes competent assessment of disease and identification of related treatment approaches, the complex functional medicine model posits multiple and individualized diagnostic and therapeutic approaches, most or many of which have reasonable underlying science and principles, but which have not been rigorously tested in a research or clinical setting. This has led to non-rigorous thinking and sometimes to uncritical acceptance of both poorly documented diagnostic procedures and ineffective therapies, resulting in less than optimal clinical care.

Safety and efficacy of a fully closed-loop control ventilation (IntelliVent-ASV®) in sedated ICU patients with acute respiratory failure: a prospective randomized crossover study

PURPOSE

IntelliVent-ASV(®) is a development of adaptive support ventilation (ASV) that automatically adjusts ventilation and oxygenation parameters. This study assessed the safety and efficacy of IntelliVent-ASV(®) in sedated intensive care unit (ICU) patients with acute respiratory failure.

METHODS

This prospective randomized crossover comparative study was conducted in a 12-bed ICU in a general hospital. Two periods of 2 h of ventilation in randomly applied ASV or IntelliVent-ASV(®) were compared in 50 sedated, passively ventilated patients. Tidal volume (V(T)), respiratory rate (RR), inspiratory pressure (P(INSP)), SpO(2) and E(T)CO(2) were continuously monitored and recorded breath by breath. Mean values over the 2-h period were calculated. Respiratory mechanics, plateau pressure (P(PLAT)) and blood gas exchanges were measured at the end of each period.

RESULTS

There was no safety issue requiring premature interruption of IntelliVent-ASV(®). Minute ventilation (MV) and V(T) decreased from 7.6 (6.5-9.5) to 6.8 (6.0-8.0) L/min (p < 0.001) and from 8.3 (7.8-9.0) to 8.1 (7.7-8.6) mL/kg PBW (p = 0.003) during IntelliVent-ASV(®) as compared to ASV. P(PLAT) and FiO(2) decreased from 24 (20-29) to 20 (19-25) cmH(2)O (p = 0.005) and from 40 (30-50) to 30 (30-39) % (p < 0.001) during IntelliVent-ASV(®) as compared to ASV. RR, P(INSP), and PEEP decreased as well during IntelliVent-ASV(®) as compared to ASV. Respiratory mechanics, pH, PaO(2) and PaO(2)/FiO(2) ratio were not different but PaCO(2) was slightly higher during IntelliVent-ASV(®) as compared to ASV.

CONCLUSIONS

In passive patients with acute respiratory failure, IntelliVent-ASV(®) was safe and able to ventilate patients with less pressure, volume and FiO(2) while producing the same results in terms of oxygenation.

A systematic review of the individual determinants of research evidence use in allied health

Background:

The use of evidence-based practice (EBP) is often not reflected in allied health (AH) practitioners’ day-to-day practice (the research-practice gap). Research suggests that considerable differences between and within AH disciplines exist, which require different approaches in order to influence practice behavior. It is therefore important to develop a better understanding of what influences individual AH practitioners’ adoption of evidence into daily practice.

Objective:

This systematic review aims to examine the individual characteristics of AH practitioners which determine their uptake of evidence into practice.

Methods:

Studies which examined individual factors or variables that influence research evidence use by any AH practitioner were included in the review. The methodological quality of the included papers was assessed using the Quality Assessment and Validity Tool for Cross-sectional Studies. A narrative summary of the findings was presented.

Results:

Six studies were included and the methodological quality scores indicated that two were weak and the remainder had moderate–weak quality. The review demonstrated that factors such as educational degree or academic qualification, involvement in research or EBP-related activities, and practitioners’ perceptions, attitudes and beliefs about research and EBP are significant predictors of self-reported research evidence use in AH. The effect of other factors such as professional characteristics, clinical setting/work environment, information-seeking behavior and sociodemographic variables are less clear. Whether there is an interaction effect between evidence-uptake factors has not been tested.

Conclusion:

Improving the research knowledge of clinicians and overcoming negative attitudes toward EBP have the potential to move AH practitioners towards regularly utilizing evidence in practice. Allied health practitioners may benefit from participation in regular educational opportunities such as case studies or journal clubs which can put them at the same level of thinking and awareness of research evidence. Future research should aim to review organizational and contextual factors and explore their interaction with individual determinants of research evidence use.

Knowledge translation in audiology: promoting the clinical application of best evidence

The impetus for evidence-based practice (EBP) has grown out of widespread concern with the quality, effectiveness (including cost-effectiveness), and efficiency of medical care received by the public. Although initially focused on medicine, EBP principles have been adopted by many of the health care professions and are often represented in practice through the development and use of clinical practice guidelines (CPGs). Audiology has been working on incorporating EBP principles into its mandate for professional practice since the mid-1990s. Despite widespread efforts to implement EBP and guidelines into audiology practice, gaps still exist between the best evidence based on research and what is being done in clinical practice. A collaborative dynamic and iterative integrated knowledge translation (KT) framework rather than a researcher-driven hierarchical approach to EBP and the development of CPGs has been shown to reduce the knowledge-to-clinical action gaps. This article provides a brief overview of EBP and CPGs, including a discussion of the barriers to implementing CPGs into clinical practice. It then offers a discussion of how an integrated KT process combined with a community of practice (CoP) might facilitate the development and dissemination of evidence for clinical audiology practice. Finally, a project that uses the knowledge-to-action (KTA) framework for the development of outcome measures in pediatric audiology is introduced.

Closing the evidence-practice gap for newborn pain using clinical networks

AIM

The Australian New Zealand Neonatal Network undertook a project to close the evidence practice gap for the management of newborn pain in neonatal units within Australia. The aim was to establish a process for using evidence to support practice change and in doing so close the existing practice evidence gap for newborn pain.

METHOD

An implementation model using a clinical network with state facilitators, local champions and project teams was used in 24 tertiary units and six district hospitals throughout Australia. The process included audit and feedback, benchmarking, educational workshops on critical appraisal and audit of family awareness of pain. Multiple types of data were collected to enable changes in practices for pain management to be evaluated.

RESULTS

There was a 21% overall improvement in the number of infants receiving breastfeeding or sucrose for procedural pain, however, breastfeeding rates remain poorly utilised. The use of a pain assessment tool increased from 14% to 22%, with 56% of units introducing the use of an assessment tool into their practice. Families became more aware that their infant can experience pain and strategies to manage the pain (from 19% to 57%) project through distribution of information.

CONCLUSION

The networks for the uptake of evidence program have enabled the topic of procedure pain in neonates to be evaluated. We have shown that through a well planned program of coordination, facilitation and using local champions and project teams a change in practice can occur.

Undergraduate medical student perceptions and use of Evidence Based Medicine: a qualitative study

BACKGROUND

Many medical schools teach the principles of Evidence Based Medicine (EBM) as a subject within their medical curriculum. Few studies have explored the barriers and enablers that students experience when studying medicine and attempting to integrate EBM in their clinical experience. The aim of this study was to identify undergraduate medical student perceptions of EBM, including their current use of its principles as students and perceived future use as clinicians.

METHODS

Third year medical students were recruited via email to participate in focus group discussions. Four focus groups were conducted separately across four hospital sites. All focus groups were conducted by the same facilitator. All discussions were transcribed verbatim, and analysed independently by the two authors according to the principles of thematic analysis.

RESULTS

Focus group discussions were conducted with 23 third-year medical students, representing three metropolitan and one rural hospital sites. Five key themes emerged from the analysis of the transcripts: (1) Rationale and observed use of EBM in practice, (2) Current use of EBM as students, (3) Perceived use of EBM as future clinicians, (4) Barriers to practicing EBM, and (5) Enablers to facilitate the integration of EBM into clinical practice. Key facilitators for promoting EBM to students include competency in EBM, mentorship and application to clinical disciplines. Barriers to EBM implementation include lack of visible application by senior clinicians and constraints by poor resourcing.

CONCLUSIONS

The principles and application of EBM is perceived by medical students to be important in both their current clinical training and perceived future work as clinicians. Future research is needed to identify how medical students incorporate EBM concepts into their clinical practice as they gain greater clinical exposure and competence.

Interventions to improve question formulation in professional practice and self-directed learning

BACKGROUND

Formulating questions is fundamental to the daily life of a healthcare worker and is a defining characteristic of professional competence and meaningful learning. With high expectations for healthcare providers to remain up-to-date with current evidence and the movement towards formalizing reflective practice as part of physician revalidation, it is important that curricula developed for improving the ability to formulate well-constructed questions are based on the best evidence.

OBJECTIVES

To assess the effectiveness of interventions for increasing the frequency and quality of questions formulated by healthcare providers in practice and the context of self-directed learning.

SEARCH STRATEGY

We obtained studies from searches of electronic bibliographic databases, and supplemented these with handsearching, checking reference lists, and consultation with experts.

SELECTION CRITERIA

We considered published and unpublished randomized controlled trials (RCTs), controlled clinical trials (CCTs), interrupted time-series (ITS), and controlled before-after (CBA) studies of any language examining interventions for increasing the quality and frequency of questions formulated by health professionals involved with direct patient care.

DATA COLLECTION AND ANALYSIS

Two review authors independently undertook all relevancy screening and 'Risk of bias' assessment in duplicate. Intervention characteristics, follow-up intervals, and measurement outcomes were diverse and precluded quantitative analysis. We have summarized data descriptively.

MAIN RESULTS

Searches identified four studies examining interventions to improve question formulation in healthcare professionals. Interventions were mostly multi-component, limited within the context of EBM and primarily in physician and resident populations. We did not identify studies examining changes in frequency of questions formulated or those within the context of reflection. Risk of bias was generally rated to be 'high risk'. Three of the four studies showed improvements in question formulation in physicians, residents, or mixed allied health populations in the short- to moderate term follow up. Only one study examined sustainability of effects at one year and reported that skills had eroded over time.

AUTHORS' CONCLUSIONS

Evidence from our review suggests that interventions to increase the quality of questions formulated in practice produce mixed results at both short- (immediately following intervention), and moderate-term follow up (up to nine months), comparatively. Although three studies reported effectiveness estimates of an educational intervention for increasing the quality of question formulation within the short term, only one study examined the effectiveness in the longer term (one year) and revealed that search skills had eroded over time. Data suggests that sustainability of effects from educational interventions for question formulation are unknown.

What are the most effective strategies for improving quality and safety of health care?

There is now a plethora of different quality improvement strategies (QIS) for optimizing health care, some clinician/patient driven, others manager/policy-maker driven. Which of these are most effective remains unclear despite expressed concerns about potential for QIS-related patient harm and wasting of resources. The objective of this study was to review published literature assessing the relative effectiveness of different QIS. Data sources comprising PubMed Clinical Queries, Cochrane Library and its Effective Practice and Organization of Care database, and HealthStar were searched for studies of QIS between January 1985 and February 2008 using search terms based on an a priori QIS classification suggested by experts. Systematic reviews of controlled trials were selected in determining effect sizes for specific QIS, which were compared as a narrative meta-review. Clinician/patient driven QIS were associated with stronger evidence of efficacy and larger effect sizes than manager/policy-maker driven QIS. The most effective strategies (>10% absolute increase in appropriate care or equivalent measure) included clinician-directed audit and feedback cycles, clinical decision support systems, specialty outreach programmes, chronic disease management programmes, continuing professional education based on interactive small-group case discussions, and patient-mediated clinician reminders. Pay-for-performance schemes directed to clinician groups and organizational process redesign were modestly effective. Other manager/policy-maker driven QIS including continuous quality improvement programmes, risk and safety management systems, public scorecards and performance reports, external accreditation, and clinical governance arrangements have not been adequately evaluated with regard to effectiveness. QIS are heterogeneous and methodological flaws in much of the evaluative literature limit validity and generalizability of results. Based on current best available evidence, clinician/patient driven QIS appear to be more effective than manager/policy-maker driven QIS although the latter have, in many instances, attracted insufficient robust evaluations to accurately determine their comparative effectiveness.

Determinants of knowledge gain in evidence-based medicine short courses: an international assessment

Background

Health care professionals worldwide attend courses and workshops to learn evidence-based medicine (EBM), but evidence regarding the impact of these educational interventions is conflicting and of low methodologic quality and lacks generalizability. Furthermore, little is known about determinants of success. We sought to measure the effect of EBM short courses and workshops on knowledge and to identify course and learner characteristics associated with knowledge acquisition.

Methods

Health care professionals with varying expertise in EBM participated in an international, multicentre before–after study. The intervention consisted of short courses and workshops on EBM offered in diverse settings, formats and intensities. The primary outcome measure was the score on the Berlin Questionnaire, a validated instrument measuring EBM knowledge that the participants completed before and after the course.

Results

A total of 15 centres participated in the study and 420 learners from North America and Europe completed the study. The baseline score across courses was 7.49 points (range 3.97–10.42 points) out of a possible 15 points. The average increase in score was 1.40 points (95% confidence interval 0.48–2.31 points), which corresponded with an effect size of 0.44 standard deviation units. Greater improvement in scores was associated (in order of greatest to least magnitude) with active participation required of the learners, a separate statistics session, fewer topics, less teaching time, fewer learners per tutor, larger overall course size and smaller group size. Clinicians and learners involved in medical publishing improved their score more than other types of learners; administrators and public health professionals improved their score less. Learners who perceived themselves to have an advanced knowledge of EBM and had prior experience as an EBM tutor also showed greater improvement than those who did not.

Interpretation

EBM course organizers who wish to optimize knowledge gain should require learners to actively participate in the course and should consider focusing on a small number of topics, giving particular attention to statistical concepts.

Evidence Searching for Evidence-based Psychology Practice

There is an increased awareness of evidence-based methodology among psychologists, but little exists in the literature about how to access the research. Moreover, the prohibitive cost of this information combined with limited time are barriers to the identification of evidence to answer clinical questions. This article presents an example of a question worked though in an evidence-based way. Methods are highlighted, including distinguishing background and foreground questions, breaking down questions into searchable statements, and adapting statements to suit both the question being asked and the resource being searched. A number of free, evidence-based resources are listed. Knowing how and where to access this information will enable practitioners to more easily use an evidence-based approach to their practice.

How to teach evidence-based medicine to teachers: reflections from a workshop experience

RATIONALE, AIMS AND OBJECTIVES

To summarize 20-year experience of conducting a workshop designed for educators who wish to improve their teaching skills of evidence based medicine (EBM). The goal is to provide tips for educators interested in replicating this educational model.

METHODS

Qualitative description of factors associated with the success of the workshop.

RESULTS

The factors considered by instructors to be most helpful are: the small group interactive design, role-play and simulation of real world learning environments, a mentorship model and high educator to learner ratio.

CONCLUSIONS

Although this experience is observational and does not represent high quality evidence, certain attributes in the design of EBM workshops may lead to better dissemination of EBM concepts. Educators may consider empirically applying some of these attributes and testing their efficacy in comparative studies.

Attitudes, awareness, and barriers regarding evidence-based surgery among surgeons and surgical nurses

Background

Evidence-based surgery (EBS) is stressed to increase efficiency and health care quality, but not all surgeons and surgical nurses use EBS in clinical practice. To define future tailor-made interventions to improve evidence-based behavior, the aim of this study was to determine the attitude and awareness among surgeons and surgical nurses as to the paradigm of EBS and the barriers experienced when practicing EBS.

Methods

In this cross-sectional study, surgeons and surgical nurses at a university hospital in Amsterdam were invited to complete the BARRIERS scale and McColl et al. questionnaire. An evidence quiz was composed for the surgeons.

Results

Response rates were 67% (29/43) for surgeons and 60% (73/122) for nurses. Attitudes toward EBS were positive. Among the surgeons, 90% were familiar with EBS terms, whereas only 40% of the nurses were. Common barriers for surgeons were conflicting results (79%, 23/29) and the methodologic inadequacy of research reports (73%, 21/29); and for nurses they were unawareness of EBS (67%, 49/73) and unclear reported research (59%, 43/73). Only about half of the convincing evidence presented in the quiz was actually applied.

Conclusions

Surgeons have a positive attitude toward EBS and are familiar with EBS terminology, but conflicting results and methodologic shortcomings of research reports are major barriers to practicing EBS. Continual confrontations with available evidence through frequent critical appraisal meetings or grand rounds and using more aggregate sources of evidence are advocated. Nurses can probably benefit from EBS training focusing on basic skills. Finally, collaboration is needed among surgeons and nurses with the same zest about EBS.

Do workshops in evidence-based practice equip participants to identify and answer questions requiring consideration of clinical research? A diagnostic skill assessment

Evidence-based practice (EBP) requires practitioners to identify and formulate questions in response to patient encounters, and to seek, select, and appraise applicable clinical research. A standardized workshop format serves as the model for training of medical educators in these skills. We developed an evaluation exercise to assess the ability to identify and solve a problem requiring the use of targeted skills and administered it to 47 North American junior faculty and residents in various specialties at the close of two short workshops in EBP. Prior to the workshop, subjects reported prior training in EBP and completed a previously validated knowledge test. Our post-workshop exercise differed from the baseline measures and required participants to spontaneously identify a suitable question in response to a simulated clinical encounter, followed by a description of a stepwise approach to answering it. They then responded to successively more explicitly prompted queries relevant to their question. We analyzed responses to identify areas of skill deficiency and potential reasons for these deficiencies. Twelve respondents (26%) initially failed to identify a suitable question in response to the clinical scenario. Ability to choose a suitable question correlated with the ability to connect an original question to an appropriate study design. Prior EBP training correlated with the pretest score but not with performance on our exercise. Overall performance correlated with ability to correctly classify their questions as pertaining to therapy, diagnosis, prognosis, or harm. We conclude that faculty and residents completing standard workshops in EBP may still lack the ability to initiate and investigate original clinical inquiries using EBP skills.

Prospective, controlled assessment of the impact of formal evidence-based medicine teaching workshop on ability to appraise the medical literature

BACKGROUND

The ability to critically appraise the calibre of studies in medical literature is increasingly important for medical professionals.

AIM

This prospective controlled study evaluated the impact of a 6-h Evidence Based Medicine (EBM) Workshop on the critical appraisal skills of medical trainees.

METHODS

Individuals attended three 2-h workshops over a 3-week period, incorporating didactic lectures in statistics, clinical trial design, appraising research papers and practical examples. Appraisal skills were assessed pre- and post-training based on grading the quality of randomised control studies (level 1 evidence), cohort studies (level 2 evidence) and case-control studies (level 3 evidence) [From Oxford Centre for Evidence Based Medicine Levels of Evidence (2001), http://www.cebm.net/critical_appraisal.asp ].

RESULTS

Overall grading improved from 39% (pre-course) to 74% (post-course), P = 0.002, with grading of levels 1, 2 and 3 studies improving from 42 to 75%, 53 to 61% and 21 to 84%, respectively.

CONCLUSIONS

We conclude that a 6-h formal EBM workshop is effective in enhancing the critical appraisal skills of medical trainees.

Teaching Evidence-based Practice: Perspectives from the Undergraduate and Post-graduate Viewpoint

Introduction: Evidence-based practice (EBP) involves making clinical decisions informed by the most relevant and valid evidence available. It has been suggested that the outcomes of teaching EBP skills may differ between undergraduates and post-graduates due to different determinants in learning. This paper reviews the current literature and discusses the impact of the teaching environment (undergraduate or post-graduate) for teaching EBP and its impact on EBP competency. Methods: A search of the literature was performed across the MEDLINE, CINAHL, PsychInfo and ERIC databases. Randomised controlled trials (RCTs) and non-randomised trials were eligible for inclusion in the paper. Studies were included for review if they explored the impact of teaching on participants’ EBP competency, consisting of critical appraisal skills, knowledge and/or behaviour. Results: Ten articles were eligible for inclusion for this review, of which 7 met all inclusion criteria. EBP competency was shown to increase regardless of whether EBP is delivered to medical students at an undergraduate or post-graduate level. EBP taught to a non-medical undergraduate audience did not modify participants’ EBP competency. No study directly compared teaching EBP to an undergraduate and post-graduate audience. Conclusions: Given the limited amount of studies included in this review, further research incorporating high- level methodologies is required to establish a clear recommendation on the research question. Key words: Competency-based education, Evidence-based practice, Graduate medical educa- tion, Undergraduate medical education

Study protocol of psychometric properties of the Spanish translation of a competence test in evidence based practice: the Fresno test

Background

There are few high-quality instruments for evaluating the effectiveness of Evidence-Based Practice (EBP) curricula with objective outcomes measures. The Fresno test is an instrument that evaluates most of EBP steps with a high reliability and validity in the English original version. The present study has the aims to translate the Fresno questionnaire into Spanish and its subsequent validation to ensure the equivalence of the Spanish version against the English original.

Methods and design

The questionnaire will be translated with the back translation technique and tested in Primary Care Teaching Units in Catalonia (PCTU). Participants will be: (a) tutors of Family Medicine residents (expert group); (b) Family Medicine residents in their second year of the Family Medicine training program (novice group), and (c) Family Medicine physicians (intermediate group). The questionnaire will be administered before and after an educational intervention. The educational intervention will be an interactive four half-day sessions designed to develop the knowledge and skills required to EBP. Responsiveness statistics used in the analysis will be the effect size, the standardised response mean and Guyatt's method. For internal consistency reliability, two measures will be used: corrected item-total correlations and Cronbach's alpha. Inter-rater reliability will be tested using Kappa coefficient for qualitative items and intra-class correlation coefficient for quantitative items and the overall score. Construct validity, item difficulty, item discrimination and feasibility will be determined.

Discussion

The validation of the Fresno questionnaire into different languages will enable the expansion of the questionnaire, as well as allowing comparison between countries and the evaluation of different teaching models.