Techniques of identifying competencies needed of doctors

Using a Modified Delphi Method for Identifying Competencies in a Syrian Undergraduate Neonatology Curriculum

OBJECTIVES

There is a global shift toward competency-based medical education (CBME) to equip medical students with essential competencies required to meet healthcare needs. Syrian medical faculties lack a formal competency-based neonatology curriculum for undergraduate medical students. Therefore, our study aimed to develop a national consensus on the required competencies for undergraduate neonatology curricula in Syria.

METHODS

This study took place at the Syrian Virtual University between October 2021 and November 2021. The authors used a modified Delphi method to determine neonatal medicine competencies. A focus group of three neonatologists and one medical education professional identified initial competencies. In the first Delphi round, 75 pediatric clinicians rated the competencies on a 5-point Likert scale. After formulating the results, a second Delphi round was conducted with 15 neonatal medicine experts. To reach an agreement, 75% of participants must score a competency as 4 or 5. Correlation coefficients were used to compare clinicians' first Delphi ratings with experts' second Delphi ratings. Competencies with a weighted response greater than 4.2 were considered essential.

RESULTS

A list of 37 competencies (22 knowledge, 6 skills, and 9 attitudes) was identified after the second Delphi round, of which 24 were considered core competencies (11 knowledge, 5 skills, and 8 attitudes). The correlation coefficient for knowledge, skills, and attitudes competencies was 0.90, 0.96, and 0.80, respectively.

CONCLUSIONS

Neonatology competencies have been identified for medical undergraduates. These competencies aim to help students achieve the required skills and enable decision-makers to implement CBME in Syria and similar countries.

Intensive Care Unit Critical Incident Analysis as an Objective Tool to Select Content for a Simulation Curriculum

INTRODUCTION

The intensive care unit (ICU) offers a unique environment where emergency events are frequent, high-stakes, and carefully documented, which makes it an ideal setting to research the specific technical skills, which are deployed during such events. This study aimed to describe a method of objectively identifying skills and scenarios, which should be prioritized for inclusion in a simulation curriculum.

METHOD

A retrospective audit of all available critical incident data (11 months) from a 36-bed tertiary ICU was performed. Code blue events were analyzed. Data were coded according to a rubric based on Le Guen and Costa-Pinto ( Intern Med J. 2020;51(8):1298-1303) tallying the occurrence of common ICU scenarios and skills.Documentation of each event was analyzed. The frequency with which a skill or scenario appeared in these events was considered as "high frequency" if it occurred in more than 20% of the events.The trainees' confidence in a particular skill was assessed by means of a self-assessment survey questionnaire (based on an anchored 6-item rating scale).

RESULTS

One hundred twenty-one incidents were analyzed. Sixteen were eliminated because of insufficient documentation. The most common skills during these emergency events were familiarity with the advanced life support trolley (34% of events), electrocardiogram (ECG) rhythm strip interpretation (32.4%), and the operation of an external defibrillator (29.5%).Most trainees surveyed are preparing to undergo training in anesthesia (58%) or intensive care (28%). Specialized areas of expertise (troubleshooting an extra-corporeal membrane oxygenation (ECMO) circuit or intra-aortic balloon pump) had the lowest confidence scores (average scores of 0.81 and 0.72).

CONCLUSIONS

We highlighted a novel, reproducible, and objective methodology by which critical incident data can be integrated with trainee self-assessment to generate a targeted simulation curriculum.

Competency lists for urban general practitioners/family physicians using the modified Delphi method

BACKGROUND

In recent years, the growing global urbanization and urban population have resulted in the emergence of various health problems unique to urban areas. Therefore, training general practitioners and family physicians who can tackle the complex health problems of urban areas and improve the health of urban people is one of the most important issues of our time. However, findings on competencies for urban general practitioners (GP) and family physicians (FP) were limited. This study aimed to identify their comprehensive and content-validated list of competencies.

METHODS

We used the modified Delphi method to develop a content-validated competency list. First, we analyzed and synthesized the competencies extracted from the literature review using qualitative thematic analysis methods to create an initial competency list of 34 items. We then assembled 39 expert panelists in four groups of study participants: physicians, nurses, patients, and medical education specialists. The expert panelists were asked to indicate their level of agreement with the lists and provide revised comments on the description of each competency via a web-based questionnaire. Their responses were analyzed quantitatively and qualitatively by the research team and used to revise the list. These processes were repeated, and the survey was completed when it was determined that consensus had been reached.

RESULTS

Three rounds of Delphi were conducted. 39 responded in the first round, 38 in the second round, and 36 in the third round. The initial list of competencies was revised and consolidated from 34 to 14 items in the first round, bringing the total to 20 items along with six new items proposed by the panelists. In the second round, it was revised and consolidated into a list of 18 items. In the third round, all 18 items were considered to have been agreed upon by the panelists, so the survey was closed.

CONCLUSION

We identified a comprehensive 18-item list of competencies for urban GP/FP in a content-validated manner. Several are newly discovered competencies in this study. The findings of this study will be useful for the future training of urban GP/FP and for solving urban health problems.

Development and Validation of a Performance Assessment Checklist Scale for Vaccine Administration

Objective Vaccination technique is a crucial skill for medical trainees to learn, especially in the current coronavirus disease 2019 pandemic. To this end, validated assessment tools are essential in teaching appropriate techniques. However, valid instruments for assessing vaccine administration skills have not yet been developed. We therefore explored the development and validation of an assessment tool for vaccination techniques based on expert consensus. Methods We implemented a modified Delphi process to develop a vaccination technique assessment tool. We then conducted a validation study to establish the reliability and validity of the tool. Results Two rounds of the modified Delphi process were performed to generate a 19-item, vaccination performance assessment checklist. In the validation study, the linear weighted kappa value for inter-rater reliability of the overall checklist score was 0.725. Spearman's correlation coefficient between the mean checklist score and the global rating was 0.98 (p<0.01). Conclusions This is a pioneering study examining the development and validation of an assessment tool for vaccine administration techniques. The tool will be widely used in vaccination-related education.

Core competencies for a biomedical laboratory scientist - a Delphi study

BACKGROUND

After completing university education, biomedical laboratory scientists work in clinical laboratories, in biomedical research laboratories, in biotech, and in pharmaceutical companies. Laboratory diagnostics have undergone rapid development over the recent years, with the pace showing no signs of abatement. This rapid development challenges the competence of the staff and will most certainly influence the education of future staff. This study aimed to examine what was considered the necessary competencies needed to pursue a career as a biomedical laboratory scientist.

METHODS

A modified Delphi technique was used, with the panel of experts expressing their views in a series of three questionnaire. Consensus was defined as the point which 75 % or more of the panel participants agreed that a particular competency was necessary.

RESULTS

The study highlights the perceived importance of mostly generic competencies that relate to quality, quality assurance, and accuracy, as well as different aspects of safety, respect, trustworthiness (towards patients/clients and colleagues), and communication skills. The results also stress the significance of self-awareness and professionality.

CONCLUSIONS

We identified important competencies for biomedical laboratory scientists. Together with complementary information from other sources, i.e., guidelines, laws, and scientific publications, the competencies identified can be used as learning outcomes in a competency-based education to provide students with all the competencies needed to work as professional biomedical laboratory scientists.

Development of a list of competencies and entrustable professional activities for resident physicians during death pronouncement: a modified Delphi study

BACKGROUND

The appropriate delivery of death pronouncements potentially affects bereaved families' wellbeing positively. Although younger physicians need to learn the competencies and entrustable professional activities (EPAs) to conduct death pronouncement independently, both of which have not been clarified. Therefore, this study aimed to develop a list of competencies and EPAs necessary for death pronouncement practice, which resident physicians need to acquire by the end of their residency training (postgraduate year 2).

METHODS

An anonymous modified Delphi study was conducted with a panel of 31 experts. The experts were invited online from general wards in hospitals with resident physicians across Japan to participate in the study using the purposive and snowball sampling method. A non-anonymous web conference was held with three additional external evaluators to finalize the item list. The consensus criterion was defined as a mean response of at least 4 points on a 5-point Likert scale for each competency and EPA item and a rating of 4 or 5 points by at least 80% of the participants.

RESULTS

Consensus was achieved, with consistently high levels of agreement across panel members, on 11 competencies and 9 EPA items. Additionally, a correspondence matrix table between competencies and EPAs was developed.

CONCLUSIONS

This study clarified the standardized educational outcomes as competencies in death pronouncement practice and the unit of professional practice of physicians who can perform this independently (EPAs), serving as a blueprint to aid the development of an educational model and evaluation method for clinical educational institutions and developers of medical school curriculums.

Methodological approaches for identifying competencies for the physiotherapy profession: a scoping review

Physiotherapy competencies inform the education and regulation of the profession. Many different methods appear to be used to identify competencies and there is no consensus on optimal methods to identify competencies. The purpose of this review is to synthesize the methodological approaches used to identify competencies for the physiotherapy profession and summarize the nature of those competencies. We searched MEDLINE, EMBASE, CINAHL, and the grey literature from inception to June 2020. Two independent reviewers screened for empirical peer-reviewed articles that aimed to identify professional physiotherapy competencies. General study characteristics, competency characteristics (e.g., target practice area), and methodological characteristics (e.g., study population, data collection and analysis method for each methodological step) were extracted. Descriptive statistics and narrative synthesis were performed. Of the 9529 references screened, 38 articles describing 35 studies published between 1980 and 2020 were included. Orthopaedics (20.0%) was the most commonly targeted area of practice. Studies used one to eight methodological steps whose objective was to generate (16 studies), validate (18 studies), assign value (21 studies), refine (10 studies), or triangulate (3 studies) competencies, or to address multiple objectives (10 studies). The most commonly used methods were surveys to assign value (n = 20, 95%), and group techniques to refine competencies (n = 7, 70%). Physiotherapists with experience in the area of competence was the most commonly consulted stakeholder group (80% of studies). This review can provide methodological guidance to stakeholders such as educators and regulators that aim to identify professional competencies in the future.

Supplementary Information

The online version contains supplementary material available at 10.1007/s44217-022-00008-9.

Unknown unknowns: can bringing patients' lived experience of long term conditions into curriculum design help our future doctors offer more holistic, socially accountable care?

Understanding the holistic impact of long-term conditions (LTCs) on a patient, due to their high prevalence, morbidity, mortality and cost, is an important part of the primary care undergraduate medicine curriculum. Increasingly, we set learning outcomes for our students anticipating what the patient would want them to know. But are our expectations congruent with what patients would want them to know? In this leading article, we propose that patient involvement in curriculum design remains inadequate. We base our argument on (1) an existing strong theoretical premise that increasing patient incorporation enhances the delivery of medical education, (2) recommendations from major health organisations, our regulator and leading healthcare educationalists and (3) a growing body of evidence that what is important to physicians may not reflect what is important to patients. We advocate that patient involvement in curriculum design will result in a better understanding of how LTCs affect their physical, psychological and social health and of their journey through health care and community support services. This learning is essential, but unfortunately often overlooked, if we are to ensure that students in their future roles plan patient care using an empathic, holistic, patient-centred and socially accountable approach.

Design, Implementation, and Evaluation of a Distance Learning Framework to Adapt to the Changing Landscape of Anatomy Instruction in Medical Education During COVID-19 Pandemic: A Proof-of-Concept Study

This study presents the design of a DL-framework to deliver anatomy teaching that provides a microfiche of the onsite anatomy learning experience during the mandated COVID-19 lockdown. First, using nominal-group technique, we identified the DL learning theories to be employed in blueprinting the DL-framework. Effectiveness of the designed DL-framework in anatomy teaching was demonstrated using the exemplar of the Head and Neck (H&N) course during COVID-19 lockdown, in the pre-clerkship curriculum at our medical school. The dissemination of the DL-framework in the anatomy course was informed by the Analyse, Design, Develop, Implement, and Evaluate (ADDIE) model. The efficiency of the DL-framework was evaluated using the first two levels of Kirkpatrick's model. Versatility of the DL-framework was demonstrated by aligning its precepts with individual domains of key learning outcomes framework. The framework's blueprint was designed amalgamating principles of: Garrison's community inquiry, Siemens' connectivism and Harasim's online-collaborative-learning; and improved using Anderson's DL-model. Following the implementation of the DL-framework in the H&N course informed by ADDIE, the framework's efficiency was evaluated. In total, 70% students responded to the survey assessing perception toward DL (Kirkpatrick's Level: 1). Descriptive analysis of the survey results showed that the DL-framework was positively received by students and attested that students had an enriched learning experience, which promoted collaborative-learning and student-autonomy. For, Kirkpatrick's Level: 2 i.e., cognitive development, we compared the summative assessment performance in the H&N course across three cohort of students. The results show that the scores of the cohort, which experienced the course entirely through DL modality was statistically higher (P < 0.01) than both the other cohorts, indicating that shift to DL did not have an adverse effect on students' learning. Using Bourdieu's Theory of Practice, we showed that the DL-framework is an efficient pedagogical approach, pertinent for medical schools to adopt; and is versatile as it attests to the key domains of students' learning outcomes in the different learning outcomes framework. To our knowledge this is the first-study of its kind where a rationale and theory-guided approach has been availed not only to blueprint a DL framework, but also to implement it in the MBBS curriculum.

Development of entrustable professional activities for residents rotating nephrology department in a Japanese university hospital: a Delphi study

OBJECTIVES

Training strategies regarding entrustable professional activities (EPAs) vary from country to country; one such strategy is for residents. However, there are no reports of EPAs developed for residents who rotate to the nephrology departments. We aimed to construct such EPAs, which could be generalised to other institutions.

DESIGN

Purposive design and a modified Delphi method to build consensus.

SETTING

The department of nephrology in a university hospital in Aichi Prefecture, Japan.

PARTICIPANTS

Based on the attainment goals used in our department, an initial list was developed within the research group. The expert panel included 25 nephrologists from our affiliate hospital. Responses were based on a 5-point method and agreement was reached if both (A) and (B) were met: (A) mean≥4 with a SD <1; (B) more than 75% of respondents rated the item 4 or more. With agreement, the item was left for the next round. This round was repeated.

RESULTS

An initial list of 11 items was developed; after three Delphi rounds and revisions, eight items remained that were then established as the final EPAs. These items can serve as a list of goals to be reached by residents who rotate to the department of nephrology. The results indicated that most of the experts believed residents should be able to perform tasks deemed necessary or urgent for all physicians, such as those that deal with hyperkalaemia and heart failure.

CONCLUSIONS

The concept of EPAs enabled us to develop goals and evaluation criteria for residents' training in nephrology. This study can serve as a springboard for future discussions and contribute to the development of resident education in nephrology.

Einsatz von Simulationspersonen in der Lehre und Prüfung psychotherapeutischer Kompetenzen

Zusammenfassung. Theoretischer Hintergrund: In der Klinischen Psychologie und Psychotherapie werden bislang kaum Simulationspersonen (SPs) in Lehre und kompetenzorientierten Prüfungen eingesetzt. Deren geplante Nutzung etwa im Rahmen von Approbationsprüfungen nach dem neuen Psychotherapeutengesetz ist sinnvoll, jedoch mit inhaltlichen und methodischen Herausforderungen verknüpft. Fragestellung: Machbarkeit des Einsatzes von SPs in Prüfungsszenarien und Ableitung von relevanten therapeutischen Fertigkeiten, die in solchen Szenarien geprüft werden können. Methode: Quantitative Evaluation der Umsetzbarkeit einer SP-basierten Prüfung psychotherapeutischer Kompetenzen (Studie 1) und qualitative Erarbeitung therapeutischer Beziehungsfertigkeiten als Basis für Szenarien in kompetenzorientierten Prüfungen mit SPs (Studie 2). Ergebnisse: Die Evaluation der SP-basierten Prüfungen zeigte eine positive Bewertung hinsichtlich ihrer Machbarkeit durch Ausbildungsteilnehmende ( N = 48) und Prüfende ( N = 2), aber auch eine begrenzte Interrater-Reliabilität. In der zweiten Studie wurden aus Interviews mit Expertinnen und Experten ( N = 12 Psychotherapeutinnen und -therapeuten) Kompetenzen für Beziehungsaufbau und Aufrechterhaltung als Grundlage für SP-basierte Prüfungsszenarien aus den Bereichen Fertigkeiten, Haltung und Wissen abgeleitet. Schlussfolgerungen: SP-basierte Lehr- und Prüfungsformate in Psychotherapie und Klinischer Psychologie sind mit inhaltlichen und methodischen Anforderungen (u. a. Falldarstellung, Interrater-Reliabilität) verbunden, die bei der Implementierung berücksichtigt werden müssen.

Lists of potential diagnoses that final-year medical students need to consider: a modified Delphi study

BACKGROUND

Contrastive learning is known to be effective in teaching medical students how to generate diagnostic hypotheses in clinical reasoning. However, there is no international consensus on lists of diagnostic considerations across different medical disciplines regarding the common signs and symptoms that should be learned as part of the undergraduate medical curriculum. In Japan, the national model core curriculum for undergraduate medical education was revised in 2016, and lists of potential diagnoses for 37 common signs, symptoms, and pathophysiology were introduced into the curriculum. This study aimed to validate the list of items based on expert consensus.

METHODS

The authors used a modified Delphi method to develop consensus among a panel of 23 expert physician-teachers in clinical reasoning from across Japan. The panel evaluated the items on a 5-point Likert scale, based on whether a disease should be hypothesized by final-year medical students considering given signs, symptoms, or pathophysiology. They also added other diseases that should be hypothesized. A positive consensus was defined as both a 75% rate of panel agreement and a mean of 4 or higher with a standard deviation of less than 1 on the 5-point scale. The study was conducted between September 2017 and March 2018.

RESULTS

This modified Delphi study identified 275 basic and 67 essential other than basic items corresponding to the potential diagnoses for 37 common signs, symptoms, and pathophysiology that Japanese medical students should master before graduation.

CONCLUSIONS

The lists developed in the study can be useful for teaching and learning how to generate initial hypotheses by encouraging students' contrastive learning. Although they were focused on the Japanese educational context, the lists and process of validation are generalizable to other countries for building national consensus on the content of medical education curricula.

Embedding Ethics Education in Clinical Clerkships by Identifying Clinical Ethics Competencies: The Vanderbilt Experience

The clinical clerkships in medical school are the first formal opportunity for trainees to apply bioethics concepts to clinical encounters. These clerkships are also typically trainees' first sustained exposure to the "reality" of working in clinical teams and the full force of the challenges and ethical tensions of clinical care. We have developed a specialized, embedded ethics curriculum for Vanderbilt University medical students during their second (clerkship) year to address the unique experience of trainees' first exposure to clinical care. Our embedded curriculum is centered around core "ethics competencies" specific to the clerkship: for Medicine, advanced planning and end-of-life discussions; for Surgery, informed consent; for Pediatrics, the patient-family-provider triad; for Obstetrics and Gynecology, women's autonomy, unborn child's interests, and partner's rights; and for Neurology/Psychiatry, decision-making capacity. In this paper, we present the rationale for these competencies, how we integrated them into the clerkships, and how we assessed these competencies. We also review the additional ethical issues that have been identified by rotating students in each clerkship and discuss our strategies for continued evolution of our ethics curriculum.

Measuring social interdependence in collaborative learning: instrument development and validation

BACKGROUND

In health professions education, several collaborative learning approaches have been used. As collaborative learning has a theoretical background of social interdependence theory, a theory informed and valid instrument to measure social interdependence is required to evaluate and compare several learning approaches. The aim of this study was to develop an instrument (the SOcial interdependence in Collaborative learning Scale; SOCS) to measure students' perceived social interdependence in collaborative learning and validate it.

METHODS

We conducted a modified Delphi procedure among stakeholders to develop the content validity of the instrument. To establish construct validity, we performed a confirmatory factor analysis, and we estimated reliability.

RESULTS

Two rounds of Delphi were conducted to develop the instrument. Confirmatory factor analysis yielded a three-factor model with 15 items, which provided an excellent fit with CMIN/df = 1.838, GFI = 0.924, CFI = 0.951, RMSEA = 0.061, and PCLOSE = 0.121. Alpha-coefficients for all factors indicated high internal consistency of all the factors.

CONCLUSION

This study describes the development and construct validation of the SOCS for measuring social interdependence in collaborative learning. This instrument will provide teachers and schools with feedback about their classroom environment.

Core Competencies in Evidence-Based Practice for Health Professionals: Consensus Statement Based on a Systematic Review and Delphi Survey

IMPORTANCE

Evidence-based practice (EBP) is necessary for improving the quality of health care as well as patient outcomes. Evidence-based practice is commonly integrated into the curricula of undergraduate, postgraduate, and continuing professional development health programs. There is, however, inconsistency in the curriculum content of EBP teaching and learning programs. A standardized set of minimum core competencies in EBP that health professionals should meet has the potential to standardize and improve education in EBP.

OBJECTIVE

To develop a consensus set of core competencies for health professionals in EBP.

EVIDENCE REVIEW

For this modified Delphi survey study, a set of EBP core competencies that should be covered in EBP teaching and learning programs was developed in 4 stages: (1) generation of an initial set of relevant EBP competencies derived from a systematic review of EBP education studies for health professionals; (2) a 2-round, web-based Delphi survey of health professionals, selected using purposive sampling, to prioritize and gain consensus on the most essential EBP core competencies; (3) consensus meetings, both face-to-face and via video conference, to finalize the consensus on the most essential core competencies; and (4) feedback and endorsement from EBP experts.

FINDINGS

From an earlier systematic review of 83 EBP educational intervention studies, 86 unique EBP competencies were identified. In a Delphi survey of 234 participants representing a range of health professionals (physicians, nurses, and allied health professionals) who registered interest (88 [61.1%] women; mean [SD] age, 45.2 [10.2] years), 184 (78.6%) participated in round 1 and 144 (61.5%) in round 2. Consensus was reached on 68 EBP core competencies. The final set of EBP core competencies were grouped into the main EBP domains. For each key competency, a description of the level of detail or delivery was identified.

CONCLUSIONS AND RELEVANCE

A consensus-based, contemporary set of EBP core competencies has been identified that may inform curriculum development of entry-level EBP teaching and learning programs for health professionals and benchmark standards for EBP teaching.

How Should We Best Train Clinical Psychology Supervisors? A Delphi Survey

Although clinical supervision plays a major part in the professional training of clinical psychologists, there are as yet no agreed guidelines for preparing supervisors for this key educational role. This article uses the Delphi approach to see whether an expert panel of UK clinical psychologists could come to a consensus on what the most suitable components of such a supervisor training programme might be.

The professional competence of speech therapists. IV: attitude and attribute base

This paper reports on an empirical study of the attitude and attribute base of speech therapy. This is the final part of a four-part report on a study of the professional competence of speech therapists. The consultative methods used in this study combined the Delphi and Nominal Group techniques with a postal questionnaire survey of specialist clinicians in the UK, the details of which are reported elsewhere. The present discussion of the attitude base of speech therapy concluded that there was a strong consensus of agreement amongst speech therapists on the attitudes and attributes considered essential to professional competence. However, it would appear that these attitudes and attributes are not exclusive to speech therapists, but are common to many professional groups.

The professional competence of speech therapists. I: introduction and methodology

This paper is the first of a four part report on a study of the professional competence of speech therapists in the United Kingdom. It considers some of the methodological approaches to the study of professional competence, and describes the methods adopted for this study. A combination of consultative and survey techniques were used to determine which knowledge, skills and attitudes were considered essential to the competence of speech therapists working in three specialities; children, learning difficulties, and elderly/adults with acquired neurological disorders. Sixty-eight speech therapy 'experts' were consulted using Delphi and Nominal Group techniques. The items of knowledge, skills and attitudes generated by these experts were then distributed to 657 specialist speech therapists in order that the lists could be validated by a larger sample of clinicians.

A Practical Methodological Approach Towards Identifying Core Competencies in Medical Education Based on Literature Trends: A Feasibility Study Based on Vestibular Schwannoma Science

BACKGROUND

Competency-based medical education (CBME) is gaining momentum in postgraduate residency and fellowship training. While randomized trials, consensus statements, and practice guidelines can help delineate some of the core competencies for CBME, they are not applicable to all clinical scenarios.

OBJECTIVE

To propose and assess the feasibility of a practical methodology for addressing this issue using radiosurgery for vestibular schwannoma (VS) science as an example.

METHODS

The Web of Science electronic database was searched using relevant terms. A 3-step review of titles and abstracts was used. Studies were classified independently and in duplicate as either efficacy or effectiveness analyses. Cohen's kappa score was used to assess inter-rater agreement.

RESULTS

Overall, 1818 surgical and 943 radiosurgical publications were identified. The number of effectiveness studies surpassed that of efficacy studies in the late 1980s for surgical studies, and in the early-to-mid 1990s among radiosurgical studies. The publication rate was higher for radiosurgery in the mid 1990s, but it paralleled that of surgical studies beyond the early 2000s. Variations in this overall trend corresponded to the emergence of studies that assessed the role of endoscopy and the utility of dose reduction in radiosurgery.

CONCLUSION

We have confirmed the feasibility and accuracy of this objective methodological approach. By understanding how the peer-reviewed literature reflects actual practice interests, educators can tailor curricula to ensure that trainees remain current. While further validation studies are needed, this methodology can serve as a supplemental strategy for identifying additional core competencies in CBME.

What Should Junior Doctors Know about the Drugs they Frequently Prescribe? A Delphi Study among Physicians in the Netherlands

The aim of this study was to identify the information about commonly prescribed drugs that junior doctors should know in order to prescribe rationally in daily practice, defined as essential drug knowledge (EDK). A two-round Internet Delphi study was carried out involving general practitioners from one practice cluster, and registrars and consultants from two Dutch academic and eight teaching hospitals. A preliminary list of 377 potential EDK items for three commonly prescribed drugs was assessed on a dichotomous scale; an item was considered EDK if at least 80% consensus was reached. The consensus list of EDK items was discussed by the research team to identify similarities between the three drugs, with a view to forming a list of general EDK items applicable to other commonly prescribed drugs. Sixty experts considered 93 of the 377 items (25%) as EDK. These items were then used to form a list of 10 general EDK items. The list of EDK items identified by primary and secondary care doctors could be used in medical curricula and training programmes and for assessing the prescribing competence of future junior doctors. Further research is needed to evaluate the generalizability of this list for other commonly prescribed drugs.

An instrument for evaluating clinical teaching in Japan: content validity and cultural sensitivity

BACKGROUND

Many instruments for evaluating clinical teaching have been developed but almost all in Western countries. None of these instruments have been validated for the Asian culture, and a literature search yielded no instruments that were developed specifically for that culture. A key element that influences content validity in developing instruments for evaluating the quality of teaching is culture. The aim of this study was to develop a culture-specific instrument with strong content validity for evaluating clinical teaching in initial medical postgraduate training in Japan.

METHODS

Based on data from a literature search and an earlier study we prepared a draft evaluation instrument. To ensure a good cultural fit of the instrument with the Asian context we conducted a modified Delphi procedure among three groups of stakeholders (five education experts, twelve clinical teachers and ten residents) to establish content validity, as this factor is particularly susceptible to cultural factors.

RESULTS

Two rounds of Delphi were conducted. Through the procedure, 52 prospective items were reworded, combined or eliminated, resulting in a 25-item instrument validated for the Japanese setting.

CONCLUSIONS

This is the first study describing the development and content validation of an instrument for evaluating clinical teaching specifically tailored to an East Asian setting. The instrument has similarities and differences compared with instruments of Western origin. Our findings suggest that designers of evaluation instruments should consider the probability that the content validity of instruments for evaluating clinical teachers can be influenced by cultural aspects.

A novel approach to needs assessment in curriculum development: going beyond consensus methods

BACKGROUND

Needs assessment should be the starting point for curriculum development. In medical education, expert opinion and consensus methods are commonly employed.

AIM

This paper showcases a more practice-grounded needs assessment approach.

METHODS

A mixed-methods approach, incorporating a national survey, practice audit, and expert consensus, was developed and piloted in thrombosis medicine; Phase 1: National survey of practicing consultants, Phase 2: Practice audit of consult service at a large academic centre and Phase 3: Focus group and modified Delphi techniques vetting Phase 1 and 2 findings.

RESULTS

Phase 1 provided information on active curricula, training and practice patterns of consultants, and volume and variety of thrombosis consults. Phase 2's practice audit provided empirical data on the characteristics of thrombosis consults and their associated learning issues. Phase 3 generated consensus on a final curricular topic list and explored issues regarding curriculum delivery and accreditation.

CONCLUSIONS

This approach offered a means of validating expert and consensus derived curricular content by incorporating a novel practice audit. By using this approach we were able to identify gaps in training programs and barriers to curriculum development. This approach to curriculum development can be applied to other postgraduate programs.

Video-assisted structured teaching to improve aseptic technique during neuraxial block

BACKGROUND

Teaching epidural catheter insertion tends to focus on developing manual dexterity rather than improving aseptic technique which usually remains poor despite increasing experience. The aim of this study was to compare epidural aseptic technique performance, by novice operators after a targeted teaching intervention, with operators taught aseptic technique before the intervention was initiated.

METHODS

Starting July 2008, two groups of second-year anaesthesia residents (pre- and post-teaching intervention) performing their 4-month obstetric anaesthesia rotation in a university affiliated centre were videotaped three to four times while performing epidural procedures. Trained blinded independent examiners reviewed the procedures. The primary outcome was a comparison of aseptic technique performance scores (0-30 points) graded on a scale task-specific checklist.

RESULTS

A total of 86 sessions by 29 residents were included in the study analysis. The intraclass correlation coefficient for inter-rater reliability for the aseptic technique was 0.90. The median aseptic technique scores for the rotation period were significantly higher in the post-intervention group [27.58, inter-quartile range (IQR) 22.33-29.50 vs 16.56, IQR 13.33-22.00]. Similar results were demonstrated when scores were analysed for low, moderate, and high levels of experience throughout the rotation.

CONCLUSIONS

Procedure-specific aseptic technique teaching, aided by video assessment and video demonstration, helped significantly improve aseptic practice by novice trainees. Future studies should consider looking at retention over longer periods of time in more senior residents.

Essential facets of competence that enable trust in graduates: a delphi study among physician educators in the Netherlands

BACKGROUND

There is a need for valid methods to assess the readiness for clinical practice of recently graduated physicians. To develop these methods, it is relevant to know the general features of trainees' performance that facilitate supervisors' trust in their ability to perform critical clinical tasks.

OBJECTIVE

To discover such essential facets of competence (FOCs), based on the opinion of experienced physician educators.

METHODS

We conducted a Delphi study, consisting of 2 rounds, among 18 experienced physician educators in the Netherlands. Mean, standard deviation, level of agreement, and skewness were calculated for the importance of FOCs for making entrustment decisions. The study yielded a list of 25 FOCs.

RESULTS

In the first round, means were between 6.50 and 7.00 on a 7-point Likert scale (SD, 0.42-2.18); in the second round, means ranged from 5.45 to 6.90 (SD, 0.3-2.02). The level of agreement was high for 92% of the FOCs in the first round and 100% of the FOCs in the second round.

CONCLUSIONS

Our Delphi study found consensus among experts about FOCs that are important for clinical entrustment decisions.

Educational content in nurse education in Japan: A Delphi study

The purpose of this study was to clarify the minimum necessary educational content in the area of nursing ethics in a basic nursing education program, and the level of students’ mastery of this content, based on a Delphi study in both educational and clinical settings. A Delphi study was conducted in three rounds with faculty members who teach nursing ethics at all 158 four-year nursing universities in Japan as targeted panelists. In this study the opinions on nursing ethics of nursing instructors responsible for hospital education at all 82 special functioning hospitals were reflected in the panelists’ opinions. Consensus was obtained on 41 items from a total of 63 items in 4 proposed frameworks. There were 20 items related to the Concept of nursing ethics, 7 items related to Ethical codes, 13 items related to Ethical issues and methods to resolve them, and 1 item related to Efforts and issues in practical and educational settings. Consensus as to desired level of mastery was reached on a total of 40 items. This agreed-upon level involved understanding of the concept for 22 items, the ability to explain the concept for 16 items, and the ability to act based on the concept for 2 items.

Validating objectives and training in Canadian paediatrics residency training programmes

BACKGROUND

Changing health care systems and learning environments with reduction in resident work hours raises the question: "Are we adequately training our paediatricians?"

AIMS

(1) Identify clinical competencies to be acquired during paediatric residency training to enable graduates to practise as consultant paediatricians; (2) Identify gaps in preparedness during training and; (3) Review and validate competencies contained in the Royal College of Physicians and Surgeons of Canada (RCPSC) objectives of training (OTR) for paediatrics.

METHODS

A questionnaire with 19 classification domains containing 92 clinical competencies was administered to RCPSC certified paediatricians who completed residency training in Canada from June 2004 to June 2008. For each competency, paediatricians were asked to indicate the importance and their degree of preparedness upon entering practice. Gap scores (GSs) between importance and preparedness were calculated.

RESULTS

Response rate was 43% (187/435); 91.3% (84/92) of competencies in the RCPSC OTR were identified as important. Paediatricians felt less than adequately prepared for 25% (23/92) of competencies; 40 competencies had GSs >10%.

CONCLUSIONS

The unique approach used in this study is useful in validating OTR as well as the preparation of residents in relation to OTR. The results indicate a potential need for additional training in specific competencies.

Establishing cutoff scores on assessments of surgical skills to determine surgical competence

OBJECTIVE

The aim of this study was to establish minimum cutoff scores on intraoperative assessments of surgical skills to determine surgical competence for vaginal hysterectomy.

STUDY DESIGN

Two surgical rating scales, the Global Rating Scale of Operative Performance and the Vaginal Surgical Skills Index, were used to evaluate trainees while performing vaginal hysterectomy. Cutoff scores were determined using the Modified Angoff method.

RESULTS

Two hundred twelve evaluations were analyzed on 76 surgeries performed by 27 trainees. Trainees were considered minimally competent to perform vaginal hysterectomy if total absolute scores (95% confidence interval) on Global Rating Scale = 18 (16.5-20.3) and Vaginal Surgical Skills Index = 32 (27.7-35.5). On average, trainees met new cutoffs after performing 21 and 27 vaginal hysterectomies, respectively. With the new cutoffs applied to the same cohort of fourth-year obstetrics and gynecology trainees, all residents achieved competency in performing vaginal hysterectomy by the end of their gynecology rotations.

CONCLUSION

Standard-setting methods using cutoff scores may be used to establish competence in vaginal surgery.

Development of the European Colposcopy Core Curriculum: use of the Delphi technique

The objective of this study was to develop a European competence-based colposcopy core curriculum using the Delphi technique. Thirty expert colposcopists from 21 countries participated. A four-round iterative questionnaire was used. Competencies were rated using a five-point Likert scale. Competences rated as 4 or more by at least 90% of the respondents were regarded as necessary for the core curriculum. Eighteen participants took part in all four rounds, and 27 were active in each of the last three rounds. Fifty-one core competences were selected from a list of 76 competences collated by the group as a whole. The majority (n = 44) of the selected core competences received a score of 4 or greater in each round. Overall, there was some evidence of increasing consensus but the individual shift in opinion was slight. The Delphi technique was an effective tool for obtaining an expert consensus and enabled group "ownership" of the identified core curriculum.

Continuing medical education in pediatric anesthesia--a theoretical overview

The importance of continuing medical education (CME) as a method of improving the quality of care of children undergoing anesthesia is universally recognized. This article, which is based on a presentation at the FEAPA European Conference on Paediatric Anaesthesia in September 2007 in Amsterdam, gives a theoretical overview of continuing education and introduces some generic educational concepts, such as the CRISIS-criteria and Kirkpatrick's evaluation model, which are as relevant to pediatric anesthesia as to other areas of medical practice. The terms CME and continuing professional develop are described. Some consideration is given to how anesthesiologists can assess the potential worth of an educational activity for their practice. No attempt will be made to judge particular educational activities, as the choice of the most appropriate activity rests primarily with the individual.

A Delphi study to develop the Association for Palliative Medicine consensus syllabus for undergraduate palliative medicine in Great Britain and Ireland

The Association for Palliative Medicine (APM) produced a previous undergraduate palliative medicine syllabus in 1992. This study describes the process of developing the new APM consensus syllabus against the background of changes in medical education and palliative medicine since 1992. The syllabus was derived by means of a Delphi study carried out amongst experts in palliative medicine across Britain and Ireland. Forty-three participants agreed to take part. Three rounds of the Delphi study took place. Consensus (75% agreement) was achieved in over 90% of the outcomes. The new syllabus is broken down into the following sections: basic principles, physical care, psychosocial care, culture, language, religious and spiritual issues, ethics and legal frameworks. Learning outcomes are categorised as essential or desirable. Using a Delphi study, we have developed a consensus syllabus for undergraduate palliative medicine. This is sufficiently flexible to allow all medical schools to ensure that their students achieve the essential learning outcomes by the time they graduate, whereas those with more generous curricular space will additionally be able to deliver selected desirable learning outcomes.

Twelve tips for preparing residents as teachers

BACKGROUND

Residents are frequently identified by medical students as their most frequent and memorable teachers; residents also teach their peers, junior and senior colleagues, other health professionals, and their patients. Many will teach in their future practice. Developing the skills to become a teacher is an important part of postgraduate education, and warrants a systematic, planned approach that may include many complementary learning opportunities.

AIMS

Our purpose is to describe one such approach: a 4-week elective experience in medical education offered to postgraduate learners.

METHOD

The paper describes the background and goals for the elective, and the various steps in planning, implementing, and evaluating such a course, drawing on the literature and mining our own experience for examples. Specifically, we address the following: needs assessment; the determination and selection of content, sequence, and teaching and learning methods; the experiential learning opportunities offered; and the emphasis on the participants' developing self-awareness of themselves as teachers, and as part of a community of teachers.

RESULTS

The program implementation, program evaluation, and response to feedback received are described.

CONCLUSION

A 4-week elective experience in medical education was positively received by participants.

The use of an evidence-based portfolio in the management of change in dental practice

In this paper the author gives his opinion about the problems of getting practices to change systems in order to institute clinical governance. There are many reasons why practices need to change and for this change to be monitored. This paper explains the need for change and the use of the evidence-based portfolio, which is produced by candidates for the Membership of the Faculty of General Dental Practice (UK) [MFGDP(UK)] examination. It can also be produced by individuals who are not taking the MFGDP(UK) examination in conjunction with the Faculty of General Dental Practice (UK)'s key skills programme. It provides a mechanism for demonstrating change and for assessing the quality of care provided by a general dental practice. The author concludes that the evidence-based portfolio will enable a practitioner to apply clinical governance in a practical way.

Developing a competence-based core curriculum in biomedical laboratory science: a Delphi study

In this study the Delphi technique has been used to develop a core curriculum for education of the biomedical scientist. The rapid development in biomedicine and the corresponding changes in methodology in biomedical laboratories demand careful planning of the education of biomedical scientists. The Delphi technique uses an anonymous panel of experts for suggestions and assessments aiming at consensus. Twenty-six experts from different kinds of hospital and university laboratories took part in the investigation. They suggested and assessed necessary competences for a recently graduated biomedical scientist, and if 75% or more of the participants agreed on a competence, it was included in the core curriculum. The final list consisted of 66 competences of varying depth, in three categories. This list contained several generic competences, concerning for example basic laboratory methods, handling of samples, dealing with apparatus and applying relevant rules and laws; basic knowledge in chemistry, preclinical medicine and laboratory methods; and finally attitudes that the panel expected in the recently graduated person. The core was sufficiently restricted to be used in a three-year programme and still leave space for about one year of electives/special study modules. It became rather traditional, e.g. it did not include competences that many recent reports consider important for the future professional.

Defining professionalism in anaesthesiology

BACKGROUND

The Royal College of Physicians and Surgeons of Canada, through the CanMEDS 2000 project has identified the role of professional as 1 of 7 roles for which physicians are expected to be accountable when dealing with patients. Each specialty is responsible for defining this role relative to their specialty.

METHODS

The qualities of professionalism for anaesthesiology were defined through a Delphi process involving Canadian anaesthesiology educators. The panellists took part in 3 rounds that identified qualities in 3 distinct areas of professionalism - humanistic qualities, personal development qualities and anaesthesiology meta-competences.

RESULTS

A total of 23 of 29 anaesthesiologists responded (79%) in round 1, with response rates to rounds 2 and 3 being 72% and 69%, respectively. Of the original 36 qualities, some were combined, definitions were changed in 23, 11 qualities were added and 4 were deleted, leaving a list of 40 qualities.

DISCUSSION

There is considerable interest in this issue among the Canadian educators in postgraduate anaesthesiology. Consensus on important professional qualities for anaesthesiologists was obtained through the Delphi technique. These qualities will form the basis of identifiable professional behaviours to which anaesthesiologists should aspire.

Shifting paradigms: from Flexner to competencies

Realizing medical education is on the brink of a major paradigm shift from structure- and process-based to competency-based education and measurement of outcomes, the authors reviewed the existing medical literature to provide practical insight into how to accomplish full implementation and evaluation of this new paradigm. They searched Medline and the Educational Resource Information Clearinghouse from the 1960s until the present, reviewed the titles and abstracts of the 469 articles the search produced, and chose 68 relevant articles for full review. The authors found that in the 1970s and 1980s much attention was given to the need for and the development of professional competencies for many medical disciplines. Little attention, however, was devoted to defining the benchmarks of specific competencies, how to attain them, or the evaluation of competence. Lack of evaluation strategies was likely one of the forces responsible for the three-decade lag between initiation of the movement and wide-spread adoption. Lessons learned from past experiences include the importance of strategic planning and faculty and learner buy-in for defining competencies. In addition, the benchmarks for defining competency and the thresholds for attaining competence must be clearly delineated. The development of appropriate assessment tools to measure competence remains the challenge of this decade, and educators must be responsible for studying the impact of this paradigm shift to determine whether its ultimate effect is the production of more competent physicians.

What knowledge and skills are essential for specialists in Clinical Pharmacology and Therapeutics? Results of a Delphi study

AIMS

To identify the knowledge and skills that should be considered essential for specialists in Clinical Pharmacology and Therapeutics (CPT) with a commitment to the National Health Service (NHS).

METHODS

A Delphi study using a sample of current specialists.

RESULTS

Members of the expert panel (20 in all, representative of the Clinical Section membership) identified 78 statements for consideration, in four domains (core of knowledge, therapeutic skills, educative skills, and investigative skills), of which 58 (74.4%) were accepted by more than two-thirds of respondents. Of these, 35 were knowledge items, whereas 23 were skills (11 therapeutic, 4 educative and 8 investigative). The large majority (79.3%) of these statements were endorsed by at least four out of five panel members.

CONCLUSIONS

Despite the varied work patterns and responsibilities of specialists in CPT, it is possible to identify a core of knowledge and skill that most consider essential for the delivery of their commitment to the NHS. The findings will provide NHS Trusts with a clear idea of what they can expect from these specialists, and will act as a checklist for specialists themselves to direct their Continuing Professional Development within the consultant appraisal process. The results also describe a curriculum for continuing education in CPT, which the BPS Clinical Section can use to develop information resources and training opportunities.

Impact of guidelines implemented in a paris university hospital: application to the use of antiemetics by cancer patients

AIMS

To assess the impact with time of guidelines on antiemetic use in an 850-bed Paris university hospital with a high proportion of cancer patients.

METHODS

Guidelines on the use of antiemetics available in cancer chemotherapy were drafted according to the Delphi technique. Their implementation was based upon a patient-specific antiemetic prescription form. To assess the impact of guideline implementation over time, discrepancies between current practice and the guidelines were compared before guideline implementation (between March and August 1995) and after implementation (between March and August 1997, and March and August 1998).

RESULTS

Before the Delphi panel's guidelines were implemented, 5-HT3 antagonists were inappropriately administered in 70% of cases. After guideline implementation, this proportion dropped significantly (P<0.0001, Fisher's exact test) to 22% between March and August 1997 and 28% between March and August 1998.

CONCLUSIONS

Implementation of guidelines seems to have resulted in significant changes with time, although a causal relationship has not been demonstrated. The development of guidelines by our hospital's multidisciplinary working group helped the various consultants to adjust medical practices to take account of these changes.

Performance on the radiology component of a critical clinical skills examination

RATIONALE AND OBJECTIVES

Clinical competence certification is now required in some specialties in medicine. A Comprehensive Clinical Assessment (CCA) was created to test mastery of critical skills by students at the end of the 3rd year of medical school.

METHODS

The CCA is a series of stations that test skills the faculty consider important for all medical students (eg, breast examination, electrocardiogram reading, chest pain assessment, ophthalmology photographs). The radiology station was designed to evaluate imaging skills believed to be taught and learned in the core 3rd-year rotations.

RESULTS

External measures (National Board Examinations, grade point average, and overall score) of clinical performance of the 608 medical students who completed the CCA examination between 1991 and 1993 were found to be correlated with the radiology station scores.

CONCLUSION

The radiology station in the CCA examination is a reproducible measure of clinical performance.

Teaching the teachers--a needs assessment of tutors for a new clinical skills course

There is an increasing trend in undergraduate education towards teaching clinical skills from a community base. A new clinical curriculum was introduced in Newcastle upon Tyne in 1995, beginning with an integrated clinical skills course. Although the attitudes and views of general practitioners (GPs) towards community-based clinical teaching have previously been reported, their perceived training needs have not been formally identified. The aims of this study were to identify the competencies needed by GPs for community-based clinical skills teaching, to compare and contrast these needs with their hospital colleagues, and to use the results to develop a teaching programme for the clinical tutors involved in the new course. In order of priority, the GPs and hospital tutors expressed similar needs: small-group teaching skills, assessing student needs, giving effective feedback and assessment of student performance, with a preference for the teaching to be organized within local teaching units. Most GPs and hospital tutors (73 and 69%, respectively) requested a distance-learning pack to complement the teaching. General practitioners rated resources for improving their individual clinical skills more highly than their hospital colleagues: for example, videotapes demonstrating examination techniques. Forty-six per cent of GP tutors had received some formal training in teaching methods compared to 29% of hospital tutors. The implications of the results for developing a 'Teaching the Teachers' course for clinical tutors are discussed.

A comparison between the roles of part-time and full-time clinical tutors: is there a difference?

Recent changes in pre-registration radiographer education required quality issues concerning students' clinical experience to be reviewed. This investigation was undertaken to illuminate one important aspect of clinical education: the way in which clinical tutors spend their time. The study described, quantified and compared the role of part-time tutors with those who are employed full-time. From the data, three main roles for tutors emerged: duties associated with students, duties of a professional nature, and duties which were considered to be developmental. On the basis of the results, it seems reasonable to suggest that, because part-time tutors devote as much time to students as those employed full-time, there is no disadvantage to students. However, further research is required concerning the role and duties of clinical tutors. In addition, recommendations for improving the current methodology are made, if the study is repeated.

Competence versus performance: which is more important?

The extensive literature in the field of competence and performance is reviewed in this paper, which argues that there is much confusion between these different constructs. It concludes that since nurse pre-registration education is concerned with producing nurses who consistently provide care of a high quality, greater emphasis should be placed upon performance in the real-life clinical setting rather than upon competence.

Effective continuing education: the CRISIS criteria

The need for continuing medical education (CME) is now well recognized. The challenge is to make it effective. CRISIS, an acronymn, stands for the criteria which must be met to produce effective CME programmes: convenience, relevance, individualization, self-assessment, interest, speculation and systematic. CRISIS is a practical tool, based on 15 years of experience in the production and evaluation of CME programmes at the Centre for Medical Education, University of Dundee. The application of the CRISIS criteria to a CME programme will highlight any areas needing improvement and will guide programme producers in the creation of new CME materials. It will also help those responsible for planning CME activities to choose from a range of existing programmes.

Impact of technology on the education of health care professionals

A rapid development in health care delivery has been the increasing use of technology. Technology will continue to transform medical practice and all health care workers are involved with technology by accident or design. This has resulted in technology dominating many health policy decisions (4).

Ten questions to ask when planning a course or curriculum

This brief practical aid to course or curriculum development cannot replace educational qualifications or experience, but it does examine ten basic questions, any of which may be all too easily neglected. These are: What are the needs in relation to the product of the training programme? What are the aims and objectives? What content should be included? How should the content be organized? What educational strategies should be adopted? What teaching methods should be used? How should assessment be carried out? How should details of the curriculum be communicated? What educational environment or climate should be fostered? How should the process be managed? Each aspect is illustrated through the analogy of car manufacturing. The ten questions are relevant in all situations where a course or curriculum is being planned, including an undergraduate degree course, a short postgraduate course or a 1-hour lecture.