Deliberate practice and the acquisition and maintenance of expert performance in medicine and related domains

Lifelong learning at work

The importance of lifelong learning in medicine is well recognised. This article explores how junior doctors can develop learning strategies for use throughout their working life

Visuospatial and technical ability in the selection and assessment of higher surgical trainees in the London deanery

INTRODUCTION

Despite awareness of the limitations of current selection and competency assessments, there is little consensus and alternatives have not been readily accepted. Essential surgical skills include visuospatial and technical ability. The aim of this study was to survey current methods of higher surgical trainee selection and assessment. We suggest ways to improve the process.

MATERIALS AND METHODS

Nine surgical training programmes in the London deanery were surveyed through questionnaires to programme directors, existing trainees and examination of deanery publications.

RESULTS

Testing of visuospatial and technical ability was piloted at selection only in a single general surgical department. Practical skills were assessed in 3/9 (33%) specialties (ENT, plastic and general surgery). Once selected, no specialty tested visuospatial and technical ability. Practical skills were tested in only 1/9 (11%) specialties (plastic surgery). The remaining 8/9 (89%) were 'assessed' by interview.

CONCLUSIONS

Lack of visuospatial and technical ability assessment was identified at selection and during higher surgical training. Airlines have long recognised early identification of these qualities as critical for efficient training. There is a need for more objective methods in this area prior to selection as time to assess surgical trainees during long apprenticeships is no longer available. We advocate a suitably validated competency-based model during and at completion of training.

Certification pass rate of 100% for fundamentals of laparoscopic surgery skills after proficiency-based training

BACKGROUND

The fundamentals of laparoscopic surgery (FLS) program has been extensively validated for use as a high-stakes examination for certification purposes, but optimal methods for its use in skills training have not been described. This study aimed to investigate the feasibility of implementing a proficiency-based FLS skills training curriculum and to evaluate its effectiveness in preparing trainees for certification.

METHODS

For this study, 21 novice medical students at two institutions viewed video tutorials, then performed one repetition of the five FLS tasks as a pretest. The pretests were scored using standard testing metrics. The trainees next practiced the tasks over a 2-month period until they achieved proficiency for all the tasks. A modified on-the-fly scoring system based on expert-derived performance was used. The trainees were posttested using the high-stakes examination format.

RESULTS

No trainee passed the certification examination at pretesting. The trainees achieved proficiency for 96% of the five tasks during training, which required 9.7 +/- 2.4 h (range, 6-14 h) and 119 +/- 31 repetitions (range, 66-161 repetitions). The trainees rated the proficiency levels as "moderately difficult" (3.0 +/- 0.7 on a 5-point scale) and "highly appropriate" (4.7 +/- 0.1 on a 5-point scale). At posttesting, 100% of the trainees passed the certification examination and demonstrated significant improvement compared with pretesting for normalized score (468 +/- 24 vs 126 +/- 75; p < 0.001), self-rated laparoscopic comfort (89.4% vs 4.8%; p < 0.001), and skill level (3.6 +/- 0.9 vs 1.2 +/- 0.5; p < 0.001, 5-point scale).

CONCLUSIONS

This proficiency-based curriculum is feasible for training novices and uniformly allows sufficient skill acquisition for FLS certification. Endorsed by the Society of American Gastrointestinal Endoscopic Surgeons (SAGES), this curriculum is available for use as an optimal method for FLS skills training. More widespread adoption of this curriculum is encouraged.

The new ACS/APDS Skills Curriculum: moving the learning curve out of the operating room

Surgical education has dramatically changed in response to numerous constraints placed on residency programs, but a substantial gap in uniform practices exist, especially in the area of skills laboratory availability and usage. Simulation-based training has gained significant momentum and will be a requirement for residencies in the near future. In response, the American College of Surgeons and the Association of Program Directors in Surgery have formed a Surgical Skills Curriculum Task Force with the aim of establishing a National Skills Curriculum. The first of three phases will undergo implementation in 2007, with subsequent phases scheduled for launch in 2008. The curriculum has been carefully structured and designed by content experts to enhance resident training through reproducible simulations, with verification of proficiency before operative experience. Free-of-charge distribution is planned through a web-based platform, and widespread adoption is encouraged. In the future, these simulation-based strategies may be useful in assuring the competency of practicing surgeons and for credentialing purposes.

Contextualized simulation and procedural skills: a view from medical education

Simulation offers an attractive solution to the profound changes affecting traditional approaches to learning clinical procedural skills. Technical developments in physical models and virtual-reality computing make it possible to practice an increasing range of procedures "in vitro." However, too narrow a focus on technical skill can overlook crucial elements of clinical care such as communication and professionalism. Patient-focused simulation (the combination of a simulated patient with an inanimate simulator or item of medical equipment) allows clinical procedures to be practiced and assessed within realistic scenarios that recreate clinical challenges by placing a real person at the center of the encounter. This paper draws on work with human clinical procedures, exploring the parallels with veterinary practice and highlighting possible developments in client-focused simulation. The paper concludes by arguing for closer collaboration and dialogue between the medical and veterinary professions, for the benefit of both.

Mastery learning of thoracentesis skills by internal medicine residents using simulation technology and deliberate practice

BACKGROUND

Thoracentesis has been identified as a core competency for hospitalists. Residency training may not provide adequate preparation to perform this procedure.

OBJECTIVE

Our aim was to use a thoracentesis simulation to assess graduating residents' proficiency in thoracentesis procedural skills and to evaluate the impact of an educational intervention on skill development to mastery standards.

DESIGN

This was a pretest-posttest mastery learning design without a control group.

SETTING

Participants were 40 third-year internal medicine residents in a university-affiliated program.

INTERVENTION

Using an observational checklist, each resident underwent baseline assessment of thoracentesis skill using a standardized clinical history, radiograph, and thoracentesis simulation. After baseline testing, residents received two 2-hour education sessions featuring a videotaped presentation and deliberate practice with the thoracentesis simulator.

MEASUREMENTS

Residents were retested after the intervention. Skill mastery was defined as meeting or exceeding the minimum passing score (MPS) set by an expert panel at thoracentesis posttest. Those who did not achieve the MPS had additional deliberate practice and were retested until the MPS was reached.

RESULTS

Performance improved 71% from pretest to posttest on the clinical skills examination. All residents met or exceeded the mastery standard. The amount of practice time needed to reach the MPS was the only predictor (negative) of posttest performance. The education program was rated highly.

CONCLUSIONS

A curriculum featuring deliberate practice dramatically increased the skills of residents in thoracentesis. Residents enjoy training and receiving evaluation and feedback in a simulated clinical environment. This mastery program illustrates a feasible and reliable mechanism to achieve procedural competency.

Non-analytical models of clinical reasoning: the role of experience

OBJECTIVE

This paper aims to summarise the evidence supporting the role of experience-based, non-analytic reasoning (NAR) or pattern recognition as a central feature of expert medical diagnosis.

METHODS

The authors examine a series of studies, primarily from their own research programme at McMaster University, that demonstrate that expert and novice diagnostic problem solving is based, to some degree, on similarity to a prior specific exemplar in the memory.

RESULTS

The studies reviewed have shown NAR to be a component of diagnostic reasoning at all levels from novice to subspecialist, and in dermatology, electrocardiography and psychiatry. The retrieval process is rapid and is not available to retrospect. It may be based on visual similarity, but can also be present in verbal descriptions. Some evidence exists that the process is unlikely to be available to introspection. Further, early hypotheses based on NAR can result in the re-interpretation of critical clinical findings.

CONCLUSIONS

Non-analytic reasoning is a central component of diagnostic expertise at all levels. Clinical teaching should recognise the centrality of this process, and aim to both enhance the process through the learning of multiple examples and to supplement the process with analytical de-biasing strategies.

An expert-performance perspective of research on medical expertise: the study of clinical performance

CONTEXT

Three decades ago Elstein et al. published their classic book on medical expertise, in which they described their failure to identify superior performance by peer-nominated diagnosticians using high- and low-fidelity simulations of the everyday practice of doctors.

OBJECTIVE

This paper reviews the results of subsequent research, with a particular emphasis on the progress toward Elstein et al.'s goal of capturing the essence of superior clinical performance in standardised settings in order to improve clinical practice.

RESULTS

Research following publication of Elstein et al.'s book was influenced by laboratory research in cognitive psychology, which resulted in a redirection of its original focus on capturing clinical performance in practice to studies of changes in cognitive processes as functions of extended clinical experience. There is currently renewed interest in linking laboratory research with studies of the acquisition of superior (expert) performance in the clinic.

CONCLUSIONS

Research on medical expertise and simulation training in technical procedures and diagnosis provide exciting opportunities for establishing translational research on the acquisition of superior (expert) performance in the clinic by capturing it with representative tasks in the laboratory, reproducing it for experimental analysis, and developing training activities, such as deliberate practice, that can induce measurable improvements in performance in the clinic.

Clinical case processing: a diagnostic versus a management focus

CONTEXT

Most studies on medical expertise research have focused on diagnostic performance, whereas patient management has been largely ignored. According to knowledge encapsulation theory, applying encapsulated knowledge is a characteristic of expert doctors' diagnostic reasoning, but it is unclear whether or not encapsulated knowledge also plays a prominent role when processing a clinical case with a management focus.

METHODS

The participants were 40 medical students (20 in Year 4 and 20 in Year 6) and 20 expert doctors (internists). Participants were asked to study the cases with either a diagnostic (Dx) or a management (Mx) focus. Subsequently, participants were asked to write down what they remembered from the case.

RESULTS

In both conditions, experts recalled fewer propositions and used more high-level inferences than medical students. Furthermore, they processed the cases faster and more accurately than medical students, but no significant difference between Mx and Dx conditions was found. Year 4 students also showed no significant differences in recall and processing speed between conditions. By contrast, Year 6 students recalled more in a Dx than in an Mx condition, but there was no significant difference in processing speed between conditions.

CONCLUSIONS

In both conditions, findings indicate that the experts' and Year 4 students' performance was not affected by processing focus. The fact that only Year 6 students were affected by processing focus might be explained by the assumption that their diagnostic knowledge and management knowledge are not fully integrated yet, a process that has already taken place in the expert's knowledge structure.

Breaking down automaticity: case ambiguity and the shift to reflective approaches in clinical reasoning

CONTEXT

Two modes of case processing have been shown to underlie diagnostic judgements: analytical and non-analytical reasoning. An optimal form of clinical reasoning is suggested to combine both modes. Conditions leading doctors to shift from the usual mode of non-analytical reasoning to reflective reasoning have not been identified. This paper reports a study aimed at exploring these conditions by investigating the effects of ambiguity of clinical cases on clinical reasoning.

METHODS

Participants were 16 internal medicine residents in the Brazilian state of Ceará. They were asked to diagnose 20 clinical cases and recall case information. The independent variable was the degree of ambiguity of clinical cases, with 2 levels: straightforward (i.e. non-ambiguous) and ambiguous. Dependent variables were processing time, diagnostic accuracy and proposition per category recalled. Data were analysed using a repeated measures design.

RESULTS

Participants processed straightforward cases faster and more accurately than ambiguous ones. The proportion of text propositions recalled was significantly lower (t[15] = 2.29, P = 0.037) in ambiguous cases, and an interaction effect between case version and proposition category was also found (F[5, 75] = 4.52, P = 0.001, d = 0.232, observed power = 0.962). Furthermore, participants recalled significantly more literal propositions from the ambiguous cases than from the straightforward cases (t[15] = 2.28, P = 0.037).

CONCLUSIONS

Ambiguity of clinical cases was shown to lead residents to switch from automatic to reflective reasoning, as indicated by longer processing time, and more literal propositions recalled in ambiguous cases.

Cognitive metaphors of expertise and knowledge: prospects and limitations for medical education

CONTEXT

Many approaches to the study of expertise in medical education have their roots most strongly established in the traditional cognitive psychology literature. As such, they take a common approach to the construction of expertise and frame their questions in a common way. This paper reflects on a few of the paradigmatic assumptions that have 'come along for the ride' with the traditional cognitive approach, and explores what might have been left out as a consequence.

METHODS

We examine the operational definition of 'expert' as it has evolved using the traditional cognitive paradigm and we explore some alternative definitions and constructions of expert performance that have arisen in parallel education research paradigms. We address 3 inter-related aspects of expertise as manifested in the traditional cognitive approach: the construction of the expert as a (routine) diagnostician; the construction of the developmental process as the (automatic and un-reflective) accrual of resources through experience, and the construction of accrued knowledge as a relatively static resource that is subsequently used and built upon with further experience.

CONCLUSIONS

We hope that, by highlighting these issues, we may begin to marry the strengths of the traditional cognitive paradigm with the strengths of these other paradigms and expand the scope of cognitive research in medical expertise.

Surgical education to improve the quality of patient care: the role of practice-based learning and improvement

Health care is going through immense change, and concerns regarding the quality of patient care and patient safety continue to be expressed in many national forums. A variety of stakeholders are demanding greater accountability from the health care profession. Education is key to supporting surgeons' efforts to provide high-quality patient care during these challenging times. Educational programs for surgeons should be founded on principles of continuous professional development (CPD) and practice-based learning and improvement (PBLI). CPD focuses on the specific needs of individual surgeons and involves lifelong learning throughout a surgeon's career. It needs to form the basis of PBLI efforts. PBLI involves a cycle of four steps--identifying areas for improvement, engaging in learning, applying new knowledge and skills to practice, and checking for improvement. Ongoing involvement in PBLI activities to address specific learning needs should positively impact a surgeon's practice and improve outcomes of surgical care.

Safe introduction of new procedures and emerging technologies in surgery: education, credentialing, and privileging

Ongoing horizon scanning is needed to identify new procedures and emerging technologies that should be evaluated for introduction into surgical practice. Following evidence-based evaluation, if a new modality is found ready for adoption in practice, surgeons need education in the safe and effective use of the new modality. The educational experience should include structured teaching and learning, verification of new knowledge and skills, preceptoring or proctoring, and monitoring of outcomes. Credentialing and privileging to perform a new procedure or use an emerging technology should be based on evaluation of knowledge and skills and outcomes of surgical care, and not merely on the numbers of procedures performed. Education of the surgical team is also essential. The entire process involving education, verification of knowledge and skills, credentialing, and privileging must be transparent. Patients need to play a central role in making informed decisions regarding their care that involves use of a new procedure or an emerging technology, and they should participate actively in their perioperative care.

A review of research into the development of radiologic expertise: implications for computer-based training

RATIONALE AND OBJECTIVES

Studies of radiologic error reveal high levels of variation between radiologists. Although it is known that experts outperform novices, we have only limited knowledge about radiologic expertise and how it is acquired.

MATERIALS AND METHODS

This review identifies three areas of research: studies of the impact of experience and related factors on the accuracy of decision-making; studies of the organization of expert knowledge; and studies of radiologists' perceptual processes.

RESULTS AND CONCLUSION

Interpreting evidence from these three paradigms in the light of recent research into perceptual learning and studies of the visual pathway has a number of conclusions for the training of radiologists, particularly for the design of computer-based learning programs that are able to illustrate the similarities and differences between diagnoses, to give access to large numbers of cases and to help identify weaknesses in the way trainees build up a global representation from fixated regions.

Demystifying "learning" in clinical rotations: do immersive patient encounters predict achievement on the clinical performance examination (CPX)?

BACKGROUND

Although the Liaison Committee on Medical Education required U.S. medical schools to quantify students' clinical encounters, optimum patient exposure needed to predict performance has been elusive. This study explored the relationship between comprehensive patient encounters logged on personal digital assistants (PDAs) during three medicine clerkships, to performance on a clinical performance examination (CPX).

METHOD

PDA log data for 166 medical students were used to identify educationally "rich" patient encounters, where students were assigned full responsibility for patient care.

RESULTS

Univariate regression analyses predicting the effect of immersive patient encounters on CPX case scores did not show statistical significance.

CONCLUSIONS

Amount of patient exposure defined by the richness of student-patient interaction did not reflect on the CPX performance for six selected cases. Further research should examine qualitatively different learning experiences occurring with patient encounters and a higher volume of exposure to predict outcomes.

A Theory-Based Curriculum for Enhancing Surgical Skillfulness

BACKGROUND

Curricula for surgical technical skills laboratories have traditionally been designed to accommodate the clinical activities of residents, so they typically consist of individual, episodic training sessions. We believe that the skills laboratory offers an opportunity to design a surgical skills curriculum based on the fundamental elements known to be important for motor skill instruction. We hypothesized that training novices with such a curriculum for a 1-month period would yield skills performance levels equivalent to those of second year surgery residents who had trained in a traditional program.

STUDY DESIGN

Fourth-year medical students served as study subjects (novice group) during a 4-week senior elective. They were taught each skill during a 1-week period. Subjects received instruction by a content expert followed by a 1-week period of deliberate practice with feedback. The novice performances were videotaped both before and after the intervention, and each videotape was evaluated in a blinded fashion by experts using a validated evaluation instrument. These results were compared with skill performance ratings of first- and second-year surgery residents that had been accumulated over the previous 3 years.

RESULTS

Average performance ratings for the novices substantially improved for all four skills after training. There was no marked difference between average performance ratings of postintervention novice scores when compared with the average scores in the resident group. Inter-rater agreement in scoring for the videotaped novice performances exceeded 0.87 (intraclass correlation) for all ratings of pre- and posttraining.

CONCLUSIONS

These results demonstrate the effectiveness of a laboratory-based training program that includes fundamentals of motor skills acquisition.

A cost-effective proficiency-based knot-tying and suturing curriculum for residency programs

BACKGROUND

The purpose of this study was to develop a structured open skills curriculum for knot-tying and suturing using expert-derived performance goals and to examine its feasibility, cost-effectiveness, and construct validity.

METHODS

Using commercially available bench models, 11 standardized tasks (ranging from 2-handed knot-tying to running subcuticular closure) were developed and scored using previously validated metrics based on time and errors. Expert performance was used to establish training endpoints and to create a video tutorial. PGY 1 residents (n = 4) were enrolled in a prospective Institutional Review Board-approved pilot study that included proctored orientation and baseline testing, self-training to proficiency, and proctored post-testing (conducted over a 4-wk period). Baseline trainee scores were compared with expert scores to evaluate construct validity.

RESULTS

The 11 tasks proved relatively robust, and excellent feedback was obtained from the trainees regarding educational benefit. Overall, trainees performed 144 +/- 33 repetitions over 11 +/- 2 h. Trainees achieved proficiency for 4.6% of the 11 tasks at baseline, 91% during training, and 84% at post-testing. Trainees demonstrated significant improvement from baseline to post-testing, validating skill acquisition; baseline trainee and expert performance were significantly different, confirming construct validity. Curriculum development cost $1200 and required 72 man-hours. Incremental training cost less than $12 per participant and required 8 man-hours per rotation using the video-based self-practice curriculum. In response to participant feedback, two of the 11 tasks were modified and a twelfth task was added.

CONCLUSIONS

This curriculum is cost-effective, feasible within the context of residency training, educationally beneficial, and demonstrates construct validity. More widespread adoption of standardized, validated skills curricula such as this by residency programs is warranted.

Complexity, risk and simulation in learning procedural skills

BACKGROUND

A complex chain of events underpins every clinical intervention, especially those involving invasive procedures. Safety requires high levels of awareness and vigilance. In this paper we propose a structured approach to procedural training, mapping each learner's evolving experience within a matrix of clinical risk and procedural complexity. We use a traffic light analogy to conceptualize a dynamic awareness of prevailing risk and the implications of moving between zones.

THE IMPORTANCE OF CONTEXT

We argue that clinical exposure can be consolidated by simulation where appropriate, ensuring that each learner gains the skills for safe care within the increasingly limited time available for training. To be effective, however, such simulation must be realistic, patient-focused, structured and grounded in an authentic clinical context. Challenge comes not only from technical difficulty but also from the need for interpersonal skills and professionalism within clinical encounters.

PATIENT FOCUSED SIMULATION

Many existing simulations focus on crises, so clinicians are in a heightened state of expectation that may not reflect their usual practice. We argue that simulation should also reflect commonly occurring non-crisis situations, allowing clinicians to develop an awareness of the complex events that underpin clinical encounters. We describe a patient-focused approach to simulation, using simulated patients and inanimate models within realistic scenarios, to ground experience in authentic clinical practice and bring together the complex elements that underpin clinical events.

APPLICATIONS

Although our argument has evolved from surgical practice and operating theatre teams, we believe it can be widely applied to the increasing number of health care professionals who perform clinical interventions.

Limited feedback and video tutorials optimize learning and resource utilization during laparoscopic simulator training

BACKGROUND

The purpose of this study was to determine the impact of instructor feedback and video tutorials on skill acquisition during proficiency-based laparoscopic suturing training.

METHODS

Performance data from a prospectively maintained database were reviewed for three groups of novices (n = 34 medical students) who completed the same proficiency-based laparoscopic suturing curriculum on a Fundamentals of Laparoscopic Surgery-type videotrainer model as part of two separate institutional review board-approved, randomized controlled trials. Group I (n = 9) watched the video tutorial once and received intense feedback during each training session; Group II (n = 13) watched the video tutorial once and received limited feedback (<10 min per session); Group III (n = 12) watched the video tutorial several times and also received limited feedback (<10 min per session). Feedback was given by the same instructor and was quantified on a 0 (none) to 4 (extensive) Likert scale.

RESULTS

Baseline characteristics were similar for all groups. All participants achieved the proficiency level (512) on two consecutive attempts. Group III required the shortest training time and number of repetitions to reach proficiency, with statistically significant differences compared with Group I (P < 0.02). This strategy led to a cost savings of $139 per trainee.

CONCLUSIONS

Limited instructor feedback appears to be superior to intense feedback during proficiency-based laparoscopic simulator training. Coupled with video tutorials, this type of feedback may accelerate learning and improve resource utilization by minimizing the need for instructor involvement.

The reliability of summative judgements based on objective structured clinical examination cases distributed across the clinical year

CONTEXT

Objective structured clinical examinations (OSCEs) can be used for formative and summative evaluation. We sought to determine the generalisability of students' summary scores aggregated from formative OSCE cases distributed across 5 clerkships during Year 3 of medical school.

METHODS

Five major clerkships held OSCEs with 2-4 cases each during their rotations. All cases used 15-minute student-standardised patient encounters and performance was assessed using clinical and communication skills checklists. As not all students completed every clerkship or OSCE case, the generalisability (G) study was an unbalanced student x (case : clerkship) design. After completion of the G study, a decision (D) study was undertaken and phi (phi) values for different cut-points were calculated.

RESULTS

The data for this report were collected over 2 academic years involving 262 Year 3 students. The G study found that 9.7% of the score variance originated from the student, 3.1% from the student-clerkship interaction, and 87.2% from the student-case nested within clerkship effect. Using the variance components from the G study, the D study suggested that if students completed 3 OSCE cases in each of the 5 different clerkships, the reliability of the aggregated scores would be 0.63. The phi, calculated at a cut-point 1 standard deviation below the mean, would be approximately 0.85.

CONCLUSIONS

Aggregating case scores from low stakes OSCEs within clerkships results in a score set that allows for very reliable decisions about which students are performing poorly. Medical schools can use OSCE case scores collected over a clinical year for summative evaluation.

Patient safety curriculum for surgical residency programs: results of a national consensus conference

BACKGROUND

The American College of Surgeons (ACS) and the Accreditation Council for Graduate Medical Education (ACGME) are committed to promoting patient safety through education. In view of the critical role of residents in the delivery of safe patient care, the ACS and ACGME sponsored jointly a national consensus conference to initiate the development of a curriculum on patient safety that may be used across all surgical residency programs.

CONCLUSIONS

National leaders in surgery with expertise in surgical care and surgical education, patient safety experts, medical educators, key stakeholders from national organizations, and surgical residents were invited to participate in the conference. Attendees considered patient safety issues within the context of the 6 core competencies defined by the ACGME and American Board of Medical Specialties (ABMS). Discussions resulted in the development of a curriculum matrix that includes listings of patient safety topics, teaching and learning strategies, and assessment methods. Guidelines for implementation and dissemination are also provided. The curriculum content underscores the need to create an organizational culture of safety and focuses on both individuals and systems. Individual residency programs may prioritize the curriculum content based on their specific needs. The ACS and ACGME will pursue development of educational modules to address the curriculum content, disseminate helpful information, and assist in implementation of new educational interventions. This effort has the potential to positively impact residency education in surgery, help surgical program directors address the core competencies, and enhance patient safety.

Debriefing in the intensive care unit: a feedback tool to facilitate bedside teaching

OBJECTIVE

To develop an assessment tool for bedside teaching in the intensive care unit (ICU) that provides feedback to residents about their performance compared with clinical best practices.

METHOD

We reviewed the literature on the assessment of resident clinical performance in critical care medicine and summarized the strengths and weaknesses of these assessments. Using debriefing after simulation as a model, we created five checklists for different situations encountered in the ICU--areas that encompass different Accreditation Council for Graduate Medical Education core competencies. Checklists were designed to incorporate clinical best practices as defined by the literature and institutional practices as defined by the critical care professionals working in our ICUs. Checklists were used at the beginning of the rotation to explicitly define our expectations to residents and were used during the rotation after a clinical encounter by the resident and supervising physician to review a resident's performance and to provide feedback to the resident on the accuracy of the resident's self-assessment of his or her performance.

RESULTS

Five "best practice" checklists were developed: central catheter placement, consultation, family discussions, resuscitation of hemorrhagic shock, and resuscitation of septic shock. On average, residents completed 2.6 checklists per rotation. Use of the cards was fairly evenly distributed, with the exception of resuscitation of hemorrhagic shock, which occurs less frequently than the other encounters in the medical ICU. Those who used more debriefing cards had higher fellow and faculty evaluations. Residents felt that debriefing cards were a useful learning tool in the ICU.

CONCLUSIONS

Debriefing sessions using checklists can be successfully implemented in ICU rotations. Checklists can be used to assess both resident performance and consistency of practice with respect to published standards of care in critical care medicine.

Avoiding complications of laparoscopic surgery: Lessons from cognitive science and crew resource management

Complications may occur during laparoscopic surgery, even with a skilled surgeon and under ideal circumstances, because human error is inevitable. We reviewed videotaped procedures where malpractice was alleged and resolved in court to evaluate potential contributing factors. We evaluated possible sources of complications related to cognitive science, systems error, equipment issues, and surgeon selection and training. The way the human brain's hard-wiring shapes information processing, as well as perceptual learning, can influence the risk of surgical complications. Situation awareness and principles derived from aviation crew resource management may be adapted to help avoid systems error. The current process used to select gynecologic surgeons and the structure of surgical training may need to be reconsidered.

Development of a web-based, specialty specific portfolio

BACKGROUND

This article illustrates the creation of a specialty specific portfolio that can be used by several different residency programs to document resident competence during a given rotation.

METHODS

Three different disciplines (anesthesiology, surgery and medicine) worked together to create a critical care medicine portfolio. We began by reviewing the curriculum requirements for critical care medicine and organized these requirements into the six ACGME core competencies. We then developed learner led exercises in each core competency that were specific to critical care. Each exercise includes assessment of resident knowledge and application, an evaluation of the exercise, a learner self-assessment of skill, and a review of performance by a faculty member. Portfolio entries are highlighted in a multi-disciplinary weekly conference and posted on a critical care web site at our University.

CONCLUSIONS

Creation of specialty specific portfolio reduces redundancy between disciplines, allows for increased time to be spent on the development of exercises specific to rotation objectives, and aids program directors in the collection of portfolio entries for each resident over the course of a residency.

Improving diagnostic capabilities of medical students via application of cognitive sciences-derived learning principles

PURPOSE

There is limited experimental evidence concerning how best to train students to perform differential diagnosis. We compared 2 different methods for training 2nd-year medical students to perform differential diagnosis (DDX) of heart failure: a traditional classroom-based lecture (control group) versus a cognitive sciences-based approach to DDX instruction implemented through a computer-based tutor (treatment group).

METHODS

Following random assignment to either group, students were trained for 75 minutes, and then given a 40-item examination comprised of cases that varied along a typicality gradient from prototypical (easy) to less typical (hard).

RESULTS

The treatment group diagnosed correctly significantly more test cases than the control group (74% versus 60%, respectively). The treatment group also diagnosed correctly significantly more cases at the extremes of the typicality gradient: 81% versus 65%, respectively, for the prototypical cases; 65% versus 48%, respectively, for the most difficult cases.

CONCLUSION

The ability to perform differential diagnosis is enhanced by training based upon principles of cognitive sciences.

The use of simulation in emergency medicine: a research agenda

Medical simulation is a rapidly expanding area within medical education. In 2005, the Society for Academic Emergency Medicine Simulation Task Force was created to ensure that the Society and its members had adequate access to information and resources regarding this new and important topic. One of the objectives of the task force was to create a research agenda for the use of simulation in emergency medical education. The authors present here the consensus document from the task force regarding suggested areas for research. These include opportunities to study reflective experiential learning, behavioral and team training, procedural simulation, computer screen-based simulation, the use of simulation for evaluation and testing, and special topics in emergency medicine. The challenges of research in the field of simulation are discussed, including the impact of simulation on patient safety. Outcomes-based research and multicenter efforts will serve to advance simulation techniques and encourage their adoption.

Analysis of Simulated Angiographic Procedures. Part 2: Extracting Efficiency Data from Audio and Video Recordings

PURPOSE

To create and test methods of extracting efficiency data from recordings of simulated renal stent procedures.

MATERIALS AND METHODS

Task analysis was performed and used to design a standardized testing protocol. Five experienced angiographers then performed 16 renal stent simulations using the Simbionix AngioMentor angiographic simulator. Audio and video recordings of these simulations were captured from multiple vantage points. The recordings were synchronized and compiled. A series of efficiency metrics (procedure time, contrast volume, and tool use) were then extracted from the recordings. The intraobserver and interobserver variability of these individual metrics was also assessed. The metrics were converted to costs and aggregated to determine the fixed and variable costs of a procedure segment or the entire procedure.

RESULTS

Task analysis and pilot testing led to a standardized testing protocol suitable for performance assessment. Task analysis also identified seven checkpoints that divided the renal stent simulations into six segments. Efficiency metrics for these different segments were extracted from the recordings and showed excellent intra- and interobserver correlations. Analysis of the individual and aggregated efficiency metrics demonstrated large differences between segments as well as between different angiographers. These differences persisted when efficiency was expressed as either total or variable costs.

CONCLUSIONS

Task analysis facilitated both protocol development and data analysis. Efficiency metrics were readily extracted from recordings of simulated procedures. Aggregating the metrics and dividing the procedure into segments revealed potential insights that could be easily overlooked because the simulator currently does not attempt to aggregate the metrics and only provides data derived from the entire procedure. The data indicate that analysis of simulated angiographic procedures will be a powerful method of assessing performance in interventional radiology.

Redefining simulator proficiency using automaticity theory

BACKGROUND

Automaticity is a characteristic of expertise defined by the ability to perform a task without significant demands on attention. Our objective was to assess whether a visual-spatial task that measures spare attentional capacity would distinguish among individuals with different levels of laparoscopic expertise.

METHODS

The performance of novices (n = 10), surgery residents (n = 9), laparoscopy experts (n = 3), and individuals previously trained (n = 7) to proficiency in laparoscopic suturing on simulators but without operative experience (trained individuals) was measured under dual-task conditions. Participants performed laparoscopic suturing for 10 minutes on a video trainer simulator using the Fundamentals of Laparoscopic Surgery suturing model (primary task) while at the same time they responded to a visual-spatial secondary task.

RESULTS

Experts and trained individuals outperformed both residents and novices on the suturing task (P < .001). Although the performance of experts and trained individuals did not differ significantly based on suturing scores, experts achieved higher secondary-task scores (P < .05).

CONCLUSIONS

A visual-spatial secondary task that assesses spare attentional capacity may help distinguish among individuals of variable laparoscopic expertise when standard performance measures fail to do so. Such automaticity metrics may improve current simulator training and assessment methods and warrants further investigation.

Surgical skill acquisition with self-directed practice using computer-based video training

BACKGROUND

Computer-based video training (CBVT) provides flexible opportunities for surgical trainees to learn fundamental technical skills, but may be ineffective in self-directed practice settings because of poor trainee self-assessment. This study examined whether CBVT is effective in a self-directed learning environment among novice trainees.

METHODS

Thirty novice trainees used CBVT to learn the 1-handed square knot while self-assessing their proficiency every 3 minutes. On reaching self-assessed skill proficiency, trainees were randomized to either cease practice or to complete additional practice. Performance was evaluated with computer and expert-based measures during practice and on pretests, posttests, and 1-week retention tests.

RESULTS

Analyses revealed performance improvements for both groups (all P < .05), but no differences between the 2 groups (all P > .05) on all tests.

CONCLUSIONS

CBVT for the 1-handed square knot is effective in a self-directed learning environment among novices. This lends support to the implementation of self-directed digital media-based learning within surgical curricula.

A randomized comparison trial of case-based learning versus human patient simulation in medical student education

OBJECTIVES

Human patient simulation (HPS), utilizing computerized, physiologically responding mannequins, has become the latest innovation in medical education. However, no substantive outcome data exist validating the advantage of HPS. The objective of this study was to evaluate the efficacy of simulation training as compared with case-based learning (CBL) among fourth-year medical students as measured by observable behavioral actions.

METHODS

A chest pain curriculum was presented during a one-month mandatory emergency medicine clerkship in 2005. Each month, students were randomized to participate in either the CBL-based or the HPS-based module. All students participated in the same end-of-clerkship chest pain objective structured clinical examination that measured 43 behaviors. Three subscales were computed: history taking, acute coronary syndrome evaluation and management, and cardiac arrest management. Mean total and subscale scores were compared across groups using a multivariate analysis of variance, with significance assessed from Hotelling's T2 statistic.

RESULTS

Students were randomly assigned to CBL (n = 52) or HPS (n = 50) groups. The groups were well balanced after random assignment, with no differences in mean age (26.7 years; range, 22-44 years), gender (male, 52.0%), or emergency medicine preference for specialty training (28.4%). Self-ratings of learning styles were similar overall: 54.9% were visual learners, 7.8% auditory learners, and 37.3% kinetic learners. Results of the multivariate analysis of variance indicated no significant effect (Hotelling's T2 [3,98] = 0.053; p = 0.164) of education modality (CBL or HPS) on any subscale or total score difference in performance.

CONCLUSIONS

HPS training offers no advantage to CBL as measured by medical student performance on a chest pain objective structured clinical examination.