Early Detection and Remediation of Problem Learners

There are myriad types of problem learners in surgical residency and most have difficulty in more than 1 competency. Programs that use a standard curriculum of study and assessment are most successful in identifying struggling learners early. Many problem learners lack appropriate systems for study; a multidisciplinary educational team that is separate from the team that evaluates the success of remediation is critical. Struggling residents who require formal remediation benefit from performance improvement plans that clearly outline the issues of concern, describe the steps required for remediation, define success of remediation, and outline consequences for failure to remediate appropriately.

Guidelines: The dos, don'ts and don't knows of remediation in medical education

INTRODUCTION

Two developing forces have achieved prominence in medical education: the advent of competency-based assessments and a growing commitment to expand access to medicine for a broader range of learners with a wider array of preparation. Remediation is intended to support all learners to achieve sufficient competence. Therefore, it is timely to provide practical guidelines for remediation in medical education that clarify best practices, practices to avoid, and areas requiring further research, in order to guide work with both individual struggling learners and development of training program policies.

METHODS

Collectively, we generated an initial list of Do's, Don'ts, and Don't Knows for remediation in medical education, which was then iteratively refined through discussions and additional evidence-gathering. The final guidelines were then graded for the strength of the evidence by consensus.

RESULTS

We present 26 guidelines: two groupings of Do's (systems-level interventions and recommendations for individual learners), along with short lists of Don'ts and Don't Knows, and our interpretation of the strength of current evidence for each guideline.

CONCLUSIONS

Remediation is a high-stakes, highly complex process involving learners, faculty, systems, and societal factors. Our synthesis resulted in a list of guidelines that summarize the current state of educational theory and empirical evidence that can improve remediation processes at individual and institutional levels. Important unanswered questions remain; ongoing research can further improve remediation practices to ensure the appropriate support for learners, institutions, and society.

Successful Remediation of an Advanced Pharmacy Practice Experience for an At-risk Student

Objective. To describe a successful remediation for an Acute Care Medicine advanced pharmacy practice experience (APPE) in a student with multiple learning deficits. Methods. A literature review of pharmacy and medical experiential remediation was conducted to identify best practices to implement prior to designing the remediation for our student case. Based on this search and experience as preceptors, a three-phase remediation was designed: one week for assessment, two weeks for development of learning skills and strategies and six weeks for an on-campus APPE. Success of the remediation was determined by student performance, as defined by the APPE preceptor, in all relevant 2013 Center for the Advancement of Pharmacy Education (CAPE) educational outcomes. Results. Baseline assessment indicated that the student was below minimal competency in six of 13 relevant 2013 CAPE educational outcomes. Upon completion of the three-part remediation, the student repeated the Acute Care Medicine APPE, achieving better than minimal competency in all 13 outcomes. The student demonstrated significant improvement in nine of 13 CAPE educational outcomes. Conclusion. This student case provides a novel and successful blueprint for remediation of APPE. However, more evidence-based literature is needed to guide educators in experiential remediation.

Rethinking Remediation: A Model to Support the Detailed Diagnosis of Clinicians' Performance Problems and the Development of Effective Remediation Plans

The successful remediation of clinicians demonstrating poor performance in the workplace is essential to ensure the provision of safe patient care. Clinicians may develop performance problems for numerous reasons, including health, personal factors, the workplace environment, or outdated knowledge/skills. Performance problems are often complex involving multifactorial issues, encompassing knowledge, skills, and professional behaviors. It is important that (where possible and appropriate) clinicians are supported through effective remediation to return them to safe clinical practice. A review of the literature demonstrated that research into remediation is in its infancy, with little known about the effectiveness of remediation programs currently. Current strategies for the development of remediation programs are mostly "intuitive"; a few draw upon established theories to inform their approach. Similarly, although it has been established that identification of the nature/scope of performance problems through assessment is an essential first step within remediation, the need for a more widespread "diagnosis" of why the problems exist is emerging. These reasons for poor performance, particularly in the context of experienced practicing clinicians, are likely to have an impact on the potential success of remediation and should be considered within the "diagnosis." A new model for diagnosing the performance problems of the clinicians has been developed, using behavioral change theories to explore known barriers to successful remediation, such as insight, motivation, attitude, self-efficacy, and the working environment, in addition to addressing known deficits regarding knowledge and skills. This novel approach is described in this article. An initial feasibility study has demonstrated the acceptability and practical implementation of our model.

Predictors of medical student remediation and their underlying causes: early lessons from a curriculum change in the University of Auckland Medical Programme

AIMS

The purpose of this study was to identify predictors of remediation in a medical programme and assess the underlying causes and the quality of remediation provided within the context of a recent curriculum change.

METHODS

A mixed methods study incorporating a retrospective cohort analysis of demographic predictors of remediation during 2013 and 2014, combined with thematic qualitative analysis of educator perspectives derived by interview on factors underlying remediation and the quality of that currently provided by the faculty.

RESULTS

17.7% of all students required some form of remedial assistance and 93% of all students offered remediation passed their year of study. Multivariate analysis showed international students (OR 4.59 95% CI 2.62-7.98) and students admitted via the Māori and Pacific Admission Scheme (OR 3.43 2.29-5.15) were significantly more likely to require remediation. Male students were also slightly more likely than their female classmates to require assistance. No effect was observed for rural origin students, completion of a prior degree or completion of clinical placement in a peripheral hospital. Knowledge application and information synthesis were the most frequently identified underlying problems. Most faculty believed remediation was successful, however, flexibility in the programme structure, improved diagnostics and improved access to dedicated teaching staff were cited as areas for improvement.

CONCLUSIONS

Remediation is required by nearly a fifth of University of Auckland medical students, with MAPAS and international students being particularly vulnerable groups. Remediation is largely successful, however, interventions addressing reasoning and knowledge application may improve its effectiveness.

How Do Medical Schools Identify and Remediate Professionalism Lapses in Medical Students? A Study of U.S. and Canadian Medical Schools

PURPOSE

Teaching and assessing professionalism is an essential element of medical education, mandated by accrediting bodies. Responding to a call for comprehensive research on remediation of student professionalism lapses, the authors explored current medical school policies and practices.

METHOD

In 2012-2013, key administrators at U.S. and Canadian medical schools accredited by the Liaison Committee on Medical Education were interviewed via telephone or e-mail. The structured interview questionnaire contained open-ended and closed questions about practices for monitoring student professionalism, strategies for remediating lapses, and strengths and limitations of current systems. The authors employed a mixed-methods approach, using descriptive statistics and qualitative analysis based on grounded theory.

RESULTS

Ninety-three (60.8%) of 153 eligible schools participated. Most (74/93; 79.6%) had specific policies and processes regarding professionalism lapses. Student affairs deans and course/clerkship directors were typically responsible for remediation oversight. Approaches for identifying lapses included incident-based reporting and routine student evaluations. The most common remediation strategies reported by schools that had remediated lapses were mandated mental health evaluation (74/90; 82.2%), remediation assignments (66/90; 73.3%), and professionalism mentoring (66/90; 73.3%). System strengths included catching minor offenses early, emphasizing professionalism schoolwide, focusing on helping rather than punishing students, and assuring transparency and good communication. System weaknesses included reluctance to report (by students and faculty), lack of faculty training, unclear policies, and ineffective remediation. In addition, considerable variability in feedforward processes existed between schools.

CONCLUSIONS

The identified strengths can be used in developing best practices until studies of the strategies' effectiveness are conducted.

Examining the predictive validity of a dynamic assessment of decoding to forecast response to tier 2 intervention

The purpose of this study was to examine the role of a dynamic assessment (DA) of decoding in predicting responsiveness to Tier 2 small-group tutoring in a response-to-intervention model. First grade students (n = 134) who did not show adequate progress in Tier 1 based on 6 weeks of progress monitoring received Tier 2 small-group tutoring in reading for 14 weeks. Student responsiveness to Tier 2 was assessed weekly with word identification fluency (WIF). A series of conditional individual growth curve analyses were completed that modeled the correlates of WIF growth (final level of performance and growth). Its purpose was to examine the predictive validity of DA in the presence of three sets of variables: static decoding measures, Tier 1 responsiveness indicators, and prereading variables (phonemic awareness, rapid letter naming, oral vocabulary, and IQ). DA was a significant predictor of final level and growth, uniquely explaining 3% to 13% of the variance in Tier 2 responsiveness depending on the competing predictors in the model and WIF outcome (final level of performance or growth). Although the additional variances explained uniquely by DA were relatively small, results indicate the potential of DA in identifying Tier 2 nonresponders.

Prediction and prevention of failure: an early intervention to assist at-risk medical students

BACKGROUND

Consistent identification and prevention of failure for at-risk medical students is challenging, failing courses is costly to all stakeholders, and there is need for further research into duration, timing and structure of interventions to help students in difficulty.

AIMS

To verify the value of a new exam two weeks into medical school as a predictor of failure, and explore the requirements for a preventative intervention.

METHODS

Students who failed the two-week exam were invited to a series of large-group workshops and small-group follow-up meetings. Participants' subsequent exam performance was compared with non-participants.

RESULTS

About 71% of students who performed poorly in the new exam subsequently failed a course. Attendance at the workshops made no difference to short- or long-term pass rates. Attendance at more than three follow-up small group sessions significantly improved pass rates two semesters later, and was influenced by teacher experience.

CONCLUSIONS

Close similarity between predictor task and target task is important for accurate prediction of failure. Consideration should be given to dose effect and class size in the prevention of failure of at-risk students, and we recommend a systemic approach to intervention/remediation programmes, involving a whole semester of mandatory, weekly small group meetings with experienced teachers.

The remediation challenge: theoretical and methodological insights from a systematic review

OBJECTIVES

Remediation is usually offered to medical students and doctors in training who underperform on written or clinical examinations. However, there is uncertainty and conflicting evidence about the effectiveness of remediation. The aim of this systematic review was to synthesise the available evidence to clarify how and why remediation interventions may have worked in order to progress knowledge on this topic.

METHODS

The MEDLINE, EMBASE, CINAHL (Cumulative Index to Nursing and Allied Health Literature), ERIC (Educational Resources Information Centre), Web of Science and Scopus databases were searched for papers published from 1984 to April 2012, using the search terms 'remedial teaching', 'education', 'medical', 'undergraduate'/or 'clinical clerkship'/or 'internship and residency', 'at risk' and 'struggling'. Only studies that included an intervention, then provided retest data, and reported at least one outcome measure of satisfaction, knowledge, skills or effects on patients were eligible for inclusion. Studies of practising doctors were excluded. Data were abstracted independently in duplicate for all items. Coding differences were resolved through discussion.

RESULTS

Thirty-one of 2113 studies met the review criteria. Most studies were published after 2000 (n=24, of which 12 were published from 2009 onwards), targeted medical students (n=22) and were designed to improve performance on an immediately subsequent examination (n=22). Control or comparison groups, conceptual frameworks, adequate sample sizes and long-term follow-up measures were rare. In studies that included long-term follow-up, improvements were not sustained. Intervention designs tended to be highly complex, but their design or reporting did not enable the identification of the active components of the remedial process.

CONCLUSIONS

Most remediation interventions in medical education focus on improving performance to pass a re-sit of an examination or assessment and provide no insight into what types of extra support work, or how much extra teaching is critical, in terms of developing learning. More recent studies are generally of better quality. Rigorous approaches to developing and evaluating remediation interventions are required.

Evolution of a remedial CME course in professionalism: addressing learner needs, developing content, and evaluating outcomes

INTRODUCTION

Scant information is available about the nature of the professional violations resulting in referral of physicians for remedial continuing medical education (CME). The CME program at Case Western Reserve University (CWRU) School of Medicine has developed the Intensive Course in Medical Ethics, Boundaries, and Professionalism (medical ethics course) for physician referrals due to ethical breaches. In this report, the authors present 7 years of data regarding the type of behavior that resulted in course referral as well as information regarding course and outcome evaluation development and participant demographics.

METHODS

The medical ethics course has been designed in consultation with licensure agencies to address the learning needs of physicians with problems in the areas of boundary maintenance and ethics. Teaching methods and outcome evaluations include lectures, case discussions, multiple-choice question tests, skill practice sessions, and writing a reflective essay based on the participants' ethical lapse. Information is also gathered regarding participant demographics, training, and practice characteristics.

RESULTS

Between September 2005 and February 2012, 358 learners participated in the course. The average age was 52 years and 73% were board certified. Of the 269 physicians who wrote a reflective essay, the reasons for referral included prescribing of controlled drugs, sexual boundary issues, providing services to family or friends, not maintaining proper medical records, and billing issues.

DISCUSSION

This report outlines the strategies used by CWRU to develop remedial CME courses using the medical ethics course as an example for course and outcome evaluation development. This is the first report characterizing the type and frequency of the medical ethics violations that result in mandatory participation in remedial CME.

Is short-term remediation after OSCE failure sustained? A retrospective analysis of the longitudinal attainment of underperforming students in OSCE assessments

BACKGROUND

Significant improvements in the delivery of criterion-based assessment techniques have improved confidence in standard setting and assessment quality. However, for underperforming students, a lack of evidence about longitudinal performance of this group poses dilemmas to educators when making decisions about the timing and nature of remediation.

AIM

To investigate the longitudinal performance of the UK undergraduate medical degree students, with a particular focus on comparing the poorly performing students (i.e. those with borderline or failing grades) with the main cohort of students.

METHOD

Over a 5-year period, 3200-student objective structured clinical examination (OSCE) assessments from a single medical school were investigated. A poorly performing subgroup of 125 students was identified and their longitudinal performance in the final 3 years of the undergraduate medical degree analysed.

RESULT

The relative performance of this student group declines across serial OSCEs, despite current methods of 'remediation and retest'.

CONCLUSIONS

This analysis demonstrates that typically students in the poorly performing subgroup achieve only short-term success with traditional remediation and retest models, and critically show an absence of longitudinal improvement. There is a clear need for institutions to develop profiling models that can help identify this student group and develop effective, research led models of remediation.

A remedial intervention linked to a formative assessment is effective in terms of improving student performance in subsequent degree examinations

BACKGROUND

Intervention may help weaker medical students improve their performance. However, the effectiveness of remedial intervention is inconclusive due to small sample sizes in previous studies. We asked: is remedial intervention linked to a formative assessment effective in terms of improving student performance in subsequent degree examinations?

METHODS

This was a retrospective, observational study of anonymous databases of student assessment outcomes. Data were analysed for students due to graduate in the years 2005-2009 (n = 909). Exam performance was compared for students who received remediation versus those who did not. The main outcome measure was summative degree examination marks.

RESULTS

After adjusting for cohort, gender, overseas versus home funding, previous degree and previous performance in the corresponding baseline third year summative exam, students receiving a remedial intervention (after poor performance on a formative objective structured clinical examination and written exams mid-fourth year) were significantly more likely to obtain an improved mark on end-of-fourth year summative written (p = 0.005) and OSCE (p = 0.001) exams compared to those students who did not receive remediation.

CONCLUSION

A remedial intervention linked to poor assessment performance predicted improved performance in later examination. There is a need for prospective studies in order to identify the effective components of remedial interventions.

A literacy tutoring experience for prospective special educators and struggling second graders

This study examined the learning of teacher candidates taking a language arts course in a special-educator preparation program and that of the second graders they tutored in a supervised field component of the course. Teacher candidates' knowledge of literacy instruction was assessed using five knowledge tasks; children were assessed on several measures of basic reading and spelling skills as well as on their knowledge of phonics concepts such as syllable types. Teacher candidates generally had inaccurate perceptions of their knowledge at pretest, but their knowledge improved significantly on all tasks after course instruction. Tutored children improved significantly from pre- to posttest on all assessments. The study suggests that carefully designed literacy coursework with field experiences can benefit both prospective special educators and struggling readers.

Preservice music teachers' predictions, perceptions, and assessment of students with special needs: the need for training in student assessment

The purpose of the current study was to examine preservice teachers' predictions and perceptions of students with special needs' levels of mastery of specific music education concepts and actual grades achieved by these students using alternative assessments and testing accommodations within two subpopulations: students with emotional and/or behavior disorders (EDBD) and students with acute cognitive delays (ACD). The preservice teachers predicted students within the EDBD class would achieve a significantly higher level of mastery of the music concepts than students within the ACD classroom. After the field experience, however, the preservice teachers' perceptions of all students' levels of mastery increased from prediction scores overall. Additionally, preservice teachers were able to execute testing accommodations and implement successful alternative assessments which gave empirical data on the students' levels of mastery of the music education concepts within the curriculum. Implications for music therapists, as consultants in special education, are discussed.

Case study analysis and the remediation of misconceptions about respiratory physiology

Most students enter the physiology classroom with one or more fundamental misconceptions about respiratory physiology. This study examined the prevalence of four respiratory misconceptions and determined the role of case analysis in the remediation of one of them. A case study was used to help students learn about oxygen transport in the blood and a conceptual diagnostic test was used to assess student understanding of the relation between Po(2) and hemoglobin saturation by probing for the corresponding (Sa/Po(2)) misconception. A 36% remediation of the Sa/Po(2) misconception was found to be associated with case analysis. This repair was selective since the frequency of three other respiratory misconceptions was found to be unchanged after classroom instruction about respiratory physiology in lectures and laboratories. Remediation of the Sa/Po(2) misconception before an instructor-led, in-class case review was superficial and temporary. Explanations provided by students who correctly answered the Sa/Po(2) conceptual diagnostic test showed improved conceptual understanding following case analysis. These results suggest that a learning strategy where students actively confront their faulty notions about respiratory physiology is useful in helping them overcome their misconceptions.

Treatment integrity in a home-based pre-reading intervention programme

Treatment integrity is an underexposed issue in the phonological awareness intervention research. The current study assessed the integrity of treatment of the families (N = 32) participating in the experimental condition of a home-based pre-reading intervention study. The participating kindergartners were all genetically at risk for developing dyslexia. Two aspects of treatment integrity, the number of lessons completed (quantity) and the quality of the administration of the programme (observed in a videotaped session), were investigated. The level of treatment integrity turned out to be 66% when completion of all lessons was taken as quantitative criterion, and about 74% when quality of the parent-child interaction was assessed. The two measures could predict the pre-reading skills at the end of kindergarten. Together they accounted for 43% of the variance in this dependent variable. Together with pre-reading scores at the pre-test the total predicted variance was 87%. The number of lessons completed still contributed 12% to the prediction after controlling for pre-test scores. The results indicated that treatment integrity indeed appears to be an important aspect of treatment outcome and should therefore be included in intervention studies.

Tutoring deaf students in higher education: a comparison of baccalaureate and sub-baccalaureate student perceptions

Seventy-three deaf college students completed a survey examining perceptions about tutoring outcomes and emphases, characteristics of tutors, and responsibilities associated with learning through tutoring. The comparisons revealed that while baccalaureate and sub-baccalaureate students have many similar perceptions about tutoring, there are also some striking differences. In particular, as compared to the sub-baccalaureate students, baccalaureate students have a stronger preference for focusing on course content and for working with tutors who actively involve them during the tutoring sessions. In addition, baccalaureate students prefer to decide the focus of the tutoring themselves while sub-baccalaureate students tend to leave the decision to the tutor. The results of the analyses with three scales measuring perceptions of tutoring dimensions are summarized and recommendations for the selection and preparation of tutors, as well as for future research, are provided.

Systematic identification and intervention for reading difficulty: case studies of children with EAL

Literacy underpins education. There is now very widespread concern over standards of literacy for children from multi-cultural backgrounds, who are learning English as a second or subsequent language, and who may have special educational needs. Research evidence suggests that the earlier children's difficulties can be identified, the more effective (and cost-effective) intervention will be, provided that the intervention is tailored to the child's abilities and skills. Nicolson and Fawcett have developed systematic procedures for identifying children at risk for reading difficulty, together with systematic teaching strategies to overcome reading difficulty. In this paper we present case studies of children with EAL (English as an additional language) drawn from a controlled study using computer interventions with secondary school children. Our findings indicate that children with EAL may be more resistant to remediation than some children with learning difficulties. The prognosis is more problematic for children with both EAL and dyslexia.

Mathematics learning disabilities: a view from developmental psychology

U.S. education suffers from shortcomings that put even children possessing adequate intellectual abilities at risk for low mathematics achievements. Consequently, identifying and understanding children whose academic failure is influenced by a genuine learning disability requires a complex "developmental" research agenda. This perspective suggests the use of sensitive research methods--clinical interviews, ethnographies--to examine the development of children's construction of knowledge in the context of schooling. Researchers should consider such factors as the adequacy of classroom instruction, the availability in children of informal knowledge, the role of motivation, the effects of specific interventions, the role and operation of different cognitive processes in constructing mathematical understanding, children's difficulties across different areas of mathematics, and the development of children's thinking throughout the school years.

Chemical dependency in students with and without learning disabilities

To determine if students with learning disabilities (LD) demonstrate a higher frequency of chemical dependency than students without learning disabilities (NLD), a total of 191 adolescents with LD (101 males and 90 females) were given the Substance Abuse Subtle Screening Inventory (SASSI). The sample consisted of 88 students with LD and 103 NLD students between the ages of 12 and 18. The SASSI is an objectively scored self-report inventory that accurately classifies adolescents as chemically dependent (CD) or not chemically dependent (NCD). A significantly higher proportion of students with LD than NLD students were classified as CD. Of the 30 students who were classified as CD, 70% were students with LD. A discriminant analysis indicated that the presence or absence of a learning disability was a better predictor of classification as CD or NCD than gender, ethnicity, age, socioeconomic status, or family composition. The implications for evaluation and educational planning for students with learning disabilities are discussed.

Cognitive remediation in traumatic brain injury: update and issues

Cognitive Remediation (CR) is a relatively new treatment technique for alleviating residual cognitive deficits following traumatic brain injury. This is a promising yet still changing technique. The future of CR as a rational and systematic endeavor requires the incorporation of important new ideas that have been emerging in allied fields. For example, the emerging field of instructional psychology has contributed relevant concepts such as scaffolding, metacognition, and generalization. It is furthermore argued that the issues of awareness, self-concept, and self-efficacy are vital to the process of CR intervention, and an integrative (holistic) approach to the remedial endeavor is thus indicated.