A Mixed-Methods Exploration of the Educational Climate and Safety Climate During the First Year of Clinical Practice in Ireland

An assessment of the simulated performance of basic clinical procedures by junior doctors during the first year of clinical practice

BACKGROUND

Upon entering the healthcare system, junior doctors may lack the skills required to care for patients, and feel unprepared for their role, with considerable variation in the level of proficiency in the performance of particular clinical procedures.

OBJECTIVE

To compare the performance and proficiency (self-report and observed) of the performance of nine basic clinical procedures.

METHODS

Seventeen interns were observed performing nine clinical procedures in a simulated setting in June 2021 (Assessment 1) and January 2022 (Assessment 2). The observers identified whether each step in the procedure was performed correctly, and provided an overall assessment of proficiency. The participants also rated their own level proficiency.

RESULTS

At Assessment 1 the number of steps performed correctly ranged from a mean of 41.9-83.5%. At Assessment 2 the number of steps performed correctly ranged from a mean of 41.9-97.8%. The most common median proficiency rating for Assessment 1 was 'close supervision', and was 'indirect supervision' at Assessment 2. There was a significant and large effect size in the improvement in performance from Assessment 1 to Assessment 2. Low correlations were found between observer and self-reported proficiency in performance of the procedures.

CONCLUSIONS

The large improvement in performance across the two assessments is encouraging. However, there is a need to address the variability in performance on graduation from medical school, and to ensure that any assessment of proficiency is not only reliant on self-report.

Longitudinal survey of self-reported level of entrustment across the first year of clinical practice

Background. Newly graduated medical students often report that they lack the skills required to care for patients, and feel unprepared for clinical practice. However, little is known about when, and if, they acquire these skills in practice. The aim of this study was to assess self-reported level of entrustment in, and frequency of performance of, the seven Entrustable Professional Activities (EPAs) comprising the EPA framework for interns in Ireland. EPAs describe discrete activities that are essential to a particular profession. Methods. Self-report survey data were collected from doctors in the Republic of Ireland during their first year of clinical practice at four different time points during 2020/21. Results. Response rates to the survey varied from 73.3% (126/172) at Time 1 to 25.6% (44/172) at Time 4. After three months, the respondents reported that they could execute all of the EPAs, inclusive of 12 essential procedural skills, with indirect supervision. As the year progressed there was an increase in the proportion of respondents reporting that they performed the EPAs at least once a week. However, the proportion of respondents performing five of the essential procedural skills (e.g. nasogastric tube insertion) remained low across all time points. Conclusion. Consideration should be given as to how to better prepare medical students to execute these EPAs, how the interns can be better supported during the first quarter of internship. The findings from this research are positive. However, there is an urgent need to carry out formal assessments of entrustability, rather than relying on self-report.

Patients' perception of safety climate in Irish general practice: a cross-sectional study

BACKGROUND

Although patients have the potential to provide important information on patient safety, considerably fewer patient-report measures of safety climate (SC) have been applied in the primary care setting as compared to secondary care. Our aim was to examine the application of a patient-report measure of safety climate in an Irish population to understand patient perceptions of safety in general practice and identify potential areas for improvement. Specifically, our research questions were: 1. What are patients' perceptions of SC in Irish general practice? 2. Do patient risk factors impact perceptions of SC? 3. Do patient responses to an open-ended question about safety enhance our understanding of patient safety beyond that obtained from a quantitative measure of SC?

METHODS

The Patient Perspective of Safety in General Practice (PPS-GP) survey was distributed to primary care patients in Ireland. The survey consisted of both Likert-response items, and free-text entry questions in relation to the safety of care. A series of five separate hierarchical regressions were used to examine the relationship between a range of patient-related variables and each of the survey subscales. A deductive content analysis approach was used to code the free-text responses.

RESULTS

A total of 584 completed online and paper surveys were received. Respondents generally had positive perceptions of safety across all five SC subscales of the PPS-GP. Regarding patient risk factors, younger age and being of non-Irish nationality were consistently associated with more negative SC perceptions. Analysis of the free-text responses revealed considerably poorer patient perceptions (n = 85, 65.4%) of the safety experience in primary care.

CONCLUSION

Our findings indicate that despite being under-utilised, patients' perceptions are a valuable source of information for measuring SC, with promising implications for safety improvement in general practice. Further consideration should be given to how best to utilise this data in order to improve safety in primary care.

The development of a framework of entrustable professional activities for the intern year in Ireland

BACKGROUND

Entrustable Professional Activities (EPAs) are units of professional practice that capture essential competencies in which trainees must become proficient before undertaking them independently. EPAs provide supervisors with a solid justification for delegating an activity to trainees. This study aimed to develop and ensure face validity of a set of EPAs for junior doctors in the first year of clinical practice in the Republic of Ireland.

METHODS

An iterative eight stage consensus building process was used to develop the set of EPAs. This process was based on international best practice recommendations for EPA development. A series of surveys and workshops with stakeholders was used to develop a framework of EPAs and associated competencies. An external stakeholder consultation survey was then conducted by the Irish Medical Council. The framework of EPAs was then benchmarked against the 13 core EPAs developed by the Association of American Medical Colleges (AAMC).

RESULTS

A framework of seven EPAs, and associated competencies resulted from this study. These EPAs address all core activities that junior doctors should be readily entrusted with at the end of the intern year, which is the first year of clinical practice in the Republic of Ireland. Each EPA contains a series of defined competencies. The final EPAs were found to be comparable to the AAMC core EPAs for entering residency.

CONCLUSIONS

A framework of EPAs for interns in Ireland that are appropriate for the intern year has been developed by key stakeholders. The implementation of the EPAs in practice is the next step, and is likely to result in an improved intern training process and increased patient safety.