Evaluation of prognostic risk models for postoperative pulmonary complications in adult patients undergoing major abdominal surgery: a systematic review and international external validation cohort study

BACKGROUND

Stratifying risk of postoperative pulmonary complications after major abdominal surgery allows clinicians to modify risk through targeted interventions and enhanced monitoring. In this study, we aimed to identify and validate prognostic models against a new consensus definition of postoperative pulmonary complications.

METHODS

We did a systematic review and international external validation cohort study. The systematic review was done in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines. We searched MEDLINE and Embase on March 1, 2020, for articles published in English that reported on risk prediction models for postoperative pulmonary complications following abdominal surgery. External validation of existing models was done within a prospective international cohort study of adult patients (≥18 years) undergoing major abdominal surgery. Data were collected between Jan 1, 2019, and April 30, 2019, in the UK, Ireland, and Australia. Discriminative ability and prognostic accuracy summary statistics were compared between models for the 30-day postoperative pulmonary complication rate as defined by the Standardised Endpoints in Perioperative Medicine Core Outcome Measures in Perioperative and Anaesthetic Care (StEP-COMPAC). Model performance was compared using the area under the receiver operating characteristic curve (AUROCC).

FINDINGS

In total, we identified 2903 records from our literature search; of which, 2514 (86·6%) unique records were screened, 121 (4·8%) of 2514 full texts were assessed for eligibility, and 29 unique prognostic models were identified. Nine (31·0%) of 29 models had score development reported only, 19 (65·5%) had undergone internal validation, and only four (13·8%) had been externally validated. Data to validate six eligible models were collected in the international external validation cohort study. Data from 11 591 patients were available, with an overall postoperative pulmonary complication rate of 7·8% (n=903). None of the six models showed good discrimination (defined as AUROCC ≥0·70) for identifying postoperative pulmonary complications, with the Assess Respiratory Risk in Surgical Patients in Catalonia score showing the best discrimination (AUROCC 0·700 [95% CI 0·683-0·717]).

INTERPRETATION

In the pre-COVID-19 pandemic data, variability in the risk of pulmonary complications (StEP-COMPAC definition) following major abdominal surgery was poorly described by existing prognostication tools. To improve surgical safety during the COVID-19 pandemic recovery and beyond, novel risk stratification tools are required.

FUNDING

British Journal of Surgery Society.

225 First Contact: A Series of Zoom-Based, Virtual on Call Shifts

Aim

COVID-19 has resulted in reduced exposure to on-call shifts where medical students could increase confidence and proficiency in task prioritisation and decision making. Existing ‘simulated on-calls’ provide a substitute in a controlled environment, however in person teaching has also been limited by COVID-19. Our virtual on-call sessions use ZOOM to replicate the higher-level learning experiences normally conferred by live simulation.

Method

We designed a series of virtual ‘on-calls’ for medical students. Participants were ‘on-call’, receiving ‘bleeps’ which were ‘answered’ by calling a facilitator via ZOOM. The facilitator would roleplay a scenario and the ‘Electronic Patient Record’ (EPR) on Google Forms contained patient notes and observations. Students needed to collect information from the facilitator and document a management plan into the EPR. Participants received ‘bleeps’ of varying complexity, urgency and relevance and were expected to prioritise and triage tasks accordingly. Evaluation was via a pre/post session quiz with separate feedback forms.

Results

23 students from 18 universities participated. Students reported increased confidence in managing on-call scenarios, and average scores improved in the post session quiz. Positive feedback was paid to the variety of scenarios, the EPR system and the feeling of realism elicited from the need to triage and prioritise jobs.

Conclusions

Our framework uses readily accessible technology to provide interactive learning experience. Feedback suggested students engaged in higher order learning and thinking, achieving our stated aims. We aim to incorporate technologies such as automation software which will allow for a scalable, free, and accessible virtual on call.

734 Evaluation of A Webinar Based Surgical Teaching Course (EDUCATE) - A Prospective Cohort Study

Aim

Anecdotal evidence suggests Foundation Year (FY) doctors start surgical rotations with less confidence than medical rotations. The study aimed to determine the effect of a national webinar-based surgical teaching course on participants’ confidence, and to assess attitudes surrounding undergraduate surgical education.

Method

This prospective cohort study is reported with reference to STROBE guidelines and received ethical approval. A series of 15 free-access webinars was developed based on the Royal College of Surgeons Undergraduate Curriculum. An expert-validated questionnaire was used to collect data before and after the course. Inclusion criteria were UK-based medical students and FY doctors who attended at least one webinar. The primary outcome was confidence in completing common tasks during surgical rotations.

Results

Completed pre-course (484) and post-course (352) questionnaires yielded 92 paired samples (63% female). 85% were medical students, representing 29 UK universities, and 15% FY doctors. Mean confidence in assessing, investigating, and implementing initial management of surgical conditions was greater after the intervention (p ≤ 0.001). Mean confidence in managing on-call tasks and starting a surgical FY job was also higher post-course greater (p ≤ 0.001). These improvements correlated with webinar attendance (p ≤ 0.05). 27.1% of participants were satisfied with the quality of undergraduate surgical education. 22.9% agreed that surgical placements prepared them well to manage surgical tasks.

Conclusions

Medical students and FY doctors report low confidence and feel unprepared in managing surgical tasks. Additionally, they report poor satisfaction with undergraduate surgical education. This shortfall may be improved through delivery of a national, accessible, targeted online webinar series and curriculum.

429 National Evaluation of Confidence and Preparedness for Surgical Rotations in Medical Students and Foundation Year Doctors

Introduction

Limited published and anecdotal evidence suggests foundation year (FY) doctors start their surgical rotations with lower confidence than medical rotations. This may be due to insufficient undergraduate practical teaching related to common surgical rotations. This study aimed to evaluate the confidence and preparedness for surgical rotations of medical students and FY doctors.

Method

An expert-validated questionnaire was distributed nationally to UK medical students and FY doctors. The primary outcome was confidence in completing common tasks during surgical rotations.

Results

491 participants (84% medical students, 16% FYs) were recruited from 36 UK medical schools. 80% were likely to pursue a career in surgery however only 7% felt confident about starting a surgical FY job. 66% felt neutral or unsatisfied about the quality of medical school surgical teaching, and 80% indicated that placements did not prepare them well to manage common FY surgical tasks. The internal reliability of the questionnaire was high (=0.939).

Conclusions

Medical students and FY doctors lack confidence and preparation for surgical jobs. High-quality, practically grounded educational courses such as the National Surgical Teaching Society (NSTS) webinar curriculum could improve confidence and preparedness for surgical rotations. Further research evaluating the benefits of such courses is warranted.