Impact of the COVID-19 Pandemic on Elective and Emergency Surgical Procedures in a University Hospital

Susceptibility of AutoML mortality prediction algorithms to model drift caused by the COVID pandemic

BACKGROUND

Concept drift and covariate shift lead to a degradation of machine learning (ML) models. The objective of our study was to characterize sudden data drift as caused by the COVID pandemic. Furthermore, we investigated the suitability of certain methods in model training to prevent model degradation caused by data drift.

METHODS

We trained different ML models with the H2O AutoML method on a dataset comprising 102,666 cases of surgical patients collected in the years 2014-2019 to predict postoperative mortality using preoperatively available data. Models applied were Generalized Linear Model with regularization, Default Random Forest, Gradient Boosting Machine, eXtreme Gradient Boosting, Deep Learning and Stacked Ensembles comprising all base models. Further, we modified the original models by applying three different methods when training on the original pre-pandemic dataset: (Rahmani K, et al, Int J Med Inform 173:104930, 2023) we weighted older data weaker, (Morger A, et al, Sci Rep 12:7244, 2022) used only the most recent data for model training and (Dilmegani C, 2023) performed a z-transformation of the numerical input parameters. Afterwards, we tested model performance on a pre-pandemic and an in-pandemic data set not used in the training process, and analysed common features.

RESULTS

The models produced showed excellent areas under receiver-operating characteristic and acceptable precision-recall curves when tested on a dataset from January-March 2020, but significant degradation when tested on a dataset collected in the first wave of the COVID pandemic from April-May 2020. When comparing the probability distributions of the input parameters, significant differences between pre-pandemic and in-pandemic data were found. The endpoint of our models, in-hospital mortality after surgery, did not differ significantly between pre- and in-pandemic data and was about 1% in each case. However, the models varied considerably in the composition of their input parameters. None of our applied modifications prevented a loss of performance, although very different models emerged from it, using a large variety of parameters.

CONCLUSIONS

Our results show that none of our tested easy-to-implement measures in model training can prevent deterioration in the case of sudden external events. Therefore, we conclude that, in the presence of concept drift and covariate shift, close monitoring and critical review of model predictions are necessary.

The Impact of the COVID-19 Pandemic on Surgical Services in Brazil's Healthcare System: A Retrospective Cross-Sectional Study

Background The coronavirus disease 2019 (COVID-19) pandemic provoked disruptions in healthcare delivery, leading to the cancellation and postponement of various health services, including surgery. Numerous countries closed their borders and established laws mandating the use of face masks and social distancing and enforced lockdowns, and various activities were constrained. Brazil, the largest and most populous country in Latin America, also experienced a rapid and sustained surge in infections and deaths. Brazil was the most severely impacted nation in Latin America. The impact of the pandemic on surgical services in Brazil has not been adequately studied since most studies only cover the early phases of the pandemic. Thus, this study aimed to assess the impact of the COVID-19 pandemic on surgical services throughout the entire period. Methods A retrospective cross-sectional design was used to examine surgical cases from 2019 to 2022 and compared the following indicators: (1) number of hospital admissions, (2) length of hospital stay (LOS) (in days), and (3) volume of urgent and elective procedures. Data was divided into four time periods, pre-pandemic (March-December 2019), pandemic (March-December 2020), recovery (March-December 2021), and post-pandemic (March-December 2022), and was analyzed for the number of admissions and LOS based on surgical procedures performed by stratifying according to region, sex, age, and type of surgery (urgent versus elective). Results The number of admissions for surgical procedures ranged between 859,646 and 4,015,624 for 2019, 686,616 and 3,419,234 for 2020, 787,791 and 3,829,019 for 2021, and 760,512 and 3,857,817 for 2022 for the category of region; 4,260,900 and 5,991,775 for 2019, 3,594,117 and 4,984,710 for 2020, 4,182,640 and 5,590,808 for 2021, and 4,077,651 and 5,561,928 for 2022 for the category of sex; and 2,170,288 and 3,186,117 for 2019, 1,516,830 and 2,825,189 for 2020, 1,748,202 and 3,030,272 for 2021, and 1,900,023 and 2,859,179 for 2022 for the category of age. The variable age showed a comparable trend, albeit with an expressive decline for surgeries in the age range of 0-19 years. The LOS (in days) for surgical procedures ranged between 110,157 and 910,846 for 2019, 58,562 and 897,734 for 2020, 67,926 and 904,137 for 2021, and 100,467 and 823,545 for 2022. Thoracic surgery indicated no statistically significant difference in the number of admissions and LOS. Elective surgeries had a decline in the number of admissions and LOS, a 13% and 9.3% decline between 2019 and 2020, respectively. Urgent surgeries experienced a slight decrease in admissions and LOS, with a decline of 2.4% and 2.8% between 2019 and 2020, respectively. Conclusions Population characteristics, such as age, sex, and region, showed decreased hospital admissions during the pandemic, followed by a recovery toward pre-pandemic levels afterward. The number of surgical admissions and the length of hospital stays decreased during the pandemic but gradually returned to pre-pandemic levels in the recovery and post-pandemic phases. Notably, thoracic surgery remained statistically consistent across all periods, indicating its emergency nature compared to other surgeries. Thus, we conclude that the pandemic had minimal impact on thoracic surgery cases, contributing to a stable trend.