1
|
Park HY, Sohn H, Hong A, Han SW, Jang Y, Yoon EK, Kim M, Park HY. Application of machine learning for delirium prediction and analysis of associated factors in hospitalized COVID-19 patients: A comparative study using the Korean Multidisciplinary cohort for delirium prevention (KoMCoDe). Int J Med Inform 2025; 195:105747. [PMID: 39644794 DOI: 10.1016/j.ijmedinf.2024.105747] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/07/2024] [Revised: 10/28/2024] [Accepted: 11/30/2024] [Indexed: 12/09/2024]
Abstract
BACKGROUND The incidence of delirium in hospitalized coronavirus disease 2019 (COVID-19) patients is linked to adverse health outcomes. Predicting the occurrence and risk factors of delirium is key to preventing its sudden onset. AIMS To explore the factors associated with delirium in hospitalized COVID-19 patients and to compare the performance of various machine learning (ML) techniques for future use in predicting delirium. METHODS We analyzed a dataset of 1,031 cases from two healthcare centers, which included 178 variables such as demographics, clinical data, and medication information. The ML techniques used in this study were extreme gradient boosting (XGB), light gradient boosting machine (LGBM), logistic regression (LR), random forest (RF), and support vector machine (SVM). RESULTS The RF model emerged as the most effective for predicting delirium, achieving an area under the curve (AUC) of 0.923. It showed a sensitivity of 0.639, accuracy of 0.900, specificity of 0.934, positive predictive value (PPV) of 0.561, negative predictive value (NPV) of 0.952, and an F1 score of 0.597. The RF model identified key variables related to delirium, including medication type (antipsychotic, sedative, opioid), duration of hospital stay, remdesivir usage, and patient age. The reliability of the model was affirmed through calibration plots and Brier score evaluations. CONCLUSIONS This research developed and validated an RF-based ML model for predicting delirium in hospitalized COVID-19 patients. The model demonstrates superior accuracy and reliability compared to other ML methods and would possibly serve as a valuable tool for managing and anticipating delirium in COVID-19 patients, with the potential to enhance patient outcomes.
Collapse
Affiliation(s)
- Hye Yoon Park
- Department of Psychiatry, Seoul National University Hospital, Seoul National University College of Medicine, South Korea; Department of Psychiatry, Seoul National University College of Medicine, South Korea
| | - Hyoju Sohn
- Center for Artificial Intelligence in Healthcare, Seoul National University Bundang Hospital Healthcare Innovation Park, South Korea
| | - Arum Hong
- Department of Psychiatry, Seoul National University Bundang Hospital, South Korea
| | - Soo Wan Han
- Department of Psychiatry, Seoul National University Hospital, Seoul National University College of Medicine, South Korea
| | - Yuna Jang
- Department of Psychiatry, Seoul National University Bundang Hospital, South Korea
| | - EKyong Yoon
- Department of Psychiatry, Seoul National University Bundang Hospital, South Korea
| | - Myeongju Kim
- Center for Artificial Intelligence in Healthcare, Seoul National University Bundang Hospital Healthcare Innovation Park, South Korea
| | - Hye Youn Park
- Department of Psychiatry, Seoul National University College of Medicine, South Korea; Department of Psychiatry, Seoul National University Bundang Hospital, South Korea.
| |
Collapse
|
2
|
Rodriguez HC, Rust BD, Roche MW, Gupta A. Artificial intelligence and machine learning in knee arthroplasty. Knee 2025; 54:28-49. [PMID: 40022960 DOI: 10.1016/j.knee.2025.02.014] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 05/23/2024] [Revised: 10/09/2024] [Accepted: 02/07/2025] [Indexed: 03/04/2025]
Abstract
BACKGROUND Artificial intelligence (AI) and its subset, machine learning (ML), have significantly impacted clinical medicine, particularly in knee arthroplasty (KA). These technologies utilize algorithms for tasks such as predictive analytics and image recognition, improving preoperative planning, intraoperative navigation, and postoperative complication anticipation. This systematic review presents AI-driven tools' clinical implications in total and unicompartmental KA, focusing on enhancing patient outcomes and operational efficiency. METHODS A systematic search was conducted across multiple databases including Cochrane Central Register of Controlled Trials, Embase, OVID Medline, PubMed, and Web of Science, following the PRISMA guidelines for studies published in the English language till March 2024. Inclusion criteria targeted adult human models without geographical restrictions, specifically related to total or unicompartmental KA. RESULTS A total of 153 relevant studies were identified, covering various aspects of ML application for KA. Topics of studies included imaging modalities (n = 28), postoperative primary KA complications (n = 26), inpatient status (length of stay, readmissions, and cost) (n = 24), implant configuration (n = 14), revision (n = 12), patient-reported outcome measures (PROMs) (n = 11), function (n = 11), procedural communication (n = 8), total knee arthroplasty/unicompartmental knee arthroplasty prediction (n = 6), outpatient status (n = 4), perioperative efficiency (n = 4), patient satisfaction (n = 3), opioid usage (n = 3). A total of 66 ML models were described, with 48.7% of studies using multiple approaches. CONCLUSION This review assesses ML applications in knee arthroplasty, highlighting their potential to improve patient outcomes. While current algorithms and AI show promise, our findings suggest areas for enhancement in predictive performance before widespread clinical adoption.
Collapse
Affiliation(s)
- Hugo C Rodriguez
- Larkin Community Hospital, Department of Orthopaedic Surgery, South Miami, FL, USA; Hospital for Special Surgery, West Palm Beach, FL, USA
| | - Brandon D Rust
- Nova Southeastern University, Dr. Kiran C. Patel College of Osteopathic Medicine, Fort Lauderdale, FL, USA
| | | | | |
Collapse
|
3
|
Holler E, Ludema C, Ben Miled Z, Rosenberg M, Kalbaugh C, Boustani M, Mohanty S. Development and Validation of a Routine Electronic Health Record-Based Delirium Prediction Model for Surgical Patients Without Dementia: Retrospective Case-Control Study. JMIR Perioper Med 2025; 8:e59422. [PMID: 39786865 PMCID: PMC11757977 DOI: 10.2196/59422] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/11/2024] [Revised: 10/15/2024] [Accepted: 11/01/2024] [Indexed: 01/12/2025] Open
Abstract
BACKGROUND Postoperative delirium (POD) is a common complication after major surgery and is associated with poor outcomes in older adults. Early identification of patients at high risk of POD can enable targeted prevention efforts. However, existing POD prediction models require inpatient data collected during the hospital stay, which delays predictions and limits scalability. OBJECTIVE This study aimed to develop and externally validate a machine learning-based prediction model for POD using routine electronic health record (EHR) data. METHODS We identified all surgical encounters from 2014 to 2021 for patients aged 50 years and older who underwent an operation requiring general anesthesia, with a length of stay of at least 1 day at 3 Indiana hospitals. Patients with preexisting dementia or mild cognitive impairment were excluded. POD was identified using Confusion Assessment Method records and delirium International Classification of Diseases (ICD) codes. Controls without delirium or nurse-documented confusion were matched to cases by age, sex, race, and year of admission. We trained logistic regression, random forest, extreme gradient boosting (XGB), and neural network models to predict POD using 143 features derived from routine EHR data available at the time of hospital admission. Separate models were developed for each hospital using surveillance periods of 3 months, 6 months, and 1 year before admission. Model performance was evaluated using the area under the receiver operating characteristic curve (AUROC). Each model was internally validated using holdout data and externally validated using data from the other 2 hospitals. Calibration was assessed using calibration curves. RESULTS The study cohort included 7167 delirium cases and 7167 matched controls. XGB outperformed all other classifiers. AUROCs were highest for XGB models trained on 12 months of preadmission data. The best-performing XGB model achieved a mean AUROC of 0.79 (SD 0.01) on the holdout set, which decreased to 0.69-0.74 (SD 0.02) when externally validated on data from other hospitals. CONCLUSIONS Our routine EHR-based POD prediction models demonstrated good predictive ability using a limited set of preadmission and surgical variables, though their generalizability was limited. The proposed models could be used as a scalable, automated screening tool to identify patients at high risk of POD at the time of hospital admission.
Collapse
Affiliation(s)
- Emma Holler
- Department of Surgery, Indiana University School of Medicine, Indianapolis, IN, United States
| | - Christina Ludema
- Department of Epidemiology & Biostatistics, Indiana University Bloomington, Bloomington, United States
| | - Zina Ben Miled
- Department of Electrical & Computer Engineering, Lamar University, Beaumont, TX, United States
| | - Molly Rosenberg
- Department of Epidemiology & Biostatistics, Indiana University Bloomington, Bloomington, United States
| | - Corey Kalbaugh
- Department of Epidemiology & Biostatistics, Indiana University Bloomington, Bloomington, United States
| | - Malaz Boustani
- Department of Medicine, Indiana University School of Medicine, Indianapolis, IN, United States
| | - Sanjay Mohanty
- Department of Surgery, Indiana University School of Medicine, Indianapolis, IN, United States
| |
Collapse
|
4
|
Chen H, Yu D, Zhang J, Li J. Machine Learning for Prediction of Postoperative Delirium in Adult Patients: A Systematic Review and Meta-analysis. Clin Ther 2024; 46:1069-1081. [PMID: 39395856 DOI: 10.1016/j.clinthera.2024.09.013] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/10/2024] [Revised: 09/03/2024] [Accepted: 09/09/2024] [Indexed: 10/14/2024]
Abstract
PURPOSE This meta-analysis aimed to evaluate the performance of machine learning (ML) models in predicting postoperative delirium (POD) and to provide guidance for clinical application. METHODS PubMed, Embase, Cochrane Library, and Web of Science databases were searched from inception to April 29, 2024. Studies reported ML models for predicting POD in adult patients were included. Data extraction and risk of bias assessment were performed using the Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis - AI (TRIPOD-AI) and Prediction model Risk Of Bias ASsessment Tool (PROBAST) tools. Meta-analysis with the area under the curve (AUC) was performed using MedCalc software. FINDINGS A total of 23 studies were included after screening. Age (n = 20, 86.95%) and Random Forest (RF) (n = 24, 17.27%) were the most frequently used feature and ML algorithm, respectively. The meta-analysis showed an overall AUC of 0.792. The ensemble models (AUC = 0.805) showed better predictive performance than single models (AUC = 0.782). Additionally, considerable variations in AUC were found among different ML algorithms, with AdaBoost (AB) demonstrating good performance with AUC of 0.870. Notably, the generalizability of these models was uncertain due to limitations in external validation and bias assessment. IMPLICATIONS The performance of ensemble models were higher than single models, and the AB algorithms demonstrated better performance, compared with other algorithms. However, further research was needed to enhance the generalizability and transparency of ML models.
Collapse
Affiliation(s)
- Hao Chen
- Department of Anesthesiology, Hebei General Hospital, Shijiazhuang, Hebei Province, China; North China University of Science and Technology, Tangshan, China
| | - Dongdong Yu
- Department of Anesthesiology, Hebei General Hospital, Shijiazhuang, Hebei Province, China
| | - Jing Zhang
- Department of Anesthesiology, Hebei General Hospital, Shijiazhuang, Hebei Province, China
| | - Jianli Li
- Department of Anesthesiology, Hebei General Hospital, Shijiazhuang, Hebei Province, China.
| |
Collapse
|
5
|
Mickley JP, Kaji ES, Khosravi B, Mulford KL, Taunton MJ, Wyles CC. Overview of Artificial Intelligence Research Within Hip and Knee Arthroplasty. Arthroplast Today 2024; 27:101396. [PMID: 39071822 PMCID: PMC11282426 DOI: 10.1016/j.artd.2024.101396] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 09/08/2023] [Revised: 03/14/2024] [Accepted: 04/02/2024] [Indexed: 07/30/2024] Open
Abstract
Hip and knee arthroplasty are high-volume procedures undergoing rapid growth. The large volume of procedures generates a vast amount of data available for next-generation analytics. Techniques in the field of artificial intelligence (AI) can assist in large-scale pattern recognition and lead to clinical insights. AI methodologies have become more prevalent in orthopaedic research. This review will first describe an overview of AI in the medical field, followed by a description of the 3 arthroplasty research areas in which AI is commonly used (risk modeling, automated radiographic measurements, arthroplasty registry construction). Finally, we will discuss the next frontier of AI research focusing on model deployment and uncertainty quantification.
Collapse
Affiliation(s)
- John P. Mickley
- Orthopedic Surgery Artificial Intelligence Laboratory (OSAIL), Department of Orthopedic Surgery, Mayo Clinic, Rochester, MN, USA
| | - Elizabeth S. Kaji
- Orthopedic Surgery Artificial Intelligence Laboratory (OSAIL), Department of Orthopedic Surgery, Mayo Clinic, Rochester, MN, USA
| | - Bardia Khosravi
- Orthopedic Surgery Artificial Intelligence Laboratory (OSAIL), Department of Orthopedic Surgery, Mayo Clinic, Rochester, MN, USA
- Radiology Informatics Lab (RIL), Department of Radiology, Mayo Clinic, Rochester, MN, USA
| | - Kellen L. Mulford
- Orthopedic Surgery Artificial Intelligence Laboratory (OSAIL), Department of Orthopedic Surgery, Mayo Clinic, Rochester, MN, USA
| | - Michael J. Taunton
- Orthopedic Surgery Artificial Intelligence Laboratory (OSAIL), Department of Orthopedic Surgery, Mayo Clinic, Rochester, MN, USA
- Department of Orthopedic Surgery, Mayo Clinic, Rochester, MN, USA
| | - Cody C. Wyles
- Orthopedic Surgery Artificial Intelligence Laboratory (OSAIL), Department of Orthopedic Surgery, Mayo Clinic, Rochester, MN, USA
- Department of Orthopedic Surgery, Mayo Clinic, Rochester, MN, USA
- Department of Clinical Anatomy, Mayo Clinic, Rochester, MN, USA
| |
Collapse
|
6
|
Brydges G, Uppal A, Gottumukkala V. Application of Machine Learning in Predicting Perioperative Outcomes in Patients with Cancer: A Narrative Review for Clinicians. Curr Oncol 2024; 31:2727-2747. [PMID: 38785488 PMCID: PMC11120613 DOI: 10.3390/curroncol31050207] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/09/2024] [Revised: 05/07/2024] [Accepted: 05/10/2024] [Indexed: 05/25/2024] Open
Abstract
This narrative review explores the utilization of machine learning (ML) and artificial intelligence (AI) models to enhance perioperative cancer care. ML and AI models offer significant potential to improve perioperative cancer care by predicting outcomes and supporting clinical decision-making. Tailored for perioperative professionals including anesthesiologists, surgeons, critical care physicians, nurse anesthetists, and perioperative nurses, this review provides a comprehensive framework for the integration of ML and AI models to enhance patient care delivery throughout the perioperative continuum.
Collapse
Affiliation(s)
- Garry Brydges
- Division of Anesthesiology, Critical Care & Pain Medicine, The University of Texas at MD Anderson Cancer Center, Houston, TX 77030, USA;
| | - Abhineet Uppal
- Department of Colon & Rectal Surgery, The University of Texas at MD Anderson Cancer Center, Houston, TX 77030, USA;
| | - Vijaya Gottumukkala
- Department of Anesthesiology & Perioperative Medicine, The University of Texas at MD Anderson Cancer Center, 1400-Unit 409, Holcombe Blvd, Houston, TX 77030, USA
| |
Collapse
|
7
|
Benovic S, Ajlani AH, Leinert C, Fotteler M, Wolf D, Steger F, Kestler H, Dallmeier D, Denkinger M, Eschweiler GW, Thomas C, Kocar TD. Introducing a machine learning algorithm for delirium prediction-the Supporting SURgery with GEriatric Co-Management and AI project (SURGE-Ahead). Age Ageing 2024; 53:afae101. [PMID: 38776213 PMCID: PMC11110913 DOI: 10.1093/ageing/afae101] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/19/2023] [Indexed: 05/24/2024] Open
Abstract
INTRODUCTION Post-operative delirium (POD) is a common complication in older patients, with an incidence of 14-56%. To implement preventative procedures, it is necessary to identify patients at risk for POD. In the present study, we aimed to develop a machine learning (ML) model for POD prediction in older patients, in close cooperation with the PAWEL (patient safety, cost-effectiveness and quality of life in elective surgery) project. METHODS The model was trained on the PAWEL study's dataset of 878 patients (no intervention, age ≥ 70, 209 with POD). Presence of POD was determined by the Confusion Assessment Method and a chart review. We selected 15 features based on domain knowledge, ethical considerations and a recursive feature elimination. A logistic regression and a linear support vector machine (SVM) were trained, and evaluated using receiver operator characteristics (ROC). RESULTS The selected features were American Society of Anesthesiologists score, multimorbidity, cut-to-suture time, estimated glomerular filtration rate, polypharmacy, use of cardio-pulmonary bypass, the Montreal cognitive assessment subscores 'memory', 'orientation' and 'verbal fluency', pre-existing dementia, clinical frailty scale, age, recent falls, post-operative isolation and pre-operative benzodiazepines. The linear SVM performed best, with an ROC area under the curve of 0.82 [95% CI 0.78-0.85] in the training set, 0.81 [95% CI 0.71-0.88] in the test set and 0.76 [95% CI 0.71-0.79] in a cross-centre validation. CONCLUSION We present a clinically useful and explainable ML model for POD prediction. The model will be deployed in the Supporting SURgery with GEriatric Co-Management and AI project.
Collapse
Affiliation(s)
- Samuel Benovic
- Institute of Geriatric Research, Ulm University Medical Center, Ulm, Germany
- Agaplesion Bethesda Clinic Ulm, Ulm, Germany
| | - Anna H Ajlani
- Institute of the History, Philosophy and Ethics of Medicine, Ulm University, Ulm, Germany
- Department of Sociology with a Focus on Innovation and Digitalization, Institute of Sociology, Johannes Kepler University Linz, Linz, Austria
| | - Christoph Leinert
- Institute of Geriatric Research, Ulm University Medical Center, Ulm, Germany
- Agaplesion Bethesda Clinic Ulm, Ulm, Germany
| | - Marina Fotteler
- Agaplesion Bethesda Clinic Ulm, Ulm, Germany
- DigiHealth Institute, Neu-Ulm University of Applied Sciences, Neu-Ulm, Germany
| | - Dennis Wolf
- Institute of Medical Systems Biology, Ulm University, Ulm, Germany
| | - Florian Steger
- Institute of the History, Philosophy and Ethics of Medicine, Ulm University, Ulm, Germany
| | - Hans Kestler
- Institute of Medical Systems Biology, Ulm University, Ulm, Germany
| | - Dhayana Dallmeier
- Institute of Geriatric Research, Ulm University Medical Center, Ulm, Germany
- Department of Epidemiology, Boston University School of Public Health, Boston, USA
| | - Michael Denkinger
- Institute of Geriatric Research, Ulm University Medical Center, Ulm, Germany
- Agaplesion Bethesda Clinic Ulm, Ulm, Germany
| | - Gerhard W Eschweiler
- Geriatric Center, University Hospital Tübingen, Tubingen, Germany
- Department of Psychiatry and Psychotherapy, Tübingen University Hospital, Tübingen, Germany
| | - Christine Thomas
- Department of Psychiatry and Psychotherapy, Tübingen University Hospital, Tübingen, Germany
- Department of Geriatric Psychiatry and Psychotherapy, Klinikum Stuttgart, Stuttgart, Germany
| | - Thomas D Kocar
- Institute of Geriatric Research, Ulm University Medical Center, Ulm, Germany
- Agaplesion Bethesda Clinic Ulm, Ulm, Germany
| |
Collapse
|
8
|
Rössler J, Shah K, Medellin S, Turan A, Ruetzler K, Singh M, Sessler DI, Maheshwari K. Development and validation of delirium prediction models for noncardiac surgery patients. J Clin Anesth 2024; 93:111319. [PMID: 37984177 DOI: 10.1016/j.jclinane.2023.111319] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/04/2023] [Revised: 10/20/2023] [Accepted: 11/05/2023] [Indexed: 11/22/2023]
Abstract
STUDY OBJECTIVE Postoperative delirium is associated with morbidity and mortality, and its incidence varies widely. Using known predisposing and precipitating factors, we sought to develop postoperative delirium prediction models for noncardiac surgical patients. DESIGN Retrospective prediction model study. SETTING Major quaternary medical center. PATIENTS Our January 2016 to June 2020 training dataset included 51,677 patients of whom 2795 patients had delirium. Our July 2020 to January 2022 validation dataset included 14,438 patients of whom 912 patients had delirium. INTERVENTIONS None. MEASUREMENTS We trained and validated two static prediction models and one dynamic delirium prediction model. For the static models, we used random survival forests and traditional Cox proportional hazard models to predict postoperative delirium from preoperative variables, or from a combination of preoperative and intraoperative variables. We also used landmark modeling to dynamically predict postoperative delirium using preoperative, intraoperative, and postoperative variables before onset of delirium. MAIN RESULTS In the validation analyses, the static random forest model had a c-statistic of 0.81 (95% CI: 0.79, 0.82) and a Brier score of 0.04 with preoperative variables only, and a c-statistic of 0.86 (95% CI: 0.84, 0.87) and a Brier score of 0.04 when preoperative and intraoperative variables were combined. The corresponding Cox models had similar discrimination metrics with slightly better calibration. The dynamic model - using all available data, i.e., preoperative, intraoperative and postoperative data - had an overall c-index of 0.84 (95% CI: 0.83, 0.85). CONCLUSIONS Using preoperative and intraoperative variables, simple static models performed as well as a dynamic delirium prediction model that also included postoperative variables. Baseline predisposing factors thus appear to contribute far more to delirium after noncardiac surgery than intraoperative or postoperative variables. Improved postoperative data capture may help improve delirium prediction and should be evaluated in future studies.
Collapse
Affiliation(s)
- Julian Rössler
- Department of Outcomes Research, Cleveland Clinic, Cleveland, OH, USA.
| | - Karan Shah
- Department of Outcomes Research, Cleveland Clinic, Cleveland, OH, USA; Department of Quantitative Health Sciences, Cleveland Clinic, OH, USA
| | - Sara Medellin
- Department of Outcomes Research, Cleveland Clinic, Cleveland, OH, USA
| | - Alparslan Turan
- Department of Outcomes Research, Cleveland Clinic, Cleveland, OH, USA; Department of General Anesthesiology, Cleveland Clinic, Cleveland, OH, USA
| | - Kurt Ruetzler
- Department of Outcomes Research, Cleveland Clinic, Cleveland, OH, USA; Department of General Anesthesiology, Cleveland Clinic, Cleveland, OH, USA
| | - Mriganka Singh
- Division of Geriatrics and Palliative Medicine, Alpert Medical School of Brown University, Providence, RI, USA; Center on Innovation-Long Term Services and Supports, Providence Veterans Administration Medical Center, Providence, RI, USA
| | - Daniel I Sessler
- Department of Outcomes Research, Cleveland Clinic, Cleveland, OH, USA
| | - Kamal Maheshwari
- Department of Outcomes Research, Cleveland Clinic, Cleveland, OH, USA; Department of General Anesthesiology, Cleveland Clinic, Cleveland, OH, USA
| |
Collapse
|
9
|
Huffman N, Pasqualini I, Khan ST, Klika AK, Deren ME, Jin Y, Kunze KN, Piuzzi NS. Enabling Personalized Medicine in Orthopaedic Surgery Through Artificial Intelligence: A Critical Analysis Review. JBJS Rev 2024; 12:01874474-202403000-00006. [PMID: 38466797 DOI: 10.2106/jbjs.rvw.23.00232] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 03/13/2024]
Abstract
» The application of artificial intelligence (AI) in the field of orthopaedic surgery holds potential for revolutionizing health care delivery across 3 crucial domains: (I) personalized prediction of clinical outcomes and adverse events, which may optimize patient selection, surgical planning, and enhance patient safety and outcomes; (II) diagnostic automated and semiautomated imaging analyses, which may reduce time burden and facilitate precise and timely diagnoses; and (III) forecasting of resource utilization, which may reduce health care costs and increase value for patients and institutions.» Computer vision is one of the most highly studied areas of AI within orthopaedics, with applications pertaining to fracture classification, identification of the manufacturer and model of prosthetic implants, and surveillance of prosthesis loosening and failure.» Prognostic applications of AI within orthopaedics include identifying patients who will likely benefit from a specified treatment, predicting prosthetic implant size, postoperative length of stay, discharge disposition, and surgical complications. Not only may these applications be beneficial to patients but also to institutions and payors because they may inform potential cost expenditure, improve overall hospital efficiency, and help anticipate resource utilization.» AI infrastructure development requires institutional financial commitment and a team of clinicians and data scientists with expertise in AI that can complement skill sets and knowledge. Once a team is established and a goal is determined, teams (1) obtain, curate, and label data; (2) establish a reference standard; (3) develop an AI model; (4) evaluate the performance of the AI model; (5) externally validate the model, and (6) reinforce, improve, and evaluate the model's performance until clinical implementation is possible.» Understanding the implications of AI in orthopaedics may eventually lead to wide-ranging improvements in patient care. However, AI, while holding tremendous promise, is not without methodological and ethical limitations that are essential to address. First, it is important to ensure external validity of programs before their use in a clinical setting. Investigators should maintain high quality data records and registry surveillance, exercise caution when evaluating others' reported AI applications, and increase transparency of the methodological conduct of current models to improve external validity and avoid propagating bias. By addressing these challenges and responsibly embracing the potential of AI, the medical field may eventually be able to harness its power to improve patient care and outcomes.
Collapse
Affiliation(s)
- Nickelas Huffman
- Cleveland Clinic, Department of Orthopaedic Surgery, Cleveland, Ohio
| | | | - Shujaa T Khan
- Cleveland Clinic, Department of Orthopaedic Surgery, Cleveland, Ohio
| | - Alison K Klika
- Cleveland Clinic, Department of Orthopaedic Surgery, Cleveland, Ohio
| | - Matthew E Deren
- Cleveland Clinic, Department of Orthopaedic Surgery, Cleveland, Ohio
| | - Yuxuan Jin
- Cleveland Clinic, Department of Orthopaedic Surgery, Cleveland, Ohio
| | - Kyle N Kunze
- Department of Orthopaedic Surgery, Hospital for Special Surgery, New York, New York
| | - Nicolas S Piuzzi
- Cleveland Clinic, Department of Orthopaedic Surgery, Cleveland, Ohio
- Department of Biomedical Engineering, Cleveland Clinic Foundation, Cleveland, Ohio
| |
Collapse
|
10
|
Lu Z, Wang B, Liu M, Yu D, Li J. Correlation analysis between plasma biomarkers albumin, fibrinogen, and their ratio with postoperative delirium in patients undergoing non-cardiac surgery: a systematic review and meta-analysis. Am J Transl Res 2024; 16:363-373. [PMID: 38463596 PMCID: PMC10918125] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/13/2023] [Accepted: 01/23/2024] [Indexed: 03/12/2024]
Abstract
OBJECTIVES This meta-analysis aimed to investigate the correlation between plasma biomarkers, such as albumin and fibrinogen, and their ratio with postoperative delirium (POD) in patients undergoing non-cardiac surgery. METHODS Relevant observational cohort studies were systematically searched in PubMed, EMBASE, CINAHL, and the Cochrane Library databases as of March 2023. This meta-analysis was conducted using RevMan 5.4.1 and Stata 15.0 software. For continuous variables with non-uniform units, the standardized mean difference (SMD) and 95% confidence intervals (CIs) were used; otherwise, the mean difference (MD) and 95% CIs were employed. The Newcastle-Ottawa Scale (NOS) was applied to assess the quality of included literature. RESULTS Eighteen studies encompassing 7,011 patients were included. The meta-analysis revealed significantly lower albumin levels (sixteen studies, 5,813 patients, SMD = -0.45, 95% CI = -0.64 to -0.26, P < 0.00001, I2 = 80%) and albumin-fibrinogen ratio (AFR) (four studies, 824 patients, MD = -0.62, 95% CI = -0.76 to -0.48, P = 0.56, I2 = 0%) in the delirious group. Conversely, higher fibrinogen concentrations (two studies, 441 patients, MD = 0.13, 95% CI = 0.02 to 0.24, P = 0.69, I2 = 0%) were observed in the delirious group. Due to high heterogeneity in albumin levels (P < 0.00001, I2 = 80%), we conducted a subgroup and sensitivity analysis, and confirmed that the association of albumin levels was not influenced by surgery type, design or delirium evaluation instruments. CONCLUSIONS Preoperative albumin, fibrinogen and AFR levels were associated with POD, potentially aiding in identifying high-risk patients and playing a key role in preventing POD.
Collapse
Affiliation(s)
- Zhenhui Lu
- Department of Anesthesiology, Hebei General HospitalShijiazhuang 050051, Hebei, China
| | - Bei Wang
- Department of Gynaecology, Hebei General HospitalShijiazhuang 050051, Hebei, China
| | - Meinv Liu
- Department of Anesthesiology, Hebei General HospitalShijiazhuang 050051, Hebei, China
| | - Dongdong Yu
- Department of Anesthesiology, Hebei General HospitalShijiazhuang 050051, Hebei, China
| | - Jianli Li
- Department of Anesthesiology, Hebei General HospitalShijiazhuang 050051, Hebei, China
| |
Collapse
|
11
|
Lee SH, Hur HJ, Kim SN, Ahn JH, Ro DH, Hong A, Park HY, Choe PG, Kim B, Park HY. Predicting delirium and the effects of medications in hospitalized COVID-19 patients using machine learning: A retrospective study within the Korean Multidisciplinary Cohort for Delirium Prevention (KoMCoDe). Digit Health 2024; 10:20552076231223811. [PMID: 38188862 PMCID: PMC10771056 DOI: 10.1177/20552076231223811] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/06/2023] [Accepted: 12/14/2023] [Indexed: 01/09/2024] Open
Abstract
Objective Delirium is commonly reported from the inpatients with Coronavirus disease 2019 (COVID-19) infection. As delirium is closely associated with adverse clinical outcomes, prediction and prevention of delirium is critical. We developed a machine learning (ML) model to predict delirium in hospitalized patients with COVID-19 and to identify modifiable factors to prevent delirium. Methods The data set (n = 878) from four medical centers was constructed. Total of 78 predictors were included such as demographic characteristics, vital signs, laboratory results and medication, and the primary outcome was delirium occurrence during hospitalization. For analysis, the extreme gradient boosting (XGBoost) algorithm was applied, and the most influential factors were selected by recursive feature elimination. Among the indicators of performance for ML model, the area under the curve of the receiver operating characteristic (AUROC) curve was selected as the evaluation metric. Results Regarding the performance of developed delirium prediction model, the accuracy, precision, recall, F1 score, and the AUROC were calculated (0.944, 0.581, 0.421, 0.485, 0.873, respectively). The influential factors of delirium in this model included were mechanical ventilation, medication (antipsychotics, sedatives, ambroxol, piperacillin/tazobactam, acetaminophen, ceftriaxone, and propacetamol), and sodium ion concentration (all p < 0.05). Conclusions We developed and internally validated an ML model to predict delirium in COVID-19 inpatients. The model identified modifiable factors associated with the development of delirium and could be clinically useful for the prediction and prevention of delirium in COVID-19 inpatients.
Collapse
Affiliation(s)
- So Hee Lee
- Department of Psychiatry, National Medical Center, Seoul,
South Korea
| | - Hyun Jung Hur
- Department of Psychiatry, Seoul National University Bundang Hospital, Seongnam, South Korea
| | - Sung Nyun Kim
- Department of Psychiatry, Seoul Medical Center, Seoul, South Korea
| | - Jang Ho Ahn
- Seoul National University College of Medicine, Seoul, South Korea
| | - Du Hyun Ro
- Department of Orthopedic Surgery, Seoul National University Hospital, Seoul, South Korea
| | - Arum Hong
- Department of Psychiatry, Seoul National University Bundang Hospital, Seongnam, South Korea
| | - Hye Yoon Park
- Department of Psychiatry, Seoul National University Hospital, Seoul National University College of Medicine, Seongnam,
South Korea
- Department of Psychiatry, Seoul National University College of Medicine, Seoul, South Korea
| | - Pyoeng Gyun Choe
- Department of Internal Medicine, Seoul National University Hospital, Seoul National University College of Medicine, Seongnam, South Korea
| | - Back Kim
- Seoul National University College of Medicine, Seoul, South Korea
| | - Hye Youn Park
- Department of Psychiatry, Seoul National University Bundang Hospital, Seongnam, South Korea
- Department of Psychiatry, Seoul National University College of Medicine, Seoul, South Korea
| |
Collapse
|
12
|
Song AL, Li YJ, Liang H, Sun YZ, Shu X, Huang JH, Yang ZY, He WQ, Zhao L, Zhu T, Zhong KH, Chen YW, Lu KZ, Yi B. Dynamic Nomogram for Predicting the Risk of Perioperative Neurocognitive Disorders in Adults. Anesth Analg 2023; 137:1257-1269. [PMID: 37973132 PMCID: PMC10629609 DOI: 10.1213/ane.0000000000006746] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 08/29/2023] [Indexed: 11/19/2023]
Abstract
BACKGROUND Simple and rapid tools for screening high-risk patients for perioperative neurocognitive disorders (PNDs) are urgently needed to improve patient outcomes. We developed an online tool with machine-learning algorithms using routine variables based on multicenter data. METHODS The entire dataset was composed of 49,768 surgical patients from 3 representative academic hospitals in China. Surgical patients older than 45 years, those undergoing general anesthesia, and those without a history of PND were enrolled. When the patient's discharge diagnosis was PND, the patient was in the PND group. Patients in the non-PND group were randomly extracted from the big data platform according to the surgical type, age, and source of data in the PND group with a ratio of 3:1. After data preprocessing and feature selection, general linear model (GLM), artificial neural network (ANN), and naive Bayes (NB) were used for model development and evaluation. Model performance was evaluated by the area under the receiver operating characteristic curve (ROCAUC), the area under the precision-recall curve (PRAUC), the Brier score, the index of prediction accuracy (IPA), sensitivity, specificity, etc. The model was also externally validated on the multiparameter intelligent monitoring in intensive care (MIMIC) Ⅳ database. Afterward, we developed an online visualization tool to preoperatively predict patients' risk of developing PND based on the models with the best performance. RESULTS A total of 1051 patients (242 PND and 809 non-PND) and 2884 patients (6.2% patients with PND) were analyzed on multicenter data (model development, test [internal validation], external validation-1) and MIMIC Ⅳ dataset (external validation-2). The model performance based on GLM was much better than that based on ANN and NB. The best-performing GLM model on validation-1 dataset achieved ROCAUC (0.874; 95% confidence interval [CI], 0.833-0.915), PRAUC (0.685; 95% CI, 0.584-0.786), sensitivity (72.6%; 95% CI, 61.4%-81.5%), specificity (84.4%; 95% CI, 79.3%-88.4%), Brier score (0.131), and IPA (44.7%), and of which the ROCAUC (0.761, 95% CI, 0.712-0.809), the PRAUC (0.475, 95% CI, 0.370-0.581), Brier score (0.053), and IPA (76.8%) on validation-2 dataset. Afterward, we developed an online tool (https://pnd-predictive-model-dynnom.shinyapps.io/ DynNomapp/) with 10 routine variables for preoperatively screening high-risk patients. CONCLUSIONS We developed a simple and rapid online tool to preoperatively screen patients' risk of PND using GLM based on multicenter data, which may help medical staff's decision-making regarding perioperative management strategies to improve patient outcomes.
Collapse
Affiliation(s)
- Ai-lin Song
- From the Department of Anesthesiology, Southwest Hospital, Third Military Medical University, Chongqing, China
| | - Yu-jie Li
- From the Department of Anesthesiology, Southwest Hospital, Third Military Medical University, Chongqing, China
| | - Hao Liang
- From the Department of Anesthesiology, Southwest Hospital, Third Military Medical University, Chongqing, China
| | - Yi-zhu Sun
- From the Department of Anesthesiology, Southwest Hospital, Third Military Medical University, Chongqing, China
| | - Xin Shu
- From the Department of Anesthesiology, Southwest Hospital, Third Military Medical University, Chongqing, China
| | - Jia-hao Huang
- From the Department of Anesthesiology, Southwest Hospital, Third Military Medical University, Chongqing, China
| | - Zhi-yong Yang
- From the Department of Anesthesiology, Southwest Hospital, Third Military Medical University, Chongqing, China
| | - Wen-quan He
- From the Department of Anesthesiology, Southwest Hospital, Third Military Medical University, Chongqing, China
| | - Lei Zhao
- Department of Anesthesiology, Xuan Wu Hospital, Capital Medical University, Beijing, China
| | - Tao Zhu
- Department of Anesthesiology, West China Hospital of Sichuan University, Chengdu, China
| | - Kun-hua Zhong
- Electronic Information Technology Research Institute, Chongqing Institute of Green and Intelligent Technology, Chinese Academy of Science, Chongqing, China
| | - Yu-wen Chen
- Electronic Information Technology Research Institute, Chongqing Institute of Green and Intelligent Technology, Chinese Academy of Science, Chongqing, China
| | - Kai-zhi Lu
- From the Department of Anesthesiology, Southwest Hospital, Third Military Medical University, Chongqing, China
| | - Bin Yi
- From the Department of Anesthesiology, Southwest Hospital, Third Military Medical University, Chongqing, China
| |
Collapse
|
13
|
Matsumoto K, Nohara Y, Sakaguchi M, Takayama Y, Fukushige S, Soejima H, Nakashima N, Kamouchi M. Temporal Generalizability of Machine Learning Models for Predicting Postoperative Delirium Using Electronic Health Record Data: Model Development and Validation Study. JMIR Perioper Med 2023; 6:e50895. [PMID: 37883164 PMCID: PMC10636625 DOI: 10.2196/50895] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/16/2023] [Revised: 09/24/2023] [Accepted: 09/29/2023] [Indexed: 10/27/2023] Open
Abstract
BACKGROUND Although machine learning models demonstrate significant potential in predicting postoperative delirium, the advantages of their implementation in real-world settings remain unclear and require a comparison with conventional models in practical applications. OBJECTIVE The objective of this study was to validate the temporal generalizability of decision tree ensemble and sparse linear regression models for predicting delirium after surgery compared with that of the traditional logistic regression model. METHODS The health record data of patients hospitalized at an advanced emergency and critical care medical center in Kumamoto, Japan, were collected electronically. We developed a decision tree ensemble model using extreme gradient boosting (XGBoost) and a sparse linear regression model using least absolute shrinkage and selection operator (LASSO) regression. To evaluate the predictive performance of the model, we used the area under the receiver operating characteristic curve (AUROC) and the Matthews correlation coefficient (MCC) to measure discrimination and the slope and intercept of the regression between predicted and observed probabilities to measure calibration. The Brier score was evaluated as an overall performance metric. We included 11,863 consecutive patients who underwent surgery with general anesthesia between December 2017 and February 2022. The patients were divided into a derivation cohort before the COVID-19 pandemic and a validation cohort during the COVID-19 pandemic. Postoperative delirium was diagnosed according to the confusion assessment method. RESULTS A total of 6497 patients (68.5, SD 14.4 years, women n=2627, 40.4%) were included in the derivation cohort, and 5366 patients (67.8, SD 14.6 years, women n=2105, 39.2%) were included in the validation cohort. Regarding discrimination, the XGBoost model (AUROC 0.87-0.90 and MCC 0.34-0.44) did not significantly outperform the LASSO model (AUROC 0.86-0.89 and MCC 0.34-0.41). The logistic regression model (AUROC 0.84-0.88, MCC 0.33-0.40, slope 1.01-1.19, intercept -0.16 to 0.06, and Brier score 0.06-0.07), with 8 predictors (age, intensive care unit, neurosurgery, emergency admission, anesthesia time, BMI, blood loss during surgery, and use of an ambulance) achieved good predictive performance. CONCLUSIONS The XGBoost model did not significantly outperform the LASSO model in predicting postoperative delirium. Furthermore, a parsimonious logistic model with a few important predictors achieved comparable performance to machine learning models in predicting postoperative delirium.
Collapse
Affiliation(s)
| | - Yasunobu Nohara
- Big Data Science and Technology, Faculty of Advanced Science and Technology, Kumamoto University, Kumamoto, Japan
| | - Mikako Sakaguchi
- Department of Nursing, Saiseikai Kumamoto Hospital, Kumamoto, Japan
| | - Yohei Takayama
- Department of Nursing, Saiseikai Kumamoto Hospital, Kumamoto, Japan
| | - Syota Fukushige
- Department of Inspection, Saiseikai Kumamoto Hospital, Kumamoto, Japan
| | - Hidehisa Soejima
- Institute for Medical Information Research and Analysis, Saiseikai Kumamoto Hospital, Kumamoto, Japan
| | - Naoki Nakashima
- Medical Information Center, Kyushu University Hospital, Fukuoka, Japan
| | - Masahiro Kamouchi
- Department of Health Care Administration and Management, Graduate School of Medical Sciences, Kyushu University, Fukuoka, Japan
- Center for Cohort Studies, Graduate School of Medical Sciences, Kyushu University, Fukuoka, Japan
| |
Collapse
|
14
|
Kurmis AP. A role for artificial intelligence applications inside and outside of the operating theatre: a review of contemporary use associated with total knee arthroplasty. ARTHROPLASTY 2023; 5:40. [PMID: 37400876 DOI: 10.1186/s42836-023-00189-0] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/03/2023] [Accepted: 04/19/2023] [Indexed: 07/05/2023] Open
Abstract
BACKGROUND Artificial intelligence (AI) has become involved in many aspects of everyday life, from voice-activated virtual assistants built into smartphones to global online search engines. Similarly, many areas of modern medicine have found ways to incorporate such technologies into mainstream practice. Despite the enthusiasm, robust evidence to support the utility of AI in contemporary total knee arthroplasty (TKA) remains limited. The purpose of this review was to provide an up-to-date summary of the use of AI in TKA and to explore its current and future value. METHODS Initially, a structured systematic review of the literature was carried out, following PRISMA search principles, with the aim of summarising the understanding of the field and identifying clinical and knowledge gaps. RESULTS A limited body of published work exists in this area. Much of the available literature is of poor methodological quality and many published studies could be best described as "demonstration of concepts" rather than "proof of concepts". There exists almost no independent validation of reported findings away from designer/host sites, and the extrapolation of key results to general orthopaedic sites is limited. CONCLUSION While AI has certainly shown value in a small number of specific TKA-associated applications, the majority to date have focused on risk, cost and outcome prediction, rather than surgical care, per se. Extensive future work is needed to demonstrate external validity and reliability in non-designer settings. Well-performed studies are warranted to ensure that the scientific evidence base supporting the use of AI in knee arthroplasty matches the global hype.
Collapse
Affiliation(s)
- Andrew P Kurmis
- Discipline of Medical Specialties, University of Adelaide, Adelaide, SA, 5005, Australia.
- Department of Orthopaedic Surgery, Lyell McEwin Hospital, Haydown Road, Elizabeth Vale, SA, 5112, Australia.
- College of Medicine & Public Health, Flinders University, Bedford Park, SA, 5042, Australia.
| |
Collapse
|
15
|
Mueller B, Street WN, Carnahan RM, Lee S. Evaluating the performance of machine learning methods for risk estimation of delirium in patients hospitalized from the emergency department. Acta Psychiatr Scand 2023; 147:493-505. [PMID: 36999191 PMCID: PMC10147581 DOI: 10.1111/acps.13551] [Citation(s) in RCA: 6] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 07/02/2022] [Revised: 03/06/2023] [Accepted: 03/23/2023] [Indexed: 04/01/2023]
Abstract
INTRODUCTION Delirium is a cerebral dysfunction seen commonly in the acute care setting. It is associated with increased mortality and morbidity and is frequently missed in the emergency department (ED) and inpatient care by clinical gestalt alone. Identifying those at risk of delirium may help prioritize screening and interventions in the hospital setting. OBJECTIVE Our objective was to leverage electronic health records to identify a clinically valuable risk estimation model for prevalent delirium in patients being transferred from the ED to inpatient units. METHODS This was a retrospective cohort study to develop and validate a risk model to detect delirium using patient data available from prior visits and ED encounter. Electronic health records were extracted for patients hospitalized from the ED between January 1, 2014, and December 31, 2020. Eligible patients were aged 65 or older, admitted to an inpatient unit from the emergency department, and had at least one DOSS assessment or CAM-ICU recorded within 72 h of hospitalization. Six machine learning models were developed to estimate the risk of delirium using clinical variables including demographic features, physiological measurements, medications administered, lab results, and diagnoses. RESULTS A total of 28,531 patients met the inclusion criteria with 8057 (28.4%) having a positive delirium screening within the outcome observation period. Machine learning models were compared using the area under the receiver operating curve (AUC). The gradient boosted machine achieved the best performance with an AUC of 0.839 (95% CI, 0.837-0.841). At a 90% sensitivity threshold, this model achieved a specificity of 53.5% (95% CI 53.0%-54.0%) a positive predictive value of 43.5% (95% CI 43.2%-43.9%), and a negative predictive value of 93.1% (95% CI 93.1%-93.2%). A random forest model and L1-penalized logistic regression also demonstrated notable performance with AUCs of 0.837 (95% CI, 0.835-0.838) and 0.831 (95% CI, 0.830-0.833) respectively. CONCLUSION This study demonstrated the use of machine learning algorithms to identify a combination of variables that enables an estimation of risk of positive delirium screens early in hospitalization to develop prevention or management protocols.
Collapse
Affiliation(s)
- Brianna Mueller
- Tippie College of Business, The University of Iowa, Iowa City, Iowa, USA
| | - W Nick Street
- Tippie College of Business, The University of Iowa, Iowa City, Iowa, USA
| | - Ryan M Carnahan
- Department of Epidemiology, The University of Iowa College of Public Health, Iowa City, Iowa, USA
| | - Sangil Lee
- Department of Emergency Medicine, The University of Iowa, Iowa City, Iowa, USA
| |
Collapse
|