1
|
Fares MY, Liu HH, da Silva Etges APB, Zhang B, Warner JJP, Olson JJ, Fedorka CJ, Khan AZ, Best MJ, Kirsch JM, Simon JE, Sanders B, Costouros JG, Zhang X, Jones P, Haas DA, Abboud JA. Utility of Machine Learning, Natural Language Processing, and Artificial Intelligence in Predicting Hospital Readmissions After Orthopaedic Surgery: A Systematic Review and Meta-Analysis. JBJS Rev 2024; 12:01874474-202408000-00011. [PMID: 39172864 DOI: 10.2106/jbjs.rvw.24.00075] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 08/24/2024]
Abstract
BACKGROUND Numerous applications and strategies have been utilized to help assess the trends and patterns of readmissions after orthopaedic surgery in an attempt to extrapolate possible risk factors and causative agents. The aim of this work is to systematically summarize the available literature on the extent to which natural language processing, machine learning, and artificial intelligence (AI) can help improve the predictability of hospital readmissions after orthopaedic and spine surgeries. METHODS This is a systematic review and meta-analysis. PubMed, Embase and Google Scholar were searched, up until August 30, 2023, for studies that explore the use of AI, natural language processing, and machine learning tools for the prediction of readmission rates after orthopedic procedures. Data regarding surgery type, patient population, readmission outcomes, advanced models utilized, comparison methods, predictor sets, the inclusion of perioperative predictors, validation method, size of training and testing sample, accuracy, and receiver operating characteristics (C-statistic), among other factors, were extracted and assessed. RESULTS A total of 26 studies were included in our final dataset. The overall summary C-statistic showed a mean of 0.71 across all models, indicating a reasonable level of predictiveness. A total of 15 articles (57%) were attributed to the spine, making it the most commonly explored orthopaedic field in our study. When comparing accuracy of prediction models between different fields, models predicting readmissions after hip/knee arthroplasty procedures had a higher prediction accuracy (mean C-statistic = 0.79) than spine (mean C-statistic = 0.7) and shoulder (mean C-statistic = 0.67). In addition, models that used single institution data, and those that included intraoperative and/or postoperative outcomes, had a higher mean C-statistic than those utilizing other data sources, and that include only preoperative predictors. According to the Prediction model Risk of Bias Assessment Tool, the majority of the articles in our study had a high risk of bias. CONCLUSION AI tools perform reasonably well in predicting readmissions after orthopaedic procedures. Future work should focus on standardizing study methodologies and designs, and improving the data analysis process, in an attempt to produce more reliable and tangible results. LEVEL OF EVIDENCE Level III. See Instructions for Authors for a complete description of levels of evidence.
Collapse
Affiliation(s)
- Mohamad Y Fares
- Rothman Institute, Thomas Jefferson University Hospital, Philadelphia, Pennsylvania
| | | | | | | | - Jon J P Warner
- Department of Orthopaedic Surgery, Harvard Medical School, Boston Shoulder Institute, Massachusetts General Hospital, Boston, Massachusetts
| | | | - Catherine J Fedorka
- Cooper Bone and Joint Institute, Cooper University Hospital, Camden, New Jersey
| | - Adam Z Khan
- Department of Orthopaedic Surgery, Southern California Permanente Medical Group, Panorama City, California
| | - Matthew J Best
- Department of Orthopaedic Surgery, Johns Hopkins Hospital, Johns Hopkins University School of Medicine, Baltimore, Maryland
| | - Jacob M Kirsch
- Department of Orthopaedic Surgery, New England Baptist Hospital, Tufts University School of Medicine, Boston, Massachusetts
| | - Jason E Simon
- Department of Orthopaedic Surgery, Massachusetts General Hospital/Newton-Wellesley Hospital, Boston, Massachusetts
| | - Brett Sanders
- Center for Sports Medicine and Orthopaedics, Chattanooga, Tennessee
| | - John G Costouros
- Institute for Joint Restoration and Research, California Shoulder Center, Menlo Park, California
| | | | | | | | - Joseph A Abboud
- Rothman Institute, Thomas Jefferson University Hospital, Philadelphia, Pennsylvania
| |
Collapse
|
2
|
Oeding JF, Pareek A, Nieboer MJ, Rhodes NG, Tiegs-Heiden CA, Camp CL, Martin RK, Moatshe G, Engebretsen L, Sanchez-Sotelo J. A Machine Learning Model Demonstrates Excellent Performance in Predicting Subscapularis Tears Based on Pre-Operative Imaging Parameters Alone. Arthroscopy 2024; 40:1044-1055. [PMID: 37716627 DOI: 10.1016/j.arthro.2023.08.084] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 02/28/2023] [Revised: 08/21/2023] [Accepted: 08/21/2023] [Indexed: 09/18/2023]
Abstract
PURPOSE To develop a machine learning model capable of identifying subscapularis tears before surgery based on imaging and physical examination findings. METHODS Between 2010 and 2020, 202 consecutive shoulders underwent arthroscopic rotator cuff repair by a single surgeon. Patient demographics, physical examination findings (including range of motion, weakness with internal rotation, lift/push-off test, belly press test, and bear hug test), and imaging (including direct and indirect signs of tearing, biceps status, fatty atrophy, cystic changes, and other similar findings) were included for model creation. RESULTS Sixty percent of the shoulders had partial or full thickness tears of the subscapularis verified during surgery (83% of these were upper third). Using only preoperative imaging-related parameters, the XGBoost model demonstrated excellent performance at predicting subscapularis tears (c-statistic, 0.84; accuracy, 0.85; F1 score, 0.87). The top 5 features included direct signs related to the presence of tearing as evidenced on magnetic resonance imaging (MRI) (changes in tendon morphology and signal), as well as the quality of the MRI and biceps pathology. CONCLUSIONS In this study, machine learning was successful in predicting subscapularis tears by MRI alone in 85% of patients, and this accuracy did not decrease by isolating the model to the top features. The top five features included direct signs related to the presence of tearing as evidenced on MRI (changes in tendon morphology and signal), as well as the quality of the MRI and biceps pathology. Last, in advanced modeling, the addition of physical examination or patient characteristics did not make a significant difference in the predictive ability of this model. LEVEL OF EVIDENCE Level III, diagnostic case-control study.
Collapse
Affiliation(s)
- Jacob F Oeding
- School of Medicine, Mayo Clinic Alix School of Medicine, Rochester, Minnesota, U.S.A.; Oslo Sports Trauma Research Center, Norwegian School of Sport Sciences, Oslo, Norway
| | - Ayoosh Pareek
- Sports Medicine and Shoulder Service, Hospital for Special Surgery, New York, New York, U.S.A.; Oslo Sports Trauma Research Center, Norwegian School of Sport Sciences, Oslo, Norway
| | - Micah J Nieboer
- Department of Orthopedic Surgery and Sports Medicine, Mayo Clinic, Rochester, Minnesota, U.S.A
| | | | | | - Christopher L Camp
- Department of Orthopedic Surgery and Sports Medicine, Mayo Clinic, Rochester, Minnesota, U.S.A
| | - R Kyle Martin
- Department of Orthopedic Surgery, University of Minnesota, Minneapolis, Minnesota, U.S.A.; Oslo Sports Trauma Research Center, Norwegian School of Sport Sciences, Oslo, Norway
| | - Gilbert Moatshe
- Oslo Sports Trauma Research Center, Norwegian School of Sport Sciences, Oslo, Norway
| | - Lars Engebretsen
- Oslo Sports Trauma Research Center, Norwegian School of Sport Sciences, Oslo, Norway
| | - Joaquin Sanchez-Sotelo
- Department of Orthopedic Surgery and Sports Medicine, Mayo Clinic, Rochester, Minnesota, U.S.A..
| |
Collapse
|
3
|
Oeding JF, Yang L, Sanchez-Sotelo J, Camp CL, Karlsson J, Samuelsson K, Pearle AD, Ranawat AS, Kelly BT, Pareek A. A practical guide to the development and deployment of deep learning models for the orthopaedic surgeon: Part III, focus on registry creation, diagnosis, and data privacy. Knee Surg Sports Traumatol Arthrosc 2024; 32:518-528. [PMID: 38426614 DOI: 10.1002/ksa.12085] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 12/13/2023] [Revised: 01/22/2024] [Accepted: 01/23/2024] [Indexed: 03/02/2024]
Abstract
Deep learning is a subset of artificial intelligence (AI) with enormous potential to transform orthopaedic surgery. As has already become evident with the deployment of Large Language Models (LLMs) like ChatGPT (OpenAI Inc.), deep learning can rapidly enter clinical and surgical practices. As such, it is imperative that orthopaedic surgeons acquire a deeper understanding of the technical terminology, capabilities and limitations associated with deep learning models. The focus of this series thus far has been providing surgeons with an overview of the steps needed to implement a deep learning-based pipeline, emphasizing some of the important technical details for surgeons to understand as they encounter, evaluate or lead deep learning projects. However, this series would be remiss without providing practical examples of how deep learning models have begun to be deployed and highlighting the areas where the authors feel deep learning may have the most profound potential. While computer vision applications of deep learning were the focus of Parts I and II, due to the enormous impact that natural language processing (NLP) has had in recent months, NLP-based deep learning models are also discussed in this final part of the series. In this review, three applications that the authors believe can be impacted the most by deep learning but with which many surgeons may not be familiar are discussed: (1) registry construction, (2) diagnostic AI and (3) data privacy. Deep learning-based registry construction will be essential for the development of more impactful clinical applications, with diagnostic AI being one of those applications likely to augment clinical decision-making in the near future. As the applications of deep learning continue to grow, the protection of patient information will become increasingly essential; as such, applications of deep learning to enhance data privacy are likely to become more important than ever before. Level of Evidence: Level IV.
Collapse
Affiliation(s)
- Jacob F Oeding
- School of Medicine, Mayo Clinic Alix School of Medicine, Rochester, Minnesota, USA
- Department of Orthopaedics, Institute of Clinical Sciences, The Sahlgrenska Academy, University of Gothenburg, Gothenburg, Sweden
| | - Linjun Yang
- Orthopedic Surgery Artificial Intelligence Laboratory (OSAIL), Department of Orthopedic Surgery, Mayo Clinic, Rochester, Minnesota, USA
| | | | - Christopher L Camp
- Department of Orthopedic Surgery, Mayo Clinic, Rochester, Minnesota, USA
| | - Jón Karlsson
- Department of Orthopaedics, Sahlgrenska University Hospital, Sahlgrenska Academy, Gothenburg University, Gothenburg, Sweden
| | - Kristian Samuelsson
- Department of Orthopaedics, Sahlgrenska University Hospital, Sahlgrenska Academy, Gothenburg University, Gothenburg, Sweden
| | - Andrew D Pearle
- Sports Medicine and Shoulder Service, Hospital for Special Surgery, New York, New York, USA
| | - Anil S Ranawat
- Sports Medicine and Shoulder Service, Hospital for Special Surgery, New York, New York, USA
| | - Bryan T Kelly
- Sports Medicine and Shoulder Service, Hospital for Special Surgery, New York, New York, USA
| | - Ayoosh Pareek
- Sports Medicine and Shoulder Service, Hospital for Special Surgery, New York, New York, USA
| |
Collapse
|
4
|
Huffman N, Pasqualini I, Khan ST, Klika AK, Deren ME, Jin Y, Kunze KN, Piuzzi NS. Enabling Personalized Medicine in Orthopaedic Surgery Through Artificial Intelligence: A Critical Analysis Review. JBJS Rev 2024; 12:01874474-202403000-00006. [PMID: 38466797 DOI: 10.2106/jbjs.rvw.23.00232] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 03/13/2024]
Abstract
» The application of artificial intelligence (AI) in the field of orthopaedic surgery holds potential for revolutionizing health care delivery across 3 crucial domains: (I) personalized prediction of clinical outcomes and adverse events, which may optimize patient selection, surgical planning, and enhance patient safety and outcomes; (II) diagnostic automated and semiautomated imaging analyses, which may reduce time burden and facilitate precise and timely diagnoses; and (III) forecasting of resource utilization, which may reduce health care costs and increase value for patients and institutions.» Computer vision is one of the most highly studied areas of AI within orthopaedics, with applications pertaining to fracture classification, identification of the manufacturer and model of prosthetic implants, and surveillance of prosthesis loosening and failure.» Prognostic applications of AI within orthopaedics include identifying patients who will likely benefit from a specified treatment, predicting prosthetic implant size, postoperative length of stay, discharge disposition, and surgical complications. Not only may these applications be beneficial to patients but also to institutions and payors because they may inform potential cost expenditure, improve overall hospital efficiency, and help anticipate resource utilization.» AI infrastructure development requires institutional financial commitment and a team of clinicians and data scientists with expertise in AI that can complement skill sets and knowledge. Once a team is established and a goal is determined, teams (1) obtain, curate, and label data; (2) establish a reference standard; (3) develop an AI model; (4) evaluate the performance of the AI model; (5) externally validate the model, and (6) reinforce, improve, and evaluate the model's performance until clinical implementation is possible.» Understanding the implications of AI in orthopaedics may eventually lead to wide-ranging improvements in patient care. However, AI, while holding tremendous promise, is not without methodological and ethical limitations that are essential to address. First, it is important to ensure external validity of programs before their use in a clinical setting. Investigators should maintain high quality data records and registry surveillance, exercise caution when evaluating others' reported AI applications, and increase transparency of the methodological conduct of current models to improve external validity and avoid propagating bias. By addressing these challenges and responsibly embracing the potential of AI, the medical field may eventually be able to harness its power to improve patient care and outcomes.
Collapse
Affiliation(s)
- Nickelas Huffman
- Cleveland Clinic, Department of Orthopaedic Surgery, Cleveland, Ohio
| | | | - Shujaa T Khan
- Cleveland Clinic, Department of Orthopaedic Surgery, Cleveland, Ohio
| | - Alison K Klika
- Cleveland Clinic, Department of Orthopaedic Surgery, Cleveland, Ohio
| | - Matthew E Deren
- Cleveland Clinic, Department of Orthopaedic Surgery, Cleveland, Ohio
| | - Yuxuan Jin
- Cleveland Clinic, Department of Orthopaedic Surgery, Cleveland, Ohio
| | - Kyle N Kunze
- Department of Orthopaedic Surgery, Hospital for Special Surgery, New York, New York
| | - Nicolas S Piuzzi
- Cleveland Clinic, Department of Orthopaedic Surgery, Cleveland, Ohio
- Department of Biomedical Engineering, Cleveland Clinic Foundation, Cleveland, Ohio
| |
Collapse
|
5
|
Kunze KN, Williams RJ, Ranawat AS, Pearle AD, Kelly BT, Karlsson J, Martin RK, Pareek A. Artificial intelligence (AI) and large data registries: Understanding the advantages and limitations of contemporary data sets for use in AI research. Knee Surg Sports Traumatol Arthrosc 2024; 32:13-18. [PMID: 38226678 DOI: 10.1002/ksa.12018] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 11/09/2023] [Accepted: 11/27/2023] [Indexed: 01/17/2024]
Affiliation(s)
- Kyle N Kunze
- Department of Orthopaedic Surgery, Hospital for Special Surgery, New York, New York, USA
- Sports Medicine and Shoulder Service, Hospital for Special Surgery, New York, New York, USA
| | - Riley J Williams
- Department of Orthopaedic Surgery, Hospital for Special Surgery, New York, New York, USA
- Sports Medicine and Shoulder Service, Hospital for Special Surgery, New York, New York, USA
| | - Anil S Ranawat
- Department of Orthopaedic Surgery, Hospital for Special Surgery, New York, New York, USA
- Sports Medicine and Shoulder Service, Hospital for Special Surgery, New York, New York, USA
| | - Andrew D Pearle
- Department of Orthopaedic Surgery, Hospital for Special Surgery, New York, New York, USA
- Sports Medicine and Shoulder Service, Hospital for Special Surgery, New York, New York, USA
| | - Bryan T Kelly
- Department of Orthopaedic Surgery, Hospital for Special Surgery, New York, New York, USA
- Sports Medicine and Shoulder Service, Hospital for Special Surgery, New York, New York, USA
| | - Jon Karlsson
- Department of Orthopaedics, Sahlgrenska University Hospital, Sahlgrenska Academy, Gothenburg University, Gothenburg, Sweden
| | - R Kyle Martin
- Department of Orthopedic Surgery, University of Minnesota, Minneapolis, Minnesota, USA
| | - Ayoosh Pareek
- Department of Orthopaedic Surgery, Hospital for Special Surgery, New York, New York, USA
- Sports Medicine and Shoulder Service, Hospital for Special Surgery, New York, New York, USA
| |
Collapse
|