1
|
Belt M, Smulders K, Schreurs BW, Hannink G. Clinical prediction models for patients undergoing total hip arthroplasty: an external validation based on a systematic review and the Dutch Arthroplasty Register. Acta Orthop 2024; 95:685-694. [PMID: 39584823 PMCID: PMC11587164 DOI: 10.2340/17453674.2024.42449] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 09/25/2023] [Accepted: 11/03/2024] [Indexed: 11/26/2024] Open
Abstract
BACKGROUND AND PURPOSE External validation is a crucial step after prediction model development. Despite increasing interest in prediction models, external validation is frequently overlooked. We aimed to evaluate whether joint registries can be utilized for external validation of prediction models, and whether published prediction models are valid for the Dutch population with a total hip arthroplasty. METHODS We identified prediction models developed in patients undergoing arthroplasty through a systematic literature search. Model variables were evaluated for availability in the Dutch Arthroplasty Registry (LROI). We assessed the model performance in terms of calibration and discrimination (area under the curve [AUC]). Furthermore, the models were updated and evaluated through intercept recalibration and logistic recalibration. RESULTS After assessing 54 papers, 19 were excluded for not describing a prediction model (n = 16) or focusing on non-TJA populations (n = 3), leaving 35 papers describing 44 prediction models. 90% (40/44) of the prediction models used outcomes or predictors missing in the LROI, such as diabetes, opioid use, and depression. 4 models could be externally validated on LROI data. The models' discrimination ranged between poor and acceptable and was similar to that in the development cohort. The calibration of the models was insufficient. The model performance improved slightly after updating. CONCLUSION External validation of the 4 models resulted in suboptimal predictive performance in the Dutch population, highlighting the importance of external validation studies.
Collapse
Affiliation(s)
- Maartje Belt
- Research Department, Sint Maartenskliniek, Nijmegen; Department of Orthopaedics, Radboud University Medical Center, Nijmegen, The Netherlands.
| | - Katrijn Smulders
- Research Department, Sint Maartenskliniek, Nijmegen, The Netherlands
| | - B Willem Schreurs
- Department of Orthopaedics, Radboud University Medical Center, Nijmegen; Dutch Arthroplasty Register (Landelijke Registratie Orthopedische Interventies), 's-Hertogenbosch, The Netherlands
| | - Gerjon Hannink
- Department of Medical Imaging, Radboud university medical center, Nijmegen, The Netherlands
| |
Collapse
|
2
|
Oettl FC, Oeding JF, Feldt R, Ley C, Hirschmann MT, Samuelsson K. The artificial intelligence advantage: Supercharging exploratory data analysis. Knee Surg Sports Traumatol Arthrosc 2024; 32:3039-3042. [PMID: 39082872 DOI: 10.1002/ksa.12389] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 07/13/2024] [Accepted: 07/13/2024] [Indexed: 10/30/2024]
Abstract
Explorative data analysis (EDA) is a critical step in scientific projects, aiming to uncover valuable insights and patterns within data. Traditionally, EDA involves manual inspection, visualization, and various statistical methods. The advent of artificial intelligence (AI) and machine learning (ML) has the potential to improve EDA, offering more sophisticated approaches that enhance its efficacy. This review explores how AI and ML algorithms can improve feature engineering and selection during EDA, leading to more robust predictive models and data-driven decisions. Tree-based models, regularized regression, and clustering algorithms were identified as key techniques. These methods automate feature importance ranking, handle complex interactions, perform feature selection, reveal hidden groupings, and detect anomalies. Real-world applications include risk prediction in total hip arthroplasty and subgroup identification in scoliosis patients. Recent advances in explainable AI and EDA automation show potential for further improvement. The integration of AI and ML into EDA accelerates tasks and uncovers sophisticated insights. However, effective utilization requires a deep understanding of the algorithms, their assumptions, and limitations, along with domain knowledge for proper interpretation. As data continues to grow, AI will play an increasingly pivotal role in EDA when combined with human expertise, driving more informed, data-driven decision-making across various scientific domains. Level of Evidence: Level V - Expert opinion.
Collapse
Affiliation(s)
- Felix C Oettl
- Hospital for Special Surgery, New York, New York, USA
- Department of Orthopedic Surgery, Balgrist University Hospital, University of Zürich, Zurich, Switzerland
| | - Jacob F Oeding
- Department of Orthopaedics, Institute of Clinical Sciences, Sahlgrenska Academy, University of Gothenburg, Gothenburg, Sweden
- Sahlgrenska Sports Medicine Center, Göteborg, Sweden
- Mayo Clinic Alix School of Medicine, Mayo Clinic, Rochester, Minnesota, USA
| | - Robert Feldt
- Department of Computer Science and Engineering, Chalmers University of Technology, Gothenburg, Sweden
| | - Christophe Ley
- Department of Mathematics, University of Luxembourg, Esch-sur-Alzette, Luxembourg
| | - Michael T Hirschmann
- Department of Orthopedic Surgery and Traumatology, Kantonspital Baselland, Liestal, Switzerland
| | - Kristian Samuelsson
- Department of Orthopaedics, Institute of Clinical Sciences, Sahlgrenska Academy, University of Gothenburg, Gothenburg, Sweden
- Sahlgrenska Sports Medicine Center, Göteborg, Sweden
- Department of Orthopaedics, Sahlgrenska University Hospital, Mölndal, Sweden
| |
Collapse
|
3
|
Lübbeke A, Cullati S, Baréa C, Cole S, Fabiano G, Silman A, Gutacker N, Agoritsas T, Hannouche D, Pinedo-Villanueva R. Development of a patient-centred tool for use in total hip arthroplasty. PLoS One 2024; 19:e0307752. [PMID: 39446871 PMCID: PMC11500863 DOI: 10.1371/journal.pone.0307752] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/20/2023] [Accepted: 07/08/2024] [Indexed: 10/26/2024] Open
Abstract
BACKGROUND The aim of this project was to develop a tool using the experience of previous patients to inform patient-centred clinical decision-making in the context of total hip arthroplasty (THA). We sought out the patients' views on what is important for them, leveraging registry data, and providing outcome information that is perceived as relevant, understandable, adapted to a specific patient's profile, and readily available. METHODS We created the information tool "Patients like me" in four steps. (1) The knowledge basis was the systematically collected detailed exposure and outcome information from the Geneva Arthroplasty Registry established 1996. (2) From the registry we randomly selected 275 patients about to undergo or having already undergone THA and asked them via interviews and a survey which benefits and harms associated with the operation and daily life with the prosthesis they perceived as most important. (3) The identified relevant data (39 predictor candidates, 15 outcomes) were evaluated using Conditional Inference Trees analysis to construct a classification algorithm for each of the 15 outcomes at three different time points/periods. Internal validity of the results was tested using bootstrapping. (4) The tool was designed by and pre-tested with patients over several iterations. RESULTS Data from 6836 primary elective THAs operated between 1996 and 2019 were included. The trajectories for the 15 outcomes from the domains pain relief, activity improvement, complication (infection, dislocation, peri-prosthetic fracture) and what to expect in the future (revision surgery, need for contralateral hip replacement) over up to 20 years after surgery were presented for all patients and for specific patient profiles. The tool was adapted to various purposes including individual use, group sessions, patient-clinician interaction and surgeon information to complement the preoperative planning. The pre-test patients' feedback to the tool was unanimously positive. They considered it interesting, clear, complete, and complementary to other information received. CONCLUSION The tool based on a survey of patients' perceived concerns and interests and the corresponding long-term data from a large institutional registry makes past patients' experience accessible, understandable, and visible for today's patients and their clinicians. It is a comprehensive illustration of trajectories of relevant outcomes from previous "Patients like me". This principle and methodology can be applied in other medical fields.
Collapse
Affiliation(s)
- Anne Lübbeke
- Division of Orthopaedics & Trauma Surgery, Geneva University Hospitals and University of Geneva, Geneva, Switzerland
- Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, University of Oxford, Oxford, United Kingdom
| | - Stéphane Cullati
- Quality of Care Service, University Hospitals of Geneva & Department of Readaptation and Geriatrics, University of Geneva, Geneva, Switzerland
| | - Christophe Baréa
- Division of Orthopaedics & Trauma Surgery, Geneva University Hospitals and University of Geneva, Geneva, Switzerland
| | - Sophie Cole
- Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, University of Oxford, Oxford, United Kingdom
| | - Gianluca Fabiano
- Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, University of Oxford, Oxford, United Kingdom
| | - Alan Silman
- Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, University of Oxford, Oxford, United Kingdom
| | - Nils Gutacker
- Centre for Health Economics, University of York, York, United Kingdom
| | - Thomas Agoritsas
- Division General Internal Medicine, Geneva University Hospitals and University of Geneva, Geneva, Switzerland
| | - Didier Hannouche
- Division of Orthopaedics & Trauma Surgery, Geneva University Hospitals and University of Geneva, Geneva, Switzerland
| | - Rafael Pinedo-Villanueva
- Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, University of Oxford, Oxford, United Kingdom
| |
Collapse
|
4
|
Venäläinen MS, Panula VJ, Eskelinen AP, Fenstad AM, Furnes O, Hallan G, Rolfson O, Kärrholm J, Hailer NP, Pedersen AB, Overgaard S, Mäkelä KT, Elo LL. Prediction of Early Adverse Events After THA: A Comparison of Different Machine-Learning Strategies Based on 262,356 Observations From the Nordic Arthroplasty Register Association (NARA) Dataset. ACR Open Rheumatol 2024; 6:669-677. [PMID: 39040016 PMCID: PMC11471944 DOI: 10.1002/acr2.11709] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/10/2023] [Revised: 04/20/2024] [Accepted: 06/08/2024] [Indexed: 07/24/2024] Open
Abstract
OBJECTIVE Preoperative risk prediction models can support shared decision-making before total hip arthroplasties (THAs). Here, we compare different machine-learning (ML) approaches to predict the six-month risk of adverse events following primary THA to obtain accurate yet simple-to-use risk prediction models. METHODS We extracted data on primary THAs (N = 262,356) between 2010 and 2018 from the Nordic Arthroplasty Register Association dataset. We benchmarked a variety of ML algorithms in terms of the area under the receiver operating characteristic curve (AUROC) for predicting the risk of revision caused by periprosthetic joint infection (PJI), dislocation or periprosthetic fracture (PPF), and death. All models were internally validated against a randomly selected test cohort (one-third of the data) that was not used for training the models. RESULTS The incidences of revisions because of PJI, dislocation, and PPF were 0.8%, 0.4%, and 0.3%, respectively, and the incidence of death was 1.2%. Overall, Lasso regression with stable iterative variable selection (SIVS) produced models using only four to five input variables but with AUROC comparable to more complex models using all 32 variables available. The SIVS-based Lasso models based on age, sex, preoperative diagnosis, bearing couple, fixation, and surgical approach predicted the risk of revisions caused by PJI, dislocations, and PPF, as well as death, with AUROCs of 0.61, 0.67, 0.76, and 0.86, respectively. CONCLUSION Our study demonstrates that satisfactory predictive potential for adverse events following THA can be reached with parsimonious modeling strategies. The SIVS-based Lasso models may serve as simple-to-use tools for clinical risk assessment in the future.
Collapse
Affiliation(s)
- Mikko S Venäläinen
- Turku University Hospital, University of Turku and Åbo Akademi University, Turku, Finland
| | | | - Antti P Eskelinen
- Coxa Hospital for Joint Replacement and University of Tampere, Tampere, Finland, and the Finnish Arthroplasty Register, Finnish Institute for Health and Welfare, Helsinki, Finland
| | | | - Ove Furnes
- Haukeland University Hospital and University of Bergen, Bergen, Norway
| | - Geir Hallan
- Haukeland University Hospital and University of Bergen, Bergen, Norway
| | - Ola Rolfson
- University of Gothenburg, Gothenburg, Sweden
| | | | | | - Alma B Pedersen
- Aarhus University Hospital and Aarhus University, Aarhus, Denmark
| | - Søren Overgaard
- Copenhagen University Hospital and University of Copenhagen, Copenhagen, Denmark
| | - Keijo T Mäkelä
- Turku University Hospital and University of Turku, Turku, Finland, and the Finnish Arthroplasty Register, Finnish Institute for Health and Welfare, Helsinki, Finland
| | - Laura L Elo
- University of Turku and Åbo Akademi University, Turku, Finland
| |
Collapse
|
5
|
Zalikha AK, Pham L, Keeley J, Hussein IH, El-Othmani MM. Frailty Among Total Hip and Knee Arthroplasty Recipients: Epidemiology and Propensity Score-weighted Analysis of Effect on In-hospital Postoperative Outcomes. J Am Acad Orthop Surg 2023; 31:292-299. [PMID: 36728666 DOI: 10.5435/jaaos-d-22-00642] [Citation(s) in RCA: 6] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 07/07/2022] [Accepted: 11/19/2022] [Indexed: 02/03/2023] Open
Abstract
INTRODUCTION Total joint arthroplasty (TJA) is one of the most successful and frequently performed procedures in the United States. The number of these procedures is projected to continue growing rapidly in the coming years, and with it comes the demand for more sophisticated perioperative risk and complication assessment. This study examines the effect of frailty on postoperative inpatient complications and hospital resource utilization after TJA. METHODS Discharge data from the National Inpatient Sample were used to identify all patients aged 50 years or older who underwent TJA between 2006 and 2015. Nonelective admissions and hip fractures were excluded. Patients were stratified into two groups with and without concomitant ICD-9 diagnostic criteria that qualified them has having frailty. An analysis comparing the 2 groups' epidemiology, medical comorbidities, and propensity score-weighted postoperative clinical and economic outcomes was done. RESULTS A total of 7,854,890 TJAs were included in this analysis, with 136,516 meeting the criteria for frailty and 7,718,374 being nonfrail. Among these patients, the average age was 67.3 years and the female distribution was 61.1%. Frail patients were found to have markedly higher rates of all but two individual comorbidities constituting the Modified Elixhauser Profile compared with nonfrail patients. Compared with the control group, frail patients were found to have increased risk of any postoperative complication, central nervous system complications, hematoma/seroma, wound dehiscence, infection, and postoperative anemia. Frail patients also had longer length of stay, higher discharge to rehabilitation facilities, and higher hospital charges. DISCUSSION Patients with frailty undergoing TJA procedures are at a markedly higher risk for developing postoperative complications and worse hospital economic outcomes. As this patient population continues to increase, it is imperative for clinicians to use their risk factors in optimizing their perioperative care and support.
Collapse
Affiliation(s)
- Abdul K Zalikha
- From the Department of Orthopaedic Surgery and Sports Medicine, Detroit Medical Center, Detroit, MI, (Zalikha), Oakland University William Beaumont School of Medicine, Auburn Hills, MI (Pham), the Department of Biomedical Sciences, Oakland University William Beaumont School of Medicine, Auburn Hills, MI (Keeley and Hussein), and the Department of Orthopaedic Surgery, Columbia University Medical Center, New York, NY (El-Othmani)
| | | | | | | | | |
Collapse
|
6
|
Appiah KOB, Khunti K, Kelly BM, Innes AQ, Liao Z, Dymond M, Middleton RG, Wainwright TW, Yates T, Zaccardi F. Patient-rated satisfaction and improvement following hip and knee replacements: Development of prediction models. J Eval Clin Pract 2023; 29:300-311. [PMID: 36172971 DOI: 10.1111/jep.13767] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 04/13/2022] [Revised: 08/18/2022] [Accepted: 08/21/2022] [Indexed: 12/01/2022]
Abstract
RATIONALE Effective preoperative assessments of determinants of health status and function may improve postoperative outcomes. AIMS AND OBJECTIVES We developed risk scores of preoperative patient factors and patient-reported outcome measures (PROMs) as predictors of patient-rated satisfaction and improvement following hip and knee replacements. PATIENTS AND METHODS Prospectively collected National Health Service and independent sector patient data (n = 30,457), including patients' self-reported demographics, comorbidities, PROMs (Oxford Hip/Knee score (OHS/OKS) and European Quality of Life (EQ5D index and health-scale), were analysed. Outcomes were defined as patient-reported satisfaction and improvement following surgery at 7-month follow-up. Univariable and multivariable-adjusted logistic regressions were undertaken to build prediction models; model discrimination was evaluated with the concordance index (c-index) and nomograms were developed to allow the estimation of probabilities. RESULTS Of the 14,651 subjects with responses for satisfaction following hip replacements 564 (3.8%) reported dissatisfaction, and 1433 (9.2%) of the 15,560 following knee replacement reported dissatisfaction. A total of 14,662 had responses for perceived improvement following hip replacement (lack of improvement in 391; 2.7%) and 15,588 following knee replacement (lack of improvements in 1092; 7.0%). Patients reporting poor outcomes had worse preoperative PROMs. Several factors, including age, gender, patient comorbidities and EQ5D, were included in the final prediction models: C-indices of these models were 0.613 and 0.618 for dissatisfaction and lack of improvement, respectively, for hip replacement and 0.614 and 0.598, respectively, for knee replacement. CONCLUSIONS Using easily accessible preoperative patient factors, including PROMs, we developed models which may help predict dissatisfaction and lack of improvement following hip and knee replacements and facilitate risk stratification and decision-making processes.
Collapse
Affiliation(s)
- Karen O B Appiah
- Leicester Diabetes Centre, Leicester General Hospital, University of Leicester, Leicester, UK.,Leicester Real World Evidence Unit, Leicester General Hospital, University of Leicester, Leicester, UK
| | - Kamlesh Khunti
- Leicester Diabetes Centre, Leicester General Hospital, University of Leicester, Leicester, UK.,Leicester Real World Evidence Unit, Leicester General Hospital, University of Leicester, Leicester, UK.,NIHR Applied Research Collaboration-East Midlands (ARC-EM), University Hospitals of Leicester NHS Trust and University of Leicester, Leicester, UK
| | | | | | | | | | - Robert G Middleton
- Nuffield Health, Epsom Gateway, Epsom, UK.,Orthopaedic Research Institute, Bournemouth University, Poole, UK
| | - Thomas W Wainwright
- Nuffield Health, Epsom Gateway, Epsom, UK.,Orthopaedic Research Institute, Bournemouth University, Poole, UK
| | - Thomas Yates
- Leicester Diabetes Centre, Leicester General Hospital, University of Leicester, Leicester, UK.,NIHR Leicester Biomedical Research Centre, University Hospitals of Leicester NHS Trust and University of Leicester, Leicester, UK
| | - Francesco Zaccardi
- Leicester Diabetes Centre, Leicester General Hospital, University of Leicester, Leicester, UK.,Leicester Real World Evidence Unit, Leicester General Hospital, University of Leicester, Leicester, UK
| |
Collapse
|
7
|
Perni S, Bojan B, Prokopovich P. A retrospective study of risk factors, causative micro-organisms and healthcare resources consumption associated with prosthetic joint infections (PJI) using the Clinical Practice Research Datalink (CPRD) Aurum database. PLoS One 2023; 18:e0282709. [PMID: 36943830 PMCID: PMC10030031 DOI: 10.1371/journal.pone.0282709] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/12/2022] [Accepted: 02/20/2023] [Indexed: 03/23/2023] Open
Abstract
BACKGROUND Prosthetic joint infection (PJI) is a serious complication after joint replacement surgery and it is associated with risk of mortality and morbidity along with high direct costs. METHODS The Clinical Practice Research Datalink (CPRD) data were utilized to quantify PJI incidence after hip or knee replacement up to 5 years after implant and a variety of risk factors related to patient characteristics, medical and treatment history along with characteristics of the original surgery were analyzed through Cox proportional hazard. RESULTS 221,826 patients (individual joints 283,789) met all the inclusion and exclusion criteria of the study; during the study follow-up period (5 years), 707 and 695 PJIs were diagnosed in hip and knee, respectively. Patients undergoing joint replacement surgery during an unscheduled hospitalization had greater risk of PJI than patients whose surgery was elective; similarly, the risk of developing PJI after a secondary hip or knee replacement was about 4 times greater than after primary arthroplasty when adjusted for all other variables considered. A previous diagnosis of PJI, even in a different joint, increased the risk of a further PJI. Distribution of average LoS per each hospitalization caused by PJI exhibited a right skewed profile with median duration [IQR] duration of 16 days [8-32] and 13 days [7.25-32] for hip and knee, respectively. PJIs causative micro-organisms were dependent on the time between initial surgery and infection offset; early PJI were more likely to be multispecies than later (years after surgery); the identification of Gram- pathogens decreased with increasing post-surgery follow-up. CONCLUSIONS This study offers a contemporary assessment of the budgetary and capacity (number and duration of hospitalizations along with the number of Accident and Emergency (A&E) visits) posed by PJIs in UK for the national healthcare system (NHS). The results to provide risk management and planning tools to health providers and policy makers in order to fully assess technologies aimed at controlling and preventing PJI. The findings add to the existing evidence-based knowledge surrounding the epidemiology and burden of PJI by quantifying patterns of PJI in patients with a relatively broad set of prevalent comorbidities.
Collapse
Affiliation(s)
- Stefano Perni
- School of Pharmacy and Pharmaceutical Sciences, Cardiff University, Cardiff, United Kingdom
| | - Bsmah Bojan
- School of Pharmacy and Pharmaceutical Sciences, Cardiff University, Cardiff, United Kingdom
| | - Polina Prokopovich
- School of Pharmacy and Pharmaceutical Sciences, Cardiff University, Cardiff, United Kingdom
| |
Collapse
|
8
|
A Surgeon's Guide to Understanding Artificial Intelligence and Machine Learning Studies in Orthopaedic Surgery. Curr Rev Musculoskelet Med 2022; 15:121-132. [PMID: 35141847 DOI: 10.1007/s12178-022-09738-7] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Accepted: 01/17/2022] [Indexed: 10/19/2022]
Abstract
PURPOSE OF REVIEW In recent years, machine learning techniques have been increasingly utilized across medicine, impacting the practice and delivery of healthcare. The data-driven nature of orthopaedic surgery presents many targets for improvement through the use of artificial intelligence, which is reflected in the increasing number of publications in the medical literature. However, the unique methodologies utilized in AI studies can present a barrier to its widespread acceptance and use in orthopaedics. The purpose of our review is to provide a tool that can be used by practitioners to better understand and ultimately leverage AI studies. RECENT FINDINGS The increasing interest in machine learning across medicine is reflected in a greater utilization of AI in recent medical literature. The process of designing machine learning studies includes study design, model choice, data collection/handling, model development, training, testing, and interpretation. Recent studies leveraging ML in orthopaedics provide useful examples for future research endeavors. This manuscript intends to create a guide discussing the use of machine learning and artificial intelligence in orthopaedic surgery research. Our review outlines the process of creating a machine learning algorithm and discusses the different model types, utilizing examples from recent orthopaedic literature to illustrate the techniques involved.
Collapse
|
9
|
Venäläinen MS, Heervä E, Hirvonen O, Saraei S, Suomi T, Mikkola T, Bärlund M, Jyrkkiö S, Laitinen T, Elo LL. Improved risk prediction of chemotherapy-induced neutropenia-model development and validation with real-world data. Cancer Med 2021; 11:654-663. [PMID: 34859963 PMCID: PMC8817096 DOI: 10.1002/cam4.4465] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/31/2021] [Revised: 11/07/2021] [Accepted: 11/16/2021] [Indexed: 11/08/2022] Open
Abstract
BACKGROUND The existing risk prediction models for chemotherapy-induced febrile neutropenia (FN) do not necessarily apply to real-life patients in different healthcare systems and the external validation of these models are often lacking. Our study evaluates whether a machine learning-based risk prediction model could outperform the previously introduced models, especially when validated against real-world patient data from another institution not used for model training. METHODS Using Turku University Hospital electronic medical records, we identified all patients who received chemotherapy for non-hematological cancer between the years 2010 and 2017 (N = 5879). An experimental surrogate endpoint was first-cycle neutropenic infection (NI), defined as grade IV neutropenia with serum C-reactive protein >10 mg/l. For predicting the risk of NI, a penalized regression model (Lasso) was developed. The model was externally validated in an independent dataset (N = 4594) from Tampere University Hospital. RESULTS Lasso model accurately predicted NI risk with good accuracy (AUROC 0.84). In the validation cohort, the Lasso model outperformed two previously introduced, widely approved models, with AUROC 0.75. The variables selected by Lasso included granulocyte colony-stimulating factor (G-CSF) use, cancer type, pre-treatment neutrophil and thrombocyte count, intravenous treatment regimen, and the planned dose intensity. The same model predicted also FN, with AUROC 0.77, supporting the validity of NI as an endpoint. CONCLUSIONS Our study demonstrates that real-world NI risk prediction can be improved with machine learning and that every difference in patient or treatment characteristics can have a significant impact on model performance. Here we outline a novel, externally validated approach which may hold potential to facilitate more targeted use of G-CSFs in the future.
Collapse
Affiliation(s)
- Mikko S Venäläinen
- Turku Bioscience Centre, University of Turku and Åbo Akademi University, Turku, Finland
| | - Eetu Heervä
- Department of Oncology, Turku University Hospital and FICAN West, Turku, Finland.,University of Turku, Turku, Finland
| | - Outi Hirvonen
- Department of Oncology, Turku University Hospital and FICAN West, Turku, Finland.,Department of Clinical Oncology, University of Turku, Turku, Finland.,Palliative Center, Turku University Hospital, Turku, Finland
| | - Sohrab Saraei
- Turku Bioscience Centre, University of Turku and Åbo Akademi University, Turku, Finland
| | - Tomi Suomi
- Turku Bioscience Centre, University of Turku and Åbo Akademi University, Turku, Finland
| | - Toni Mikkola
- Tays Research Services, Clinical Informatics Team, Tampere University Hospital and University of Tampere, Tampere, Finland
| | - Maarit Bärlund
- Department of Oncology, Tays Cancer Centre, Tampere University Hospital, Tampere, Finland.,Faculty of Medicine and Health Technology, Tampere University, Tampere, Finland
| | - Sirkku Jyrkkiö
- Department of Oncology, Turku University Hospital and FICAN West, Turku, Finland
| | - Tarja Laitinen
- Department of Pulmonary Medicine, University of Turku and Turku University Hospital, Turku, Finland.,Administration Center, Tampere University Hospital and University of Tampere, Tampere, Finland
| | - Laura L Elo
- Turku Bioscience Centre, University of Turku and Åbo Akademi University, Turku, Finland.,Institute of Biomedicine, University of Turku, Turku, Finland
| |
Collapse
|
10
|
Mahmoudian M, Venäläinen MS, Klén R, Elo LL. Stable Iterative Variable Selection. Bioinformatics 2021; 37:4810-4817. [PMID: 34270690 PMCID: PMC8665768 DOI: 10.1093/bioinformatics/btab501] [Citation(s) in RCA: 13] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Abstract] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/30/2020] [Revised: 05/20/2021] [Accepted: 07/14/2021] [Indexed: 11/13/2022] Open
Abstract
Motivation The emergence of datasets with tens of thousands of features, such as high-throughput omics biomedical data, highlights the importance of reducing the feature space into a distilled subset that can truly capture the signal for research and industry by aiding in finding more effective biomarkers for the question in hand. A good feature set also facilitates building robust predictive models with improved interpretability and convergence of the applied method due to the smaller feature space. Results Here, we present a robust feature selection method named Stable Iterative Variable Selection (SIVS) and assess its performance over both omics and clinical data types. As a performance assessment metric, we compared the number and goodness of the selected feature using SIVS to those selected by Least Absolute Shrinkage and Selection Operator regression. The results suggested that the feature space selected by SIVS was, on average, 41% smaller, without having a negative effect on the model performance. A similar result was observed for comparison with Boruta and caret RFE. Availability and implementation The method is implemented as an R package under GNU General Public License v3.0 and is accessible via Comprehensive R Archive Network (CRAN) via https://cran.r-project.org/package=sivs. Supplementary information Supplementary data are available at Bioinformatics online.
Collapse
Affiliation(s)
- Mehrad Mahmoudian
- Turku Bioscience Centre, University of Turku and Åbo Akademi University, Turku, Finland.,Department of Future Technologies, University of Turku, Turku, Finland
| | - Mikko S Venäläinen
- Turku Bioscience Centre, University of Turku and Åbo Akademi University, Turku, Finland
| | - Riku Klén
- Turku Bioscience Centre, University of Turku and Åbo Akademi University, Turku, Finland
| | - Laura L Elo
- Turku Bioscience Centre, University of Turku and Åbo Akademi University, Turku, Finland.,Institute of Biomedicine, University of Turku, Turku, Finland
| |
Collapse
|