Explainable machine learning to identify risk factors for unplanned hospital readmissions in Nova Scotian hospitals

OBJECTIVE

A report from the Canadian Institute for Health Information found unplanned hospital readmissions (UHR) common, costly, and potentially avoidable, estimating a $1.8 billion cost to the Canadian healthcare system associated with inpatient readmissions within 30 days of discharge for the studied period (11 months). The first step towards addressing this costly problem is enabling early detection of patients at risk through detecting UHR risk factors.

METHODOLOGY

We utilized Machine Learning and explainability tools to examine risk factors for UHR within 30 days of discharge, utilizing data from Nova Scotian (Canada) healthcare institutions (2015-2022). To the best of our knowledge, our research constitutes the most comprehensive study on UHR risk factors for the province.

RESULTS

We found that predicting UHR solely from healthcare data has limitations, as discharge information often falls short of accurately predicting readmission occurrences. However, despite this inherent limitation, integrating explainability tools offers insights into the underlying factors contributing to readmission risk, empowering medical personnel with information to improve patient care and outcomes. As part of this work, we identify and report risk factors for UHR and build a guideline to support medical personnel's decision-making regarding targeted post-discharge follow-ups. We found that conditions such as heart failure and Chronic Obstructive Pulmonary Disease (COPD) are associated with a higher likelihood of readmission. Patients admitted for procedures related to childbirth have a lower probability of readmission. We studied the impact of the admission type, patient characteristics, and patient stay characteristics on UHR. For example, we found that new and elective admission patients are less likely to be readmitted, while patients who received a transfusion are more likely to be readmitted.

CONCLUSIONS

We validated the risk factors and the guidelines using real-world data. Our results suggested that our proposal correctly identifies risk factors and effectively produces valuable guidelines for medical personnel. The guideline evaluation suggests we can screen half the patients while capturing more than 72% of the readmission episodes. Our study contributes insights into the challenge of identifying risk factors for UHR while providing a practical guideline for healthcare professionals to identify factors influencing patient readmission, particularly within Nova Scotia.

Predicting the risk of hospital readmissions using a machine learning approach: a case study on patients undergoing skin procedures

Introduction

Even with modern advancements in medical care, one of the persistent challenges hospitals face is the frequent readmission of patients. These recurrent admissions not only escalate healthcare expenses but also amplify mental and emotional strain on patients.

Methods

This research delved into two primary areas: unraveling the pivotal factors causing the readmissions, specifically targeting patients who underwent dermatological treatments, and determining the optimal machine learning algorithms that can foresee potential readmissions with higher accuracy.

Results

Among the multitude of algorithms tested, including logistic regression (LR), support vector machine (SVM), random forest (RF), Naïve Bayesian (NB), artificial neural network (ANN), xgboost (XG), and k-nearest neighbor (KNN), it was noted that two models-XG and RF-stood out in their prediction prowess. A closer inspection of the data brought to light certain patterns. For instance, male patients and those between the ages of 21 and 40 had a propensity to be readmitted more frequently. Moreover, the months of March and April witnessed a spike in these readmissions, with ~6% of the patients returning within just a month after their first admission.

Discussion

Upon further analysis, specific determinants such as the patient's age and the specific hospital where they were treated emerged as key indicators influencing the likelihood of their readmission.

Real-World Data for Interdisciplinary Health Care Research: A Case Example

Real-word data (RWD) refer to data relating to patient health status and/or the delivery of health care routinely collected from a variety of sources, including electronic health records, medical claims data, and patient-generated data. Data sets that combine personal health data stored in different sources can provide a more complete picture of an individual's health and can be used to improve population health through research and practice. The 2-tiered aim of this article is to provide a brief introduction to using RWD in health care research and to present a case study that demonstrates data curation and data merge from different sources while highlighting the benefits and limitations of using RWD. The current digital health ecosystem and value-based care approach highlight the need to use RWD to catalyze the advancement of health care research and practice. This is an excellent field that nurse researchers can lead, as they have an innate understanding of such data and data sources.

A Novel AUC Maximization Imbalanced Learning Approach for Predicting Composite Outcomes in COVID-19 Hospitalized Patients

The COVID-19 patient data for composite outcome prediction often comes with class imbalance issues, i.e., only a small group of patients develop severe composite events after hospital admission, while the rest do not. An ideal COVID-19 composite outcome prediction model should possess strong imbalanced learning capability. The model also should have fewer tuning hyperparameters to ensure good usability and exhibit potential for fast incremental learning. Towards this goal, this study proposes a novel imbalanced learning approach called Imbalanced maximizing-Area Under the Curve (AUC) Proximal Support Vector Machine (ImAUC-PSVM) by the means of classical PSVM to predict the composite outcomes of hospitalized COVID-19 patients within 30 days of hospitalization. ImAUC-PSVM offers the following merits: (1) it incorporates straightforward AUC maximization into the objective function, resulting in fewer parameters to tune. This makes it suitable for handling imbalanced COVID-19 data with a simplified training process. (2) Theoretical derivations reveal that ImAUC-PSVM has the same analytical solution form as PSVM, thus inheriting the advantages of PSVM for handling incremental COVID-19 cases through fast incremental updating. We built and internally and externally validated our proposed classifier using real COVID-19 patient data obtained from three separate sites of Mayo Clinic in the United States. Additionally, we validated it on public datasets using various performance metrics. Experimental results demonstrate that ImAUC-PSVM outperforms other methods in most cases, showcasing its potential to assist clinicians in triaging COVID-19 patients at an early stage in hospital settings, as well as in other prediction applications.

Improving Resource Utilization for Arthroplasty Care by Leveraging Machine Learning and Optimization: A Systematic Review

Background

There is a growing demand for total joint arthroplasty (TJA) surgery. The applications of machine learning (ML), mathematical optimization, and computer simulation have the potential to improve efficiency of TJA care delivery through outcome prediction and surgical scheduling optimization, easing the burden on health-care systems. The purpose of this study was to evaluate strategies using advances in analytics and computational modeling that may improve planning and the overall efficiency of TJA care.

Methods

A systematic review including MEDLINE, Embase, and IEEE Xplore databases was completed from inception to October 3, 2022, for identification of studies generating ML models for TJA length of stay, duration of surgery, and hospital readmission prediction. A scoping review of optimization strategies in elective surgical scheduling was also conducted.

Results

Twenty studies were included for evaluating ML predictions and 17 in the scoping review of scheduling optimization. Among studies generating linear or logistic control models alongside ML models, only 1 found a control model to outperform its ML counterpart. Furthermore, neural networks performed superior to or at the same level as conventional ML models in all but 1 study. Implementation of mathematical and simulation strategies improved the optimization efficiency when compared to traditional scheduling methods at the operational level.

Conclusions

High-performing predictive ML-based models have been developed for TJA, as have mathematical strategies for elective surgical scheduling optimization. By leveraging artificial intelligence for outcome prediction and surgical optimization, there exist greater opportunities for improved resource utilization and cost-savings in TJA than when using traditional modeling and scheduling methods.

Concerns surrounding application of artificial intelligence in hip and knee arthroplasty

Literature surrounding artificial intelligence (AI)-related applications for hip and knee arthroplasty has proliferated. However, meaningful advances that fundamentally transform the practice and delivery of joint arthroplasty are yet to be realized, despite the broad range of applications as we continue to search for meaningful and appropriate use of AI. AI literature in hip and knee arthroplasty between 2018 and 2021 regarding image-based analyses, value-based care, remote patient monitoring, and augmented reality was reviewed. Concerns surrounding meaningful use and appropriate methodological approaches of AI in joint arthroplasty research are summarized. Of the 233 AI-related orthopaedics articles published, 178 (76%) constituted original research, while the rest consisted of editorials or reviews. A total of 52% of original AI-related research concerns hip and knee arthroplasty (n = 92), and a narrative review is described. Three studies were externally validated. Pitfalls surrounding present-day research include conflating vernacular (“AI/machine learning”), repackaging limited registry data, prematurely releasing internally validated prediction models, appraising model architecture instead of inputted data, withholding code, and evaluating studies using antiquated regression-based guidelines. While AI has been applied to a variety of hip and knee arthroplasty applications with limited clinical impact, the future remains promising if the question is meaningful, the methodology is rigorous and transparent, the data are rich, and the model is externally validated. Simple checkpoints for meaningful AI adoption include ensuring applications focus on: administrative support over clinical evaluation and management; necessity of the advanced model; and the novelty of the question being answered.

Cite this article: Bone Joint J 2022;104-B(12):1292–1303.

Evaluation of the Clinical, Technical, and Financial Aspects of Cost-Effectiveness Analysis of Artificial Intelligence in Medicine: Scoping Review and Framework of Analysis

BACKGROUND

Cost-effectiveness analysis of artificial intelligence (AI) in medicine demands consideration of clinical, technical, and economic aspects to generate impactful research of a novel and highly versatile technology.

OBJECTIVE

We aimed to systematically scope existing literature on the cost-effectiveness of AI and to extract and summarize clinical, technical, and economic dimensions required for a comprehensive assessment.

METHODS

A scoping literature review was conducted to map medical, technical, and economic aspects considered in studies on the cost-effectiveness of medical AI. Based on these, a framework for health policy analysis was developed.

RESULTS

Among 4820 eligible studies, 13 met the inclusion criteria for our review. Internal medicine and emergency medicine were the clinical disciplines most frequently analyzed. Most of the studies included were from the United States (5/13, 39%), assessed solutions requiring market access (9/13, 69%), and proposed optimization of direct resources as the most frequent value proposition (7/13, 53%). On the other hand, technical aspects were not uniformly disclosed in the studies we analyzed. A minority of articles explicitly stated the payment mechanism assumed (5/13, 38%), while it remained unspecified in the majority (8/13, 62%) of studies.

CONCLUSIONS

Current studies on the cost-effectiveness of AI do not allow to determine if the investigated AI solutions are clinically, technically, and economically viable. Further research and improved reporting on these dimensions seem relevant to recommend and assess potential use cases for this technology.

Artificial intelligence in diagnosis of knee osteoarthritis and prediction of arthroplasty outcomes: a review

Background

Artificial intelligence is an emerging technology with rapid growth and increasing applications in orthopaedics. This study aimed to summarize the existing evidence and recent developments of artificial intelligence in diagnosing knee osteoarthritis and predicting outcomes of total knee arthroplasty.

Methods

PubMed and EMBASE databases were searched for articles published in peer-reviewed journals between January 1, 2010 and May 31, 2021. The terms included: ‘artificial intelligence’, ‘machine learning’, ‘knee’, ‘osteoarthritis’, and ‘arthroplasty’. We selected studies focusing on the use of AI in diagnosis of knee osteoarthritis, prediction of the need for total knee arthroplasty, and prediction of outcomes of total knee arthroplasty. Non-English language articles and articles with no English translation were excluded. A reviewer screened the articles for the relevance to the research questions and strength of evidence.

Results

Machine learning models demonstrated promising results for automatic grading of knee radiographs and predicting the need for total knee arthroplasty. The artificial intelligence algorithms could predict postoperative outcomes regarding patient-reported outcome measures, patient satisfaction and short-term complications. Important weaknesses of current artificial intelligence algorithms included the lack of external validation, the limitations of inherent biases in clinical data, the requirement of large datasets in training, and significant research gaps in the literature.

Conclusions

Artificial intelligence offers a promising solution to improve detection and management of knee osteoarthritis. Further research to overcome the weaknesses of machine learning models may enhance reliability and allow for future use in routine healthcare settings.

Systematic Review of Health Economic Evaluations Focused on Artificial Intelligence in Healthcare: The Tortoise and the Cheetah

OBJECTIVES

This study aimed to systematically review recent health economic evaluations (HEEs) of artificial intelligence (AI) applications in healthcare. The aim was to discuss pertinent methods, reporting quality and challenges for future implementation of AI in healthcare, and additionally advise future HEEs.

METHODS

A systematic literature review was conducted in 2 databases (PubMed and Scopus) for articles published in the last 5 years. Two reviewers performed independent screening, full-text inclusion, data extraction, and appraisal. The Consolidated Health Economic Evaluation Reporting Standards and Philips checklist were used for the quality assessment of included studies.

RESULTS

A total of 884 unique studies were identified; 20 were included for full-text review, covering a wide range of medical specialties and care pathway phases. The most commonly evaluated type of AI was automated medical image analysis models (n = 9, 45%). The prevailing health economic analysis was cost minimization (n = 8, 40%) with the costs saved per case as preferred outcome measure. A total of 9 studies (45%) reported model-based HEEs, 4 of which applied a time horizon >1 year. The evidence supporting the chosen analytical methods, assessment of uncertainty, and model structures was underreported. The reporting quality of the articles was moderate as on average studies reported on 66% of Consolidated Health Economic Evaluation Reporting Standards items.

CONCLUSIONS

HEEs of AI in healthcare are limited and often focus on costs rather than health impact. Surprisingly, model-based long-term evaluations are just as uncommon as model-based short-term evaluations. Consequently, insight into the actual benefits offered by AI is lagging behind current technological developments.

Artificial intelligence in orthopaedics: A scoping review

There is a growing interest in the application of artificial intelligence (AI) to orthopaedic surgery. This review aims to identify and characterise research in this field, in order to understand the extent, range and nature of this work, and act as springboard to stimulate future studies. A scoping review, a form of structured evidence synthesis, was conducted to summarise the use of AI in orthopaedics. A literature search (1946-2019) identified 222 studies eligible for inclusion. These studies were predominantly small and retrospective. There has been significant growth in the number of papers published in the last three years, mainly from the USA (37%). The majority of research used AI for image interpretation (45%) or as a clinical decision tool (25%). Spine (43%), knee (23%) and hip (14%) were the regions of the body most commonly studied. The application of artificial intelligence to orthopaedics is growing. However, the scope of its use so far remains limited, both in terms of its possible clinical applications, and the sub-specialty areas of the body which have been studied. A standardized method of reporting AI studies would allow direct assessment and comparison. Prospective studies are required to validate AI tools for clinical use.

Artificial Learning and Machine Learning Decision Guidance Applications in Total Hip and Knee Arthroplasty: A Systematic Review

BACKGROUND

Artificial intelligence (AI) and machine learning (ML) modeling in hip and knee arthroplasty (total joint arthroplasty [TJA]) is becoming more commonplace. This systematic review aims to quantify the accuracy of current AI- and ML-based application for cognitive support and decision-making in TJA.

METHODS

A comprehensive search of publications was conducted through the EMBASE, Medline, and PubMed databases using relevant keywords to maximize the sensitivity of the search. No limits were placed on level of evidence or timing of the study. Findings were reported according to the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines. Analysis of variance testing with post-hoc Tukey test was applied to compare the area under the curve (AUC) of the models.

RESULTS

After application of inclusion and exclusion criteria, 49 studies were included in this review. The application of AI/ML-based models and average AUC is as follows: cost prediction-0.77, LOS and discharges-0.78, readmissions and reoperations-0.66, preoperative patient selection/planning-0.79, adverse events and other postoperative complications-0.84, postoperative pain-0.83, postoperative patient-reported outcomes measures and functional outcome-0.81. Significant variability in model AUC across the different decision support applications was found (P < .001) with the AUC for readmission and reoperation models being significantly lower than that of the other decision support categories.

CONCLUSIONS

AI/ML-based applications in TJA continue to expand and have the potential to optimize patient selection and accurately predict postoperative outcomes, complications, and associated costs. On average, the AI/ML models performed best in predicting postoperative complications, pain, and patient-reported outcomes and were less accurate in predicting hospital readmissions and reoperations.

Current Trends in Readmission Prediction: An Overview of Approaches

Hospital readmission shortly after discharge threatens the quality of patient care and leads to increased medical care costs. In the United States, hospitals with high readmission rates are subject to federal financial penalties. This concern calls for incentives for healthcare facilities to reduce their readmission rates by predicting patients who are at high risk of readmission. Conventional practices involve the use of rule-based assessment scores and traditional statistical methods, such as logistic regression, in developing risk prediction models. The recent advancements in machine learning driven by improved computing power and sophisticated algorithms have the potential to produce highly accurate predictions. However, the value of such models could be overrated. Meanwhile, the use of other flexible models that leverage simple algorithms offer great transparency in terms of feature interpretation, which is beneficial in clinical settings. This work presents an overview of the current trends in risk prediction models developed in the field of readmission. The various techniques adopted by researchers in recent years are described, and the topic of whether complex models outperform simple ones in readmission risk stratification is investigated.

Economic Evaluations of Artificial Intelligence in Ophthalmology

ABSTRACT

Artificial intelligence (AI) is expected to cause significant medical quality enhancements and cost-saving improvements in ophthalmology. Although there has been a rapid growth of studies on AI in the recent years, real-world adoption of AI is still rare. One reason may be because the data derived from economic evaluations of AI in health care, which policy makers used for adopting new technology, have been fragmented and scarce. Most data on economics of AI in ophthalmology are from diabetic retinopathy (DR) screening. Few studies classified costs of AI software, which has been considered as a medical device, into direct medical costs. These costs of AI are composed of initial and maintenance costs. The initial costs may include investment in research and development, and costs for validation of different datasets. Meanwhile, the maintenance costs include costs for algorithms upgrade and hardware maintenance in the long run. The cost of AI should be balanced between manufacturing price and reimbursements since it may pose significant challenges and barriers to providers. Evidence from cost-effectiveness analyses showed that AI, either standalone or used with humans, was more cost-effective than manual DR screening. Notably, economic evaluation of AI for DR screening can be used as a model for AI to other ophthalmic diseases.

The SMART Framework: Selection of Machine learning Algorithms with ReplicaTions - a Case Study on the Microvascular Complications of Diabetes

Over 34 million people in the US have diabetes, a major cause of blindness, renal failure, and amputations. Machine learning (ML) models can predict high-risk patients to help prevent adverse outcomes. Selecting the 'best' prediction model for a given disease, population, and clinical application is challenging due to the hundreds of health-related ML models in the literature and the increasing availability of ML methodologies. To support this decision process, we developed the Selection of Machine-learning Algorithms with ReplicaTions (SMART) Framework that integrates building and selecting ML models with decision theory. We build ML models and estimate performance for multiple plausible future populations with a replicated nested cross-validation technique. We rank ML models by simulating decision-maker priorities, using a range of accuracy measures (e.g., AUC) and robustness metrics from decision theory (e.g., minimax Regret). We present the SMART Framework through a case study on the microvascular complications of diabetes using data from the ACCORD clinical trial. We compare selections made by risk-averse, -neutral, and -seeking decision-makers, finding agreement in 80% of the risk-averse and risk-neutral selections, with the risk-averse selections showing consistency for a given complication. We also found that the models that best predicted outcomes in the validation set were those with low performance variance on the testing set, indicating a risk-averse approach in model selection is ideal when there is a potential for high population feature variability. The SMART Framework is a powerful, interactive tool that incorporates various ML algorithms and stakeholder preferences, generalizable to new data and technological advancements.

Economic evaluations of big data analytics for clinical decision-making: a scoping review

OBJECTIVE

Much has been invested in big data analytics to improve health and reduce costs. However, it is unknown whether these investments have achieved the desired goals. We performed a scoping review to determine the health and economic impact of big data analytics for clinical decision-making.

MATERIALS AND METHODS

We searched Medline, Embase, Web of Science and the National Health Services Economic Evaluations Database for relevant articles. We included peer-reviewed papers that report the health economic impact of analytics that assist clinical decision-making. We extracted the economic methods and estimated impact and also assessed the quality of the methods used. In addition, we estimated how many studies assessed "big data analytics" based on a broad definition of this term.

RESULTS

The search yielded 12 133 papers but only 71 studies fulfilled all eligibility criteria. Only a few papers were full economic evaluations; many were performed during development. Papers frequently reported savings for healthcare payers but only 20% also included costs of analytics. Twenty studies examined "big data analytics" and only 7 reported both cost-savings and better outcomes.

DISCUSSION

The promised potential of big data is not yet reflected in the literature, partly since only a few full and properly performed economic evaluations have been published. This and the lack of a clear definition of "big data" limit policy makers and healthcare professionals from determining which big data initiatives are worth implementing.

The Economic Impact of Artificial Intelligence in Health Care: Systematic Review

Background

Positive economic impact is a key decision factor in making the case for or against investing in an artificial intelligence (AI) solution in the health care industry. It is most relevant for the care provider and insurer as well as for the pharmaceutical and medical technology sectors. Although the broad economic impact of digital health solutions in general has been assessed many times in literature and the benefit for patients and society has also been analyzed, the specific economic impact of AI in health care has been addressed only sporadically.

Objective

This study aimed to systematically review and summarize the cost-effectiveness studies dedicated to AI in health care and to assess whether they meet the established quality criteria.

Methods

In a first step, the quality criteria for economic impact studies were defined based on the established and adapted criteria schemes for cost impact assessments. In a second step, a systematic literature review based on qualitative and quantitative inclusion and exclusion criteria was conducted to identify relevant publications for an in-depth analysis of the economic impact assessment. In a final step, the quality of the identified economic impact studies was evaluated based on the defined quality criteria for cost-effectiveness studies.

Results

Very few publications have thoroughly addressed the economic impact assessment, and the economic assessment quality of the reviewed publications on AI shows severe methodological deficits. Only 6 out of 66 publications could be included in the second step of the analysis based on the inclusion criteria. Out of these 6 studies, none comprised a methodologically complete cost impact analysis. There are two areas for improvement in future studies. First, the initial investment and operational costs for the AI infrastructure and service need to be included. Second, alternatives to achieve similar impact must be evaluated to provide a comprehensive comparison.

Conclusions

This systematic literature analysis proved that the existing impact assessments show methodological deficits and that upcoming evaluations require more comprehensive economic analyses to enable economic decisions for or against implementing AI technology in health care.