Identification of important factors in an inpatient fall risk prediction model to improve the quality of care using EHR and electronic administrative data: A machine-learning approach

Using machine learning models to predict falls in hospitalised adults

BACKGROUND

Identifying patients at high risk of falling is crucial in implementing effective fall prevention programs. While the integration of information systems is becoming more widespread in the healthcare industry, it poses a significant challenge in analysing vast amounts of data to identify factors that could enhance patient safety.

OBJECTIVE

To determine fall-associated factors and develop high-performance prediction tools for at-risk patients in acute and sub-acute care services in Australia.

METHODS

A retrospective study of 672,400 patients admitted to acute and sub-acute care services within a large metropolitan tertiary health service in Victoria, Australia, between January 1, 2019, and December 31, 2021. Data were obtained from four sources: the Department of Health Victorian Admitted Episodes Dataset, RiskManTM, electronic health records, and the health workforce dataset. Machine learning techniques, including Random Forest and Deep Neural Network models, were used to analyse the data, predict patient falls, and identify the most important risk factors for falls in this population. Model performance was evaluated using accuracy, F1-score, precision, recall, specificity, Matthew's correlation coefficient, and the area under the receiver operating characteristic curve (AUC).

RESULTS

The deep neural network and random forest models were highly accurate in predicting hospital patient falls. The deep neural network model achieved an accuracy of 0.988 and a specificity of 0.999, while the RF achieved an accuracy of 0.989 and a specificity of 1.000. The top 20 variables impacting falls were compared across both models, and 12 common factors were identified. These factors can be broadly classified into three categories: patient-related factors, staffing-related factors, and admission-related factors. Although not all factors are modifiable, they must be considered when planning fall prevention interventions.

CONCLUSION

The study demonstrated machine learning's potential to predict falls and identify key risk factors. Further validation across diverse populations and settings is essential for broader applicability.

Machine learning evaluation of inequities and disparities associated with nurse sensitive indicator safety events

PURPOSE

To use machine learning to examine health equity and clinical outcomes in patients who experienced a nurse sensitive indicator (NSI) event, defined as a fall, a hospital-acquired pressure injury (HAPI) or a hospital-acquired infection (HAI).

DESIGN

This was a retrospective observational study from a single academic hospital over six calendar years (2016-2021). Machine learning was used to examine patients with an NSI compared to those without.

METHODS

Inclusion criteria: all adult inpatient admissions (2016-2021). Three approaches were used to analyze the NSI group compared to the No-NSI group. In the univariate analysis, descriptive statistics, and absolute standardized differences (ASDs) were employed to compare the demographics and clinical variables of patients who experienced a NSI and those who did not experience any NSIs. For the multivariate analysis, a light grading boosting machine (LightGBM) model was utilized to comprehensively examine the relationships associated with the development of an NSI. Lastly, a simulation study was conducted to quantify the strength of associations obtained from the machine learning model.

RESULTS

From 163,507 admissions, 4643 (2.8%) were associated with at least one NSI. The mean, standard deviation (SD) age was 59.5 (18.2) years, males comprised 82,397 (50.4%). Non-Hispanic White 84,760 (51.8%), non-Hispanic Black 8703 (5.3%), non-Hispanic Asian 23,368 (14.3%), non-Hispanic Other 14,284 (8.7%), and Hispanic 30,271 (18.5%). Race and ethnicity alone were not associated with occurrence of an NSI. The NSI group had a statistically significant longer length of stay (LOS), longer intensive care unit (ICU) LOS, and was more likely to have an emergency admission compared to the group without an NSI. The simulation study results demonstrated that likelihood of NSI was higher in patients admitted under the major diagnostic categories (MDC) associated with circulatory, digestive, kidney/urinary tract, nervous, and infectious and parasitic disease diagnoses.

CONCLUSION

In this study, race/ethnicity was not associated with the risk of an NSI event. The risk of an NSI event was associated with emergency admission, longer LOS, longer ICU-LOS and certain MDCs (circulatory, digestive, kidney/urinary, nervous, infectious, and parasitic diagnoses).

CLINICAL RELEVANCE

Machine learning methodologies provide a new mechanism to investigate NSI events through the lens of health equity/disparity. Understanding which patients are at higher risk for adverse outcomes can help hospitals improve nursing care and prevent NSI injury and harm.

Improved accuracy and efficiency of primary care fall risk screening of older adults using a machine learning approach

BACKGROUND

While many falls are preventable, they remain a leading cause of injury and death in older adults. Primary care clinics largely rely on screening questionnaires to identify people at risk of falls. Limitations of standard fall risk screening questionnaires include suboptimal accuracy, missing data, and non-standard formats, which hinder early identification of risk and prevention of fall injury. We used machine learning methods to develop and evaluate electronic health record (EHR)-based tools to identify older adults at risk of fall-related injuries in a primary care population and compared this approach to standard fall screening questionnaires.

METHODS

Using patient-level clinical data from an integrated healthcare system consisting of 16-member institutions, we conducted a case-control study to develop and evaluate prediction models for fall-related injuries in older adults. Questionnaire-derived prediction with three questions from a commonly used fall risk screening tool was evaluated. We then developed four temporal machine learning models using routinely available longitudinal EHR data to predict the future risk of fall injury. We also developed a fall injury-prevention clinical decision support (CDS) implementation prototype to link preventative interventions to patient-specific fall injury risk factors.

RESULTS

Questionnaire-based risk screening achieved area under the receiver operating characteristic curve (AUC) up to 0.59 with 23% to 33% similarity for each pair of three fall injury screening questions. EHR-based machine learning risk screening showed significantly improved performance (best AUROC = 0.76), with similar prediction performance between 6-month and one-year prediction models.

CONCLUSIONS

The current method of questionnaire-based fall risk screening of older adults is suboptimal with redundant items, inadequate precision, and no linkage to prevention. A machine learning fall injury prediction method can accurately predict risk with superior sensitivity while freeing up clinical time for initiating personalized fall prevention interventions. The developed algorithm and data science pipeline can impact routine primary care fall prevention practice.

Findings from three methods to identify falls in hospitals: Results from the Ambient Intelligent Geriatric Management system fall prevention trial

OBJECTIVE

To (a) compare characteristics of patients who fall with those of patients who did not fall; and (b) characterise falls (time, injury severity and location) through three fall reporting methods (incident system reports, medical notes and clinician reports).

METHODS

A substudy design within a stepped-wedge clinical trial was used: 3239 trial participants were recruited from two inpatient Geriatric Evaluation and Management Units and one general medicine ward in two Australian states. To compare the characteristics of patients who had fallen with those who had not, descriptive tests were used. To characterise falls through three reporting methods, bivariate logistic regressions were used.

RESULTS

Patients who had fallen were more likely than patients who had not fallen to be cognitively impaired (51% vs. 29%, p < 0.01), admitted with falls (38% vs. 28%, p = 0.01) and have poor health outcomes such as prolonged length of stay (24 [16-34] vs. 12 [8-19] days [IQR], p < 0.01) and less likely to be discharged directly to the community (62% vs. 47%, p < 0.01). Most falls were captured from medical notes (93%), with clinician (71%) and incident reports (68%) missing 21%-25% of falls. The proportion of injurious falls identified through incident reports was higher than medical records or clinician reports (40% vs. 34% vs. 37%).

CONCLUSIONS

This study reaffirms the need to improve reporting falls in incident systems and at clinical handover to the team leader. Research should continue to use more than one method of identifying falls, but include data from medical records. Many falls cause injury, resulting in poor health outcomes.

Clinical Prediction Models for Hospital-Induced Delirium Using Structured and Unstructured Electronic Health Record Data: Protocol for a Development and Validation Study

BACKGROUND

Hospital-induced delirium is one of the most common and costly iatrogenic conditions, and its incidence is predicted to increase as the population of the United States ages. An academic and clinical interdisciplinary systems approach is needed to reduce the frequency and impact of hospital-induced delirium.

OBJECTIVE

The long-term goal of our research is to enhance the safety of hospitalized older adults by reducing iatrogenic conditions through an effective learning health system. In this study, we will develop models for predicting hospital-induced delirium. In order to accomplish this objective, we will create a computable phenotype for our outcome (hospital-induced delirium), design an expert-based traditional logistic regression model, leverage machine learning techniques to generate a model using structured data, and use machine learning and natural language processing to produce an integrated model with components from both structured data and text data.

METHODS

This study will explore text-based data, such as nursing notes, to improve the predictive capability of prognostic models for hospital-induced delirium. By using supervised and unsupervised text mining in addition to structured data, we will examine multiple types of information in electronic health record data to predict medical-surgical patient risk of developing delirium. Development and validation will be compliant to the Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD) statement.

RESULTS

Work on this project will take place through March 2024. For this study, we will use data from approximately 332,230 encounters that occurred between January 2012 to May 2021. Findings from this project will be disseminated at scientific conferences and in peer-reviewed journals.

CONCLUSIONS

Success in this study will yield a durable, high-performing research-data infrastructure that will process, extract, and analyze clinical text data in near real time. This model has the potential to be integrated into the electronic health record and provide point-of-care decision support to prevent harm and improve quality of care.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

DERR1-10.2196/48521.

In-hospital fall prediction using machine learning algorithms and the Morse fall scale in patients with acute stroke: a nested case-control study

BACKGROUND

Falls are one of the most common accidents in medical institutions, which can threaten the safety of inpatients and negatively affect their prognosis. Herein, we developed a machine learning (ML) model for fall prediction in patients with acute stroke and compared its accuracy with that of the existing fall risk prediction tool, the Morse Fall Scale (MFS).

METHODS

This is a retrospective nested case-control study. The initial sample size was 8462 admitted to a single cerebrovascular specialty hospital with acute stroke. A total of 156 fall events occurred, and each fall case was randomly matched with six control cases. Six ML algorithms were used, namely, regularized logistic regression, support vector machine, naïve Bayes (NB), k-nearest neighbors, random forest, and extreme-gradient boosting (XGB).

RESULTS

We included 156 in the fall group and 934 in the non-fall group. The mean ages of the fall and non-fall groups were 68.3 (± 12.2) and 65.3 (± 12.9) years old, respectively. The MFS total score was significantly higher in the fall group (54.3 ± 18.3) than in the non-fall group (37.7 ± 14.7). The area under the receiver operating curve (AUROC) of the MFS in predicting falls was 0.76 (0.73-0.79). XGB had the highest AUROC of 0.85 (0.78-0.92), and XGB and NB had the highest F1 score of 0.44.

CONCLUSIONS

The AUROC values of all of ML algorithms were similar to those of the MFS in predicting fall risk in patients with acute stroke, allowing for accurate and efficient fall screening.

Utilizing standardized nursing terminologies in implementing an AI-powered fall-prevention tool to improve patient outcomes: a multihospital study

OBJECTIVES

Standardized nursing terminologies (SNTs) are necessary to ensure consistent knowledge expression and compare the effectiveness of nursing practice across settings. This study investigated whether SNTs can support semantic interoperability and outcoming tracking over time by implementing an AI-powered CDS tool for fall prevention across multiple EMR systems.

MATERIALS AND METHODS

The study involved 3 tertiary academic hospitals and 1 public hospital with different EMR systems and nursing terms, and employed an AI-powered CDS tool that determines the fall risk within the next hour (prediction model) and recommends tailored care plans (CDS functions; represented by SNTs). The prediction model was mapped to local data elements and optimized using local data sets. The local nursing statements in CDS functions were mapped using an ICNP-based inpatient fall-prevention catalog. Four implementation models were compared, and patient outcomes and nursing activities were observed longitudinally at one site.

RESULTS

The postimplementation approach was practical for disseminating the AI-powered CDS tool for nursing. The 4 hospitals successfully implemented prediction models with little performance variation; the AUROCs were 0.8051-0.9581. The nursing process data contributed markedly to fall-risk predictions. The local nursing statements on preventing falls covered 48.0%-86.7% of statements. There was no significant longitudinal decrease in the fall rate (P = .160, 95% CI = -1.21 to 0.21 per 1000 hospital days), but rates of interventions provided by nurses were notably increased.

CONCLUSION

SNTs contributed to achieving semantic interoperability among multiple EMR systems to disseminate AI-powered CDS tools and automatically track nursing and patient outcomes.

Development and Internal Validation of a Prediction Model for Falls Using Electronic Health Records in a Hospital Setting

OBJECTIVE

Fall prevention is important in many hospitals. Current fall-risk-screening tools have limited predictive accuracy specifically for older inpatients. Their administration can be time-consuming. A reliable and easy-to-administer tool is desirable to identify older inpatients at higher fall risk. We aimed to develop and internally validate a prognostic prediction model for inpatient falls for older patients.

DESIGN

Retrospective analysis of a large cohort drawn from hospital electronic health record data.

SETTING AND PARTICIPANTS

Older patients (≥70 years) admitted to a university medical center (2016 until 2021).

METHODS

The outcome was an inpatient fall (≥24 hours of admission). Two prediction models were developed using regularized logistic regression in 5 imputed data sets: one model without predictors indicating missing values (Model-without) and one model with these additional predictors indicating missing values (Model-with). We internally validated our whole model development strategy using 10-fold stratified cross-validation. The models were evaluated using discrimination (area under the receiver operating characteristic curve) and calibration (plot assessment). We determined whether the areas under the receiver operating characteristic curves (AUCs) of the models were significantly different using DeLong test.

RESULTS

Our data set included 21,286 admissions. In total, 470 (2.2%) had a fall after 24 hours of admission. The Model-without had 12 predictors and Model-with 13, of which 4 were indicators of missing values. The AUCs of the Model-without and Model-with were 0.676 (95% CI 0.646-0.707) and 0.695 (95% CI 0.667-0.724). The AUCs between both models were significantly different (P = .013). Calibration was good for both models.

CONCLUSIONS AND IMPLICATIONS

Both the Model-with and Model-without indicators of missing values showed good calibration and fair discrimination, where the Model-with performed better. Our models showed competitive performance to well-established fall-risk-screening tools, and they have the advantage of being based on routinely collected data. This may substantially reduce the burden on nurses, compared with nonautomatic fall-risk-screening tools.

Machine learning based estimation of dynamic balance and gait adaptability in persons with neurological diseases using inertial sensors

Poor dynamic balance and impaired gait adaptation to different contexts are hallmarks of people with neurological disorders (PwND), leading to difficulties in daily life and increased fall risk. Frequent assessment of dynamic balance and gait adaptability is therefore essential for monitoring the evolution of these impairments and/or the long-term effects of rehabilitation. The modified dynamic gait index (mDGI) is a validated clinical test specifically devoted to evaluating gait facets in clinical settings under a physiotherapist's supervision. The need of a clinical environment, consequently, limits the number of assessments. Wearable sensors are increasingly used to measure balance and locomotion in real-world contexts and may permit an increase in monitoring frequency. This study aims to provide a preliminary test of this opportunity by using nested cross-validated machine learning regressors to predict the mDGI scores of 95 PwND via inertial signals collected from short steady-state walking bouts derived from the 6-minute walk test. Four different models were compared, one for each pathology (multiple sclerosis, Parkinson's disease, and stroke) and one for the pooled multipathological cohort. Model explanations were computed on the best-performing solution; the model trained on the multipathological cohort yielded a median (interquartile range) absolute test error of 3.58 (5.38) points. In total, 76% of the predictions were within the mDGI's minimal detectable change of 5 points. These results confirm that steady-state walking measurements provide information about dynamic balance and gait adaptability and can help clinicians identify important features to improve upon during rehabilitation. Future developments will include training of the method using short steady-state walking bouts in real-world settings, analysing the feasibility of this solution to intensify performance monitoring, providing prompt detection of worsening/improvements, and complementing clinical assessments.

External validation of the Johns Hopkins Fall Risk Assessment Tool in older Dutch hospitalized patients

PURPOSE

Fall prevention is a safety goal in many hospitals. The performance of the Johns Hopkins Fall Risk Assessment Tool (JHFRAT) in older inpatients is largely unknown. We aimed to assess the JHFRAT performance in a large sample of Dutch older inpatients, including its trend over time.

METHODS

We used an Electronic Health Records (EHR) dataset with hospitalized patients (≥ 70), admitted for ≥ 24 h between 2016 and 2021. Inpatient falls were extracted from structured and free-text data. We assessed the association between JHFRAT and falls using logistic regression. For test accuracy, we calculated sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV). Discrimination was measured by the AUC. For calibration, we plotted the predicted fall probability with the actual probability of falls. For time-related effects, we calculated the AUC per 6 months (using data of patients admitted during the 6 months' time interval) and plotted these different AUC values over time. Furthermore, we compared the model (JHFRAT and falls) with and without adjusting for seasonal influenza, COVID-19, spring, summer, fall or winter periods.

RESULTS

Data included 17,263 admissions with at least 1 JHFRAT measurement, a median age of 76 and a percentage female of 47%. The in-hospital fall prevalence was 2.5%. JHFRAT [OR = 1.11 (1.03-1.20)] and its subcategories were significantly associated with falls. For medium/high risk of falls (JHFRAT > 5), sensitivity was 73%, specificity 51%, PPV 4% and NPV 99%. The overall AUC was 0.67, varying over time between 0.62 and 0.71 (for 6 months' time intervals). Seasonal influenza did affect the association between JHFRAT and falls. COVID-19, spring, summer, fall or winter did not affect the association.

CONCLUSIONS

Our results show an association between JHFRAT and falls, a low discrimination by JHFRAT for older inpatients and over-prediction in the calibration. Improvements in the fall-risk assessment are warranted to improve efficiency.

Research Trends in Artificial Intelligence-Associated Nursing Activities Based on a Review of Academic Studies Published From 2001 to 2020

The present study referred to the technology-based learning model to conduct a systematic review of the dimensions of nursing activities, research samples, research methods, roles of artificial intelligence, applied artificial intelligence algorithms, evaluation measure of algorithms, and research foci. Based on the Preferred Reporting Items for Systematic Reviews and Meta-Analyses procedure, this study obtained and analyzed a total of 102 high-quality artificial intelligence-associated nursing activities studies published from 2001 to 2020 in the Web of Science database. The results showed: (1) In terms of nursing activities, nursing management was explored the most, followed by nursing assessment; (2) quantitative methods were most frequently adopted in artificial intelligence-associated nursing activities studies to investigate issues related to patients, followed by nursing staff; (3) the most adopted roles of artificial intelligence in artificial intelligence-associated nursing activities studies were profiling and prediction, followed by assessment and evaluation; (4) artificial intelligence-associated nursing activities studies frequently mixed applied artificial intelligence algorithms and evaluation measure of algorithms; (5) in the dimension of research foci, most studies mainly paid attention to the design or evaluation of the artificial intelligence systems/instruments, followed by investigating the correlation and affect issues. Based on the findings, several recommendations are raised as a reference for future researchers, educators, and policy makers.

The role of artificial intelligence in enhancing clinical nursing care: A scoping review

AIM

To present an overview of how artificial intelligence has been used to improve clinical nursing care.

BACKGROUND

Artificial intelligence has been reshaping the healthcare industry but little is known about its applicability in enhancing nursing care.

EVALUATION

A scoping review was conducted. Seven electronic databases (CINAHL, Cochrane Library, EMBASE, IEEE Xplore, PubMed, Scopus, and Web of Science) were searched from 1 January 2010 till 20 December 2020. Grey literature and reference lists of included articles were also searched.

KEY ISSUES

Thirty-seven studies encapsulating the use of artificial intelligence in improving clinical nursing care were included in this review. Six use cases were identified - documentation, formulating nursing diagnoses, formulating nursing care plans, patient monitoring, patient care prediction such as falls prediction (most common) and wound management. Various techniques of machine learning and classification were used for predictive analyses and to improve nurses' preparedness and management of patients' conditions CONCLUSION: This review highlighted the potential of artificial intelligence in improving the quality of nursing care. However, more randomized controlled trials in real-life healthcare settings should be conducted to enhance the rigor of evidence.

IMPLICATIONS FOR NURSING MANAGEMENT

Education in the application of artificial intelligence should be promoted to empower nurses to lead technological transformations and not passively trail behind others.

Identifying nursing sensitive indicators from electronic health records in acute cardiac care-Towards intelligent automated assessment of care quality

AIM

The aim of this study is to explore the potential of using electronic health records for assessment of nursing care quality through nursing-sensitive indicators in acute cardiac care.

BACKGROUND

Nursing care quality is a multifaceted phenomenon, making a holistic assessment of it difficult. Quality assessment systems in acute cardiac care units could benefit from big data-based solutions that automatically extract and help interpret data from electronic health records.

METHODS

This is a deductive descriptive study that followed the theory of value-added analysis. A random sample from electronic health records of 230 patients was analysed for selected indicators. The data included documentation in structured and free-text format.

RESULTS

One thousand six hundred seventy-six expressions were extracted and divided into (1) established and (2) unestablished expressions, providing positive, neutral and negative descriptions related to care quality.

CONCLUSIONS

Electronic health records provide a potential source of information for information systems to support assessment of care quality. More research is warranted to develop, test and evaluate the effectiveness of such tools in practice.

IMPLICATIONS FOR NURSING MANAGEMENT

Knowledge-based health care management would benefit from the development and implementation of advanced information systems, which use continuously generated already available real-time big data for improved data access and interpretation to better support nursing management in quality assessment.

Development of a predictive inpatient falls risk model using machine learning

AIM

The aims of this study were to create a model that detects the population at risk of falls taking into account a fall prevention variable and to know the effect on the model's performance when not considering it.

BACKGROUND

Traditionally, instruments for detecting fall risk are based on risk factors, not mitigating factors. Machine learning, which allows working with a wider range of variables, could improve patient risk identification.

METHODS

The sample was composed of adult patients admitted to the Internal Medicine service (total, n = 22,515; training, n = 11,134; validation, n = 11,381). A retrospective cohort design was used and we applied machine learning technics. Variables were extracted from electronic medical records electronic medical records.

RESULTS

The Two-Class Bayes Point Machine algorithm was selected. Model-A (with a fall prevention variable) obtained better results than Model-B (without it) in sensitivity (0.74 vs. 0.71), specificity (0.82 vs. 0.74), and AUC (0.82 vs. 0.78).

CONCLUSIONS

Fall prevention was a key variable. The model that included it detected the risk of falls better than the model without it.

IMPLICATIONS FOR NURSING MANAGEMENT

We created a decision-making support tool that helps nurses to identify patients at risk of falling. When it is integrated in the electronic medical records, it decreases nurses' workloads by not having to collect information manually.

Can the Electronic Health Record Predict Risk of Falls in Hospitalized Patients by Using Artificial Intelligence? A Meta-analysis

Because of an aging population worldwide, the increasing prevalence of falls and their consequent injuries are becoming a safety, health, and social-care issue among elderly people. We conducted a meta-analysis to investigate the benchmark of prediction power when using the EHR with artificial intelligence to predict risk of falls in hospitalized patients. The CHARMS guideline was used in this meta-analysis. We searched PubMed, Cochrane, and EMBASE. The pooled sensitivity and specificity were calculated, and the summary receiver operating curve was formed to investigate the predictive power of artificial intelligence models. The PROBAST table was used to assess the quality of the selected studies. A total of 132 846 patients were included in this meta-analysis. The pooled area under the curve of the collected research was estimated to be 0.78. The pooled sensitivity was 0.63 (95% confidence interval, 0.52-0.72), whereas the pooled specificity was 0.82 (95% confidence interval, 0.73-0.88). The quality of our selected studies was high, with most of them being evaluated with low risk of bias and low concern for applicability. Our study demonstrates that using the EHR with artificial intelligence to predict the risk of falls among hospitalized patients is feasible. Future clinical applications are anticipated.

Development and validation of interpretable machine learning models for inpatient fall events and electronic medical record integration

OBJECTIVE

Falls are one of the most frequently occurring adverse events among hospitalized patients. The Morse Fall Scale, which has been widely used for fall risk assessment, has the two limitations of low specificity and difficulty in practical implementation. The aim of this study was to develop and validate an interpretable machine learning model for prediction of falls to be integrated in an electronic medical record (EMR) system.

METHODS

This was a retrospective study involving a tertiary teaching hospital in Seoul, Korea. Based on the literature, 83 known predictors were grouped into seven categories. Interpretable fall event prediction models were developed using multiple machine learning models including gradient boosting and Shapley values.

RESULTS

Overall, 191,778 cases with 272 fall events (0.1%) were included in the analysis. With the validation cohort of 2020, the area under the receiver operating curve (AUROC) of the gradient boosting model was 0.817 (95% confidence interval [CI], 0.720-0.904), better performance than random forest (AUROC, 0.801; 95% CI, 0.708-0.890), logistic regression (AUROC, 0.802; 95% CI, 0.721-0.878), artificial neural net (AUROC, 0.736; 95% CI, 0.650-0.821), and conventional Morse fall score (AUROC, 0.652; 95% CI, 0.570-0.715). The model's interpretability was enhanced at both the population and patient levels. The algorithm was later integrated into the current EMR system.

CONCLUSION

We developed an interpretable machine learning prediction model for inpatient fall events using EMR integration formats.

Analyzing Hospital High Length of Stay Outliers in Turkey

Purpose: The aim of this study is to examine length of stay (LOS) outliers by analyzing hospital administrative database. Material and Methods: The Turkish Ministry of Health DRG grouper database was utilized to obtain hospital administrative data on discharges for 15 training and research hospitals in 2012. For each diagnosis-related group (DRG), the geometric mean plus two standard deviations were calculated to identify outliers. Analyses were conducted using descriptive statistics and logistic regression using generalized estimating equations (GEE). Results: High LOS outliers found to be 4.4 % of the cases, they were responsible for 24.50 percent of all discharge days. Alcohol, drug use disorders, burns, and diseases of the ear, nose, mouth, and throat were the factors that had the greatest impact on high LOS outliers, according to the multivariate model. Conclusion: A quarter of all inpatient days are made up of LOS outliers. Burns, neonate cases, and alcohol/drug use issues should all be carefully evaluated. In order to improve clinical quality and effectively manage hospital resources, hospital administrators and health policy makers should take length of stay outliers into consideration.

Standard Vocabularies to Improve Machine Learning Model Transferability With Electronic Health Record Data: Retrospective Cohort Study Using Health Care-Associated Infection

BACKGROUND

With the widespread adoption of electronic healthcare records (EHRs) by US hospitals, there is an opportunity to leverage this data for the development of predictive algorithms to improve clinical care. A key barrier in model development and implementation includes the external validation of model discrimination, which is rare and often results in worse performance. One reason why machine learning models are not externally generalizable is data heterogeneity. A potential solution to address the substantial data heterogeneity between health care systems is to use standard vocabularies to map EHR data elements. The advantage of these vocabularies is a hierarchical relationship between elements, which allows the aggregation of specific clinical features to more general grouped concepts.

OBJECTIVE

This study aimed to evaluate grouping EHR data using standard vocabularies to improve the transferability of machine learning models for the detection of postoperative health care-associated infections across institutions with different EHR systems.

METHODS

Patients who underwent surgery from the University of Utah Health and Intermountain Healthcare from July 2014 to August 2017 with complete follow-up data were included. The primary outcome was a health care-associated infection within 30 days of the procedure. EHR data from 0-30 days after the operation were mapped to standard vocabularies and grouped using the hierarchical relationships of the vocabularies. Model performance was measured using the area under the receiver operating characteristic curve (AUC) and F₁-score in internal and external validations. To evaluate model transferability, a difference-in-difference metric was defined as the difference in performance drop between internal and external validations for the baseline and grouped models.

RESULTS

A total of 5775 patients from the University of Utah and 15,434 patients from Intermountain Healthcare were included. The prevalence of selected outcomes was from 4.9% (761/15,434) to 5% (291/5775) for surgical site infections, from 0.8% (44/5775) to 1.1% (171/15,434) for pneumonia, from 2.6% (400/15,434) to 3% (175/5775) for sepsis, and from 0.8% (125/15,434) to 0.9% (50/5775) for urinary tract infections. In all outcomes, the grouping of data using standard vocabularies resulted in a reduced drop in AUC and F₁-score in external validation compared to baseline features (all P<.001, except urinary tract infection AUC: P=.002). The difference-in-difference metrics ranged from 0.005 to 0.248 for AUC and from 0.075 to 0.216 for F₁-score.

CONCLUSIONS

We demonstrated that grouping machine learning model features based on standard vocabularies improved model transferability between data sets across 2 institutions. Improving model transferability using standard vocabularies has the potential to improve the generalization of clinical prediction models across the health care system.

A model for predicting fall risks of hospitalized elderly in Taiwan-A machine learning approach based on both electronic health records and comprehensive geriatric assessment

Backgrounds

Falls are currently one of the important safety issues of elderly inpatients. Falls can lead to their injury, reduced mobility and comorbidity. In hospitals, it may cause medical disputes and staff guilty feelings and anxiety. We aimed to predict fall risks among hospitalized elderly patients using an approach of artificial intelligence.

Materials and methods

Our working hypothesis was that if hospitalized elderly patients have multiple risk factors, their incidence of falls is higher. Artificial intelligence was then used to predict the incidence of falls of these patients. We enrolled those elderly patients aged >65 years old and were admitted to the geriatric ward during 2018 and 2019, at a single medical center in central Taiwan. We collected 21 physiological and clinical data of these patients from their electronic health records (EHR) with their comprehensive geriatric assessment (CGA). Data included demographic information, vital signs, visual ability, hearing ability, previous medication, and activity of daily living. We separated data from a total of 1,101 patients into 3 datasets: (a) training dataset, (b) testing dataset and (c) validation dataset. To predict incidence of falls, we applied 6 models: (a) Deep neural network (DNN), (b) machine learning algorithm extreme Gradient Boosting (XGBoost), (c) Light Gradient Boosting Machine (LightGBM), (d) Random Forest, (e) Stochastic Gradient Descent (SGD) and (f) logistic regression.

Results

From modeling data of 1,101 elderly patients, we found that machine learning algorithm XGBoost, LightGBM, Random forest, SGD and logistic regression were successfully trained. Finally, machine learning algorithm XGBoost achieved 73.2% accuracy.

Conclusion

This is the first machine-learning based study using both EHR and CGA to predict fall risks of elderly. Multiple risk factors of falls in hospitalized elderly patients can be put into a machine learning model to predict future falls for early planned actions. Future studies should be focused on the model fitting and accuracy of data analysis.

Integration of Artificial Intelligence and Blockchain Technology in Healthcare and Agriculture

Over the last decade, the healthcare sector has accelerated its digitization and electronic health records (EHRs). As information technology progresses, the notion of intelligent health also gathers popularity. By combining technologies such as the internet of things (IoT) and artificial intelligence (AI), innovative healthcare modifies and enhances traditional medical systems in terms of efficiency, service, and personalization. On the other side, intelligent healthcare systems are incredibly vulnerable to data breaches and other malicious assaults. Recently, blockchain technology has emerged as a potentially transformative option for enhancing data management, access control, and integrity inside healthcare systems. Integrating these advanced approaches in agriculture is critical for managing food supply chains, drug supply chains, quality maintenance, and intelligent prediction. This study reviews the literature, formulates a research topic, and analyzes the applicability of blockchain to the agriculture/food industry and healthcare, with a particular emphasis on AI and IoT. This article summarizes research on the newest blockchain solutions paired with AI technologies for strengthening and inventing new technological standards for the healthcare ecosystems and food industry.

Fallacy of Median Door‐to‐ECG Time: Hidden Opportunities for STEMI Screening Improvement

Background

ST‐segment elevation myocardial infarction (STEMI) guidelines recommend screening arriving emergency department (ED) patients for an early ECG in those with symptoms concerning for myocardial ischemia. Process measures target median door‐to‐ECG (D2E) time of 10 minutes.

Methods and Results

This 3‐year descriptive retrospective cohort study, including 676 ED‐diagnosed patients with STEMI from 10 geographically diverse facilities across the United States, examines an alternative approach to quantifying performance: proportion of patients meeting the goal of D2E≤10 minutes. We also identified characteristics associated with D2E>10 minutes and estimated the proportion of patients with screening ECG occurring during intake, triage, and main ED care periods. We found overall median D2E was 7 minutes (IQR:4–16; range: 0–1407 minutes; range of ED medians: 5–11 minutes). Proportion of patients with D2E>10 minutes was 37.9% (ED range: 21.5%–57.1%). Patients with D2E>10 minutes, compared to those with D2E≤10 minutes, were more likely female (32.8% versus 22.6%, P=0.005), Black (23.4% versus 12.4%, P=0.005), non‐English speaking (24.6% versus 19.5%, P=0.032), diabetic (40.2% versus 30.2%, P=0.010), and less frequently reported chest pain (63.3% versus 87.4%, P<0.001). ECGs were performed during ED intake in 62.1% of visits, ED triage in 25.3%, and main ED care in 12.6%.

Conclusions

Examining D2E>10 minutes can identify opportunities to improve care for more ED patients with STEMI. Our findings suggest sex, race, language, and diabetes are associated with STEMI diagnostic delays. Moving the acquisition of ECGs completed during triage to intake could achieve the D2E≤10 minutes goal for 87.4% of ED patients with STEMI. Sophisticated screening, accounting for differential risk and diversity in STEMI presentations, may further improve timely detection.

Nonlaboratory-based risk assessment model for coronary heart disease screening: Model development and validation

BACKGROUND

Identifying groups at high risk of coronary heart disease (CHD) is important to reduce mortality due to CHD. Although machine learning methods have been introduced, many require laboratory or imaging parameters, which are not always readily available; thus, their wide applications are limited.

OBJECTIVE

The aim of this study was to develop and validate a simple, efficient, and joint machine learning model for identifying individuals at high risk of CHD using easily obtainable nonlaboratory parameters.

METHODS

This prospective study used data from the Henan Rural Cohort Study, which was conducted in rural areas of Henan Province, China, between July 2015 and September 2017. A joint machine learning model was developed by selecting and combining four base machine learning algorithms, including logistic regression (LR), artificial neural network (ANN), random forest (RF), and gradient boosting machine (GBM). We used readily accessible variables, including demographics, medical and family history, lifestyle and dietary factors, and anthropometric data, to inform the model. The model was also externally validated by a cohort of individuals from the Dongfeng-Tongji cohort study. Model discrimination was assessed by using the area under the receiver operating characteristic curve (AUC), and calibration was measured by using the Brier score (BS).

RESULTS

A total of 38 716 participants (mean [SD] age, 55.64[12.19] years; 23449[60.6%] female) from the Henan Rural Cohort Study and 17 958 subjects (mean [SD] age, 62.74 [7.59] years; 10,076 [56.1%] female) from the Dongfeng-Tongji cohort study were included in the analysis. Age, waist circumference, pulse pressure, heart rate, family history of CHD, education level, family history of type 2 diabetes mellitus (T2DM), and family history of dyslipidaemia were strongly associated with the development of CHD. In regard to internal validation, the model we built demonstrated good discrimination (AUC, 0.844 (95% CI 0.828-0.860)) and had acceptable calibration (BS, 0. 066). In regard to external validation, the model performed well with clearly useful discrimination (AUC, 0.792 (95% CI 0.774-0.810)) and robust calibration (BS, 0.069).

CONCLUSIONS

In this study, the novel and simple, machine learning-based model comprising readily accessible variables accurately identified individuals at high risk of CHD. This model has the potential to be widely applied for large-scale screening of CHD populations, especially in medical resource-constrained settings.

TRIAL REGISTRATION

The Henan Rural Cohort Study has been registered at the Chinese Clinical Trial Register. (Trial registration: ChiCTR-OOC-15006699. Registered 6 July 2015 - Retrospectively registered) http://www.chictr.org.cn/showproj.aspx?proj=11375.

Patient Falls, Nurse Communication, and Nurse Hourly Rounding in Acute Care: Linking Patient Experience and Outcomes

Research has consistently found a link between hourly nurse rounding and patient outcomes, including reduced falls, reduced pressure ulcers, reduced call light usage, and improved patient experience; however, little research exists specific to patient falls and nurse rounding in acute care settings. This study adds to the body of knowledge by statistically quantifying and providing linkages between nurse rounding frequency and patient fall rates using data from 31 military treatment facilities comprehensively over a period from fiscal year (FY) 2017 through FY2019. Poisson regression results indicated that hourly nurse rounding was associated with a reduction of more than 21% in fall rates (incidence rate ratio = 0.79, P < .01) relative to infrequent rounding, and poorly rated nurse communication was associated with an 8.6-fold increase in patient fall rates relative to highly rated nurse communication (incidence rate ratio = 8.6, P < .01).

Data Science Trends Relevant to Nursing Practice: A Rapid Review of the 2020 Literature

BACKGROUND

The term "data science" encompasses several methods, many of which are considered cutting edge and are being used to influence care processes across the world. Nursing is an applied science and a key discipline in health care systems in both clinical and administrative areas, making the profession increasingly influenced by the latest advances in data science. The greater informatics community should be aware of current trends regarding the intersection of nursing and data science, as developments in nursing practice have cross-professional implications.

OBJECTIVES

This study aimed to summarize the latest (calendar year 2020) research and applications of nursing-relevant patient outcomes and clinical processes in the data science literature.

METHODS

We conducted a rapid review of the literature to identify relevant research published during the year 2020. We explored the following 16 topics: (1) artificial intelligence/machine learning credibility and acceptance, (2) burnout, (3) complex care (outpatient), (4) emergency department visits, (5) falls, (6) health care-acquired infections, (7) health care utilization and costs, (8) hospitalization, (9) in-hospital mortality, (10) length of stay, (11) pain, (12) patient safety, (13) pressure injuries, (14) readmissions, (15) staffing, and (16) unit culture.

RESULTS

Of 16,589 articles, 244 were included in the review. All topics were represented by literature published in 2020, ranging from 1 article to 59 articles. Numerous contemporary data science methods were represented in the literature including the use of machine learning, neural networks, and natural language processing.

CONCLUSION

This review provides an overview of the data science trends that were relevant to nursing practice in 2020. Examinations of such literature are important to monitor the status of data science's influence in nursing practice.

Evaluation of Machine Learning Techniques to Predict the Likelihood of Mental Health Conditions Following a First mTBI

Objective

Limited research has evaluated the utility of machine learning models and longitudinal data from electronic health records (EHR) to forecast mental health outcomes following a traumatic brain injury (TBI). The objective of this study is to assess various data science and machine learning techniques and determine their efficacy in forecasting mental health (MH) conditions among active duty Service Members (SMs) following a first diagnosis of mild traumatic brain injury (mTBI).

Materials and Methods

Patient demographics and encounter metadata of 35,451 active duty SMs who have sustained an initial mTBI, as documented within the EHR, were obtained. All encounter records from a year prior and post index mTBI date were collected. Patient demographics, ICD-9-CM and ICD-10 codes, enhanced diagnostic related groups, and other risk factors estimated from the year prior to index mTBI were utilized to develop a feature vector representative of each patient. To embed temporal information into the feature vector, various window configurations were devised. Finally, the presence or absence of mental health conditions post mTBI index date were used as the outcomes variable for the models.

Results

When evaluating the machine learning models, neural network techniques showed the best overall performance in identifying patients with new or persistent mental health conditions post mTBI. Various window configurations were tested and results show that dividing the observation window into three distinct date windows [−365:−30, −30:0, 0:14] provided the best performance. Overall, the models described in this paper identified the likelihood of developing MH conditions at [14:90] days post-mTBI with an accuracy of 88.2%, an AUC of 0.82, and AUC-PR of 0.66.

Discussion

Through the development and evaluation of different machine learning models we have validated the feasibility of designing algorithms to forecast the likelihood of developing mental health conditions after the first mTBI. Patient attributes including demographics, symptomatology, and other known risk factors proved to be effective features to employ when training ML models for mTBI patients. When patient attributes and features are estimated at different time window, the overall performance increase illustrating the importance of embedding temporal information into the models. The addition of temporal information not only improved model performance, but also increased interpretability and clinical utility.

Conclusion

Predictive analytics can be a valuable tool for understanding the effects of mTBI, particularly when identifying those individuals at risk of negative outcomes. The translation of these models from retrospective study into real-world validation models is imperative in the mitigation of negative outcomes with appropriate and timely interventions.

A new approach to identifying patients with elevated risk for Fabry disease using a machine learning algorithm

Background

Fabry disease (FD) is a rare genetic disorder characterized by glycosphingolipid accumulation and progressive damage across multiple organ systems. Due to its heterogeneous presentation, the condition is likely significantly underdiagnosed. Several approaches, including provider education efforts and newborn screening, have attempted to address underdiagnosis of FD across the age spectrum, with limited success. Artificial intelligence (AI) methods present another option for improving diagnosis. These methods isolate common health history patterns among patients using longitudinal real-world data, and can be particularly useful when patients experience nonspecific, heterogeneous symptoms over time. In this study, the performance of an AI tool in identifying patients with FD was analyzed. The tool was calibrated using de-identified health record data from a large cohort of nearly 5000 FD patients, and extracted phenotypic patterns from these records. The tool then used this FD pattern information to make individual-level estimates of FD in a testing dataset. Patterns were reviewed and confirmed with medical experts.

Results

The AI tool demonstrated strong analytic performance in identifying FD patients. In out-of-sample testing, it achieved an area under the receiver operating characteristic curve (AUROC) of 0.82. Strong performance was maintained when testing on male-only and female-only cohorts, with AUROCs of 0.83 and 0.82 respectively. The tool identified small segments of the population with greatly increased prevalence of FD: in the 1% of the population identified by the tool as at highest risk, FD was 23.9 times more prevalent than in the population overall. The AI algorithm used hundreds of phenotypic signals to make predictions and included both familiar symptoms associated with FD (e.g. renal manifestations) as well as less well-studied characteristics.

Conclusions

The AI tool analyzed in this study performed very well in identifying Fabry disease patients using structured medical history data. Performance was maintained in all-male and all-female cohorts, and the phenotypic manifestations of FD highlighted by the tool were reviewed and confirmed by clinical experts in the condition. The platform’s analytic performance, transparency, and ability to generate predictions based on existing real-world health data may allow it to contribute to reducing persistent underdiagnosis of Fabry disease.

Usability of Electronic Health records in Predicting Short-term falls: Machine learning Applications in Senior Care Facilities (Preprint)

Background

Short-term fall prediction models that use electronic health records (EHRs) may enable the implementation of dynamic care practices that specifically address changes in individualized fall risk within senior care facilities.

Objective

The aim of this study is to implement machine learning (ML) algorithms that use EHR data to predict a 3-month fall risk in residents from a variety of senior care facilities providing different levels of care.

Methods

This retrospective study obtained EHR data (2007-2021) from Juniper Communities’ proprietary database of 2785 individuals primarily residing in skilled nursing facilities, independent living facilities, and assisted living facilities across the United States. We assessed the performance of 3 ML-based fall prediction models and the Juniper Communities’ fall risk assessment. Additional analyses were conducted to examine how changes in the input features, training data sets, and prediction windows affected the performance of these models.

Results

The Extreme Gradient Boosting model exhibited the highest performance, with an area under the receiver operating characteristic curve of 0.846 (95% CI 0.794-0.894), specificity of 0.848, diagnostic odds ratio of 13.40, and sensitivity of 0.706, while achieving the best trade-off in balancing true positive and negative rates. The number of active medications was the most significant feature associated with fall risk, followed by a resident’s number of active diseases and several variables associated with vital signs, including diastolic blood pressure and changes in weight and respiratory rates. The combination of vital signs with traditional risk factors as input features achieved higher prediction accuracy than using either group of features alone.

Conclusions

This study shows that the Extreme Gradient Boosting technique can use a large number of features from EHR data to make short-term fall predictions with a better performance than that of conventional fall risk assessments and other ML models. The integration of routinely collected EHR data, particularly vital signs, into fall prediction models may generate more accurate fall risk surveillance than models without vital signs. Our data support the use of ML models for dynamic, cost-effective, and automated fall predictions in different types of senior care facilities.

Clinical Impact of an Analytic Tool for Predicting the Fall Risk in Inpatients: Controlled Interrupted Time Series

BACKGROUND

Patient falls are a common cause of harm in acute-care hospitals worldwide. They are a difficult, complex, and common problem requiring a great deal of nurses' time, attention, and effort in practice. The recent rapid expansion of health care predictive analytic applications and the growing availability of electronic health record (EHR) data have resulted in the development of machine learning models that predict adverse events. However, the clinical impact of these models in terms of patient outcomes and clinicians' responses is undetermined.

OBJECTIVE

The purpose of this study was to determine the impact of an electronic analytic tool for predicting fall risk on patient outcomes and nurses' responses.

METHODS

A controlled interrupted time series (ITS) experiment was conducted in 12 medical-surgical nursing units at a public hospital between May 2017 and April 2019. In six of the units, the patients' fall risk was assessed using the St. Thomas' Risk Assessment Tool in Falling Elderly Inpatients (STRATIFY) system (control units), while in the other six, a predictive model for inpatient fall risks was implemented using routinely obtained data from the hospital's EHR system (intervention units). The primary outcome was the rate of patient falls; secondary outcomes included the rate of falls with injury and analysis of process metrics (nursing interventions that are designed to mitigate the risk of fall).

RESULTS

During the study period, there were 42,476 admissions, of which 707 were for falls and 134 for fall injuries. Allowing for differences in the patients' characteristics and baseline process metrics, the number of patients with falls differed between the control (n=382) and intervention (n=325) units. The mean fall rate increased from 1.95 to 2.11 in control units and decreased from 1.92 to 1.79 in intervention units. A separate ITS analysis revealed that the immediate reduction was 29.73% in the intervention group (z=-2.06, P=.039) and 16.58% in the control group (z=-1.28, P=.20), but there was no ongoing effect. The injury rate did not differ significantly between the two groups (0.42 vs 0.31, z=1.50, P=.134). Among the process metrics, the risk-targeted interventions increased significantly over time in the intervention group.

CONCLUSIONS

This early-stage clinical evaluation revealed that implementation of an analytic tool for predicting fall risk may to contribute to an awareness of fall risk, leading to positive changes in nurses' interventions over time.

TRIAL REGISTRATION

Clinical Research Information Service (CRIS), Republic of Korea KCT0005286; https://cris.nih.go.kr/cris/search/detailSearch.do/16984.

Reducing Fall-related Revisits for Elderly Diabetes Patients in Emergency Departments: A Transition Flow Model

This paper introduces a transition flow model to study fall-related emergency department (ED) revisits for elderly patients with diabetes. Five diabetes classes are used to classify patients at discharge, within 7-day revisits, and between 8 and 30-day revisits. Analytical formulas to evaluate patient revisiting risks are derived. To reduce revisits, sensitivity analysis is introduced to identify the most critical, i.e., dominant, factors whose changes can lead to the largest reduction in revisits. In addition, a case study at University of Wisconsin (UW) Hospital ED is described to illustrate the applicability of the model.