Comparison of logistic regression and neural network analysis applied to predicting living setting after hip fracture

Systematic review of machine-learning models in orthopaedic trauma

Aims

Machine-learning (ML) prediction models in orthopaedic trauma hold great promise in assisting clinicians in various tasks, such as personalized risk stratification. However, an overview of current applications and critical appraisal to peer-reviewed guidelines is lacking. The objectives of this study are to 1) provide an overview of current ML prediction models in orthopaedic trauma; 2) evaluate the completeness of reporting following the Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD) statement; and 3) assess the risk of bias following the Prediction model Risk Of Bias Assessment Tool (PROBAST) tool.

Methods

A systematic search screening 3,252 studies identified 45 ML-based prediction models in orthopaedic trauma up to January 2023. The TRIPOD statement assessed transparent reporting and the PROBAST tool the risk of bias.

Results

A total of 40 studies reported on training and internal validation; four studies performed both development and external validation, and one study performed only external validation. The most commonly reported outcomes were mortality (33%, 15/45) and length of hospital stay (9%, 4/45), and the majority of prediction models were developed in the hip fracture population (60%, 27/45). The overall median completeness for the TRIPOD statement was 62% (interquartile range 30 to 81%). The overall risk of bias in the PROBAST tool was low in 24% (11/45), high in 69% (31/45), and unclear in 7% (3/45) of the studies. High risk of bias was mainly due to analysis domain concerns including small datasets with low number of outcomes, complete-case analysis in case of missing data, and no reporting of performance measures.

Conclusion

The results of this study showed that despite a myriad of potential clinically useful applications, a substantial part of ML studies in orthopaedic trauma lack transparent reporting, and are at high risk of bias. These problems must be resolved by following established guidelines to instil confidence in ML models among patients and clinicians. Otherwise, there will remain a sizeable gap between the development of ML prediction models and their clinical application in our day-to-day orthopaedic trauma practice.

Artificial Intelligence for Hip Fracture Detection and Outcome Prediction: A Systematic Review and Meta-analysis

IMPORTANCE

Artificial intelligence (AI) enables powerful models for establishment of clinical diagnostic and prognostic tools for hip fractures; however the performance and potential impact of these newly developed algorithms are currently unknown.

OBJECTIVE

To evaluate the performance of AI algorithms designed to diagnose hip fractures on radiographs and predict postoperative clinical outcomes following hip fracture surgery relative to current practices.

DATA SOURCES

A systematic review of the literature was performed using the MEDLINE, Embase, and Cochrane Library databases for all articles published from database inception to January 23, 2023. A manual reference search of included articles was also undertaken to identify any additional relevant articles.

STUDY SELECTION

Studies developing machine learning (ML) models for the diagnosis of hip fractures from hip or pelvic radiographs or to predict any postoperative patient outcome following hip fracture surgery were included.

DATA EXTRACTION AND SYNTHESIS

This study followed the Preferred Reporting Items for Systematic Reviews and Meta-analyses and was registered with PROSPERO. Eligible full-text articles were evaluated and relevant data extracted independently using a template data extraction form. For studies that predicted postoperative outcomes, the performance of traditional predictive statistical models, either multivariable logistic or linear regression, was recorded and compared with the performance of the best ML model on the same out-of-sample data set.

MAIN OUTCOMES AND MEASURES

Diagnostic accuracy of AI models was compared with the diagnostic accuracy of expert clinicians using odds ratios (ORs) with 95% CIs. Areas under the curve for postoperative outcome prediction between traditional statistical models (multivariable linear or logistic regression) and ML models were compared.

RESULTS

Of 39 studies that met all criteria and were included in this analysis, 18 (46.2%) used AI models to diagnose hip fractures on plain radiographs and 21 (53.8%) used AI models to predict patient outcomes following hip fracture surgery. A total of 39 598 plain radiographs and 714 939 hip fractures were used for training, validating, and testing ML models specific to diagnosis and postoperative outcome prediction, respectively. Mortality and length of hospital stay were the most predicted outcomes. On pooled data analysis, compared with clinicians, the OR for diagnostic error of ML models was 0.79 (95% CI, 0.48-1.31; P = .36; I2 = 60%) for hip fracture radiographs. For the ML models, the mean (SD) sensitivity was 89.3% (8.5%), specificity was 87.5% (9.9%), and F1 score was 0.90 (0.06). The mean area under the curve for mortality prediction was 0.84 with ML models compared with 0.79 for alternative controls (P = .09).

CONCLUSIONS AND RELEVANCE

The findings of this systematic review and meta-analysis suggest that the potential applications of AI to aid with diagnosis from hip radiographs are promising. The performance of AI in diagnosing hip fractures was comparable with that of expert radiologists and surgeons. However, current implementations of AI for outcome prediction do not seem to provide substantial benefit over traditional multivariable predictive statistics.

Effect of Previous Crops and Soil Physicochemical Properties on the Population of Verticillium dahliae in the Iberian Peninsula

The soil infestation of Verticillium dahliae has significant Verticillium wilt of olive (VWO) with epidemiological consequences which could limit the expansion of the crop. In this context, there is a misunderstood history of the crops and soil property interactions associated with inoculum density (ID) increases in the soil. In this study, the effect of the combination of both factors was assessed on the ID of V. dahliae in the olive-growing areas of the Iberian Peninsula. Afterwards, the relationship of the ID to the mentioned factors was explored. The detection percentage and ID were higher in Spain than Portugal, even though the fields with a very favourable VWO history had a higher ID than that of the fields with a barely favourable history, regardless of the origin. The soil physicochemical parameters were able to detect the degree to which the ID was increased by the previous cropping history. By using a decision tree classifier, the percentage of clay was the best indicator for the V. dahliae ID regardless of the history of the crops. However, active limestone and the cation exchange capacity were only suitable ID indicators when <2 or 4 host crops of the pathogen were established in the field for five years, respectively. The V. dahliae ID was accurately predicted in this study for the orchard choices in the establishment of the olive.

Development and internal validation of a machine-learning-developed model for predicting 1-year mortality after fragility hip fracture

Background

Fragility hip fracture increases morbidity and mortality in older adult patients, especially within the first year. Identification of patients at high risk of death facilitates modification of associated perioperative factors that can reduce mortality. Various machine learning algorithms have been developed and are widely used in healthcare research, particularly for mortality prediction. This study aimed to develop and internally validate 7 machine learning models to predict 1-year mortality after fragility hip fracture.

Methods

This retrospective study included patients with fragility hip fractures from a single center (Siriraj Hospital, Bangkok, Thailand) from July 2016 to October 2018. A total of 492 patients were enrolled. They were randomly categorized into a training group (344 cases, 70%) or a testing group (148 cases, 30%). Various machine learning techniques were used: the Gradient Boosting Classifier (GB), Random Forests Classifier (RF), Artificial Neural Network Classifier (ANN), Logistic Regression Classifier (LR), Naive Bayes Classifier (NB), Support Vector Machine Classifier (SVM), and K-Nearest Neighbors Classifier (KNN). All models were internally validated by evaluating their performance and the area under a receiver operating characteristic curve (AUC).

Results

For the testing dataset, the accuracies were GB model = 0.93, RF model = 0.95, ANN model = 0.94, LR model = 0.91, NB model = 0.89, SVM model = 0.90, and KNN model = 0.90. All models achieved high AUCs that ranged between 0.81 and 0.99. The RF model also provided a negative predictive value of 0.96, a positive predictive value of 0.93, a specificity of 0.99, and a sensitivity of 0.68.

Conclusions

Our machine learning approach facilitated the successful development of an accurate model to predict 1-year mortality after fragility hip fracture. Several machine learning algorithms (eg, Gradient Boosting and Random Forest) had the potential to provide high predictive performance based on the clinical parameters of each patient. The web application is available at www.hipprediction.com. External validation in a larger group of patients or in different hospital settings is warranted to evaluate the clinical utility of this tool.

Trial registration

Thai Clinical Trials Registry (22 February 2021; reg. no. TCTR20210222003).

Supplementary Information

The online version contains supplementary material available at 10.1186/s12877-022-03152-x.

Development of a machine learning algorithm for predicting in-hospital and 1-year mortality after traumatic spinal cord injury

BACKGROUND CONTEXT

Current prognostic tools such as the Injury Severity Score (ISS) that predict mortality following trauma do not adequately consider the unique characteristics of traumatic spinal cord injury (tSCI).

PURPOSE

Our aim was to develop and validate a prognostic tool that can predict mortality following tSCI.

STUDY DESIGN

Retrospective review of a prospective cohort study.

PATIENT SAMPLE

Data was collected from 1245 persons with acute tSCI who were enrolled in the Rick Hansen Spinal Cord Injury Registry between 2004 and 2016.

OUTCOME MEASURES

In-hospital and 1-year mortality following tSCI.

METHODS

Machine learning techniques were used on patient-level data (n=849) to develop the Spinal Cord Injury Risk Score (SCIRS) that can predict mortality based on age, neurological level and completeness of injury, AOSpine classification of spinal column injury morphology, and Abbreviated Injury Scale scores. Validation of the SCIRS was performed by testing its accuracy in an independent validation cohort (n=396) and comparing its performance to the ISS, a measure which is used to predict mortality following general trauma.

RESULTS

For 1-year mortality prediction, the values for the Area Under the Receiver Operating Characteristic Curve (AUC) for the development cohort were 0.84 (standard deviation=0.029) for the SCIRS and 0.55 (0.041) for the ISS. For the validation cohort, AUC values were 0.86 (0.051) for the SCIRS and 0.71 (0.074) for the ISS. For in-hospital mortality, AUC values for the development cohort were 0.87 (0.028) and 0.60 (0.050) for the SCIRS and ISS, respectively. For the validation cohort, AUC values were 0.85 (0.054) for the SCIRS and 0.70 (0.079) for the ISS.

CONCLUSIONS

The SCIRS can predict in-hospital and 1-year mortality following tSCI more accurately than the ISS. The SCIRS can be used in research to reduce bias in estimating parameters and can help adjust for coefficients during model development. Further validation using larger sample sizes and independent datasets is needed to assess its reliability and to evaluate using it as an assessment tool to guide clinical decision-making and discussions with patients and families.

Design Comorbidity Portfolios to Improve Treatment Cost Prediction of Asthma Using Machine Learning

Comorbidity is an important factor to consider when trying to predict the cost of treating asthma patients. When an asthmatic patient suffered from comorbidity, the cost of treating such a patient becomes dependent on the nature of the comorbidity. Therefore, lack of recognition of comorbidity on asthmatic patient poses a challenge in predicting the cost of treatment. In this study, we proposed a comorbidity portfolio design that improves the prediction cost of treating asthmatic patients by regrouping frequently occurred comorbidities in different cost groups. In the experiment, predictive models, including logistic regression, random forest, support vector machine, classification regression tree, and backpropagation neural network were trained with real-world data of asthmatic patients from 2012 to 2014 in a large city of China. The 10-fold cross validation and random search algorithm were employed to optimize the hyper-parameters. We recorded significant improvements using our model, which are attributed to comorbidity portfolios in area under curve (AUC) and sensitivity increase of 46.89% (standard deviation: 4.45%) and 101.07% (standard deviation: 44.94%), respectively. In risk analysis of comorbidity on cost, respiratory diseases with a cumulative proportion in the adjusted odds ratio of 36.38% (95%CI: 27.61%, 47.86%) and circulatory diseases with a cumulative proportion in the adjusted odds ratio of 23.83% (95%CI: 15.95%, 35.22%) are the dominant risks of asthmatic patients that affects the treatment cost. It is found that the comorbidity portfolio is robust, and provides a better prediction of the high-cost of treating asthmatic patients. The preliminary characterization of the joint risk of multiple comorbidities posed on cost are also reported. This study will be of great help in improving cost prediction and comorbidity management.

Comparison of artificial neural network and logistic regression models for prediction of outcomes in trauma patients: A systematic review and meta-analysis

BACKGROUND

Currently, two models of artificial neural network (ANN) and logistic regression (LR) are known as models that extensively used in medical sciences. The aim of this study was to compare the ANN and LR models in prediction of Health-related outcomes in traumatic patients using a systematic review.

METHODS

The study was planned and conducted based on the Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) checklist. A literature search of published studies was conducted using PubMed, Embase, Web of knowledge, Scopus, and Google Scholar in May 2018. Joanna Briggs Institute (JBI) checklists was used for assessing the quality of the included articles.

RESULTS

The literature searches yielded 326 potentially relevant studies from the primary searches. Overall, the review included 10 unique studies. The results of this study showed that the area under curve (AUC) for the ANN was 0.91, (95% CI 0.89-0.83) and 0.89, (95% CI 0.87-90) for the LR in random effect model. The accuracy rate for ANN and LR in random effect models were 90.5, (95% CI, 87.6-94.2) and 83.2, (95% CI 75.1-91.2), respectively.

CONCLUSION

The results of our study showed that ANN has better performance than LR in predicting the terminal outcomes of traumatic patients in both the AUC and accuracy rate. Using an ANN to predict the final implications of trauma patients can provide more accurate clinical decisions.

eDoctor: machine learning and the future of medicine

Machine learning (ML) is a burgeoning field of medicine with huge resources being applied to fuse computer science and statistics to medical problems. Proponents of ML extol its ability to deal with large, complex and disparate data, often found within medicine and feel that ML is the future for biomedical research, personalized medicine, computer-aided diagnosis to significantly advance global health care. However, the concepts of ML are unfamiliar to many medical professionals and there is untapped potential in the use of ML as a research tool. In this article, we provide an overview of the theory behind ML, explore the common ML algorithms used in medicine including their pitfalls and discuss the potential future of ML in medicine.

Determinants of functional outcome in hip fracture: the role of comorbidity

BACKGROUND AND AIMS

Executed studies did not clearly identify which index of comorbidity was an independent outcome determinant. The aim of this prospective observational cohort study was to address this issue.

METHODS

We analyzed 200 consecutive patients with hip fracture. All patients underwent rehabilitation. At admission comorbidity was assessed through the cumulative severity, severity index, and comorbidity index of the Cumulative Illness Rating Scale. Discharge scores and effectiveness in the Functional Independence Measure motor subscale, and discharge destination were the outcome measures. Multivariate regression analyses were performed to identify determinants of outcome.

RESULTS

Mini Mental State Examination and comorbidity index of the Cumulative Illness Rating Scale were important independent determinants of final (respectively, β = 0.46 and -0.25) and effectiveness (respectively, β = 0.47 and -0.25) in motor Functional Independence Measure scores, while hip strength and Rankin score were determinants of final motor Functional Independence Measure score (respectively, β = 0.21 and -0.20). Comorbidity index of the Cumulative Illness Rating Scale (odds ratio 8.18 for ≥3 versus < 3 comorbidity score; 95% confidence interval, 1.03-64.7) and Geriatric Depression Scale (odds ratio 4.02 for ≥6 versus ≤5 depression scale score; 95% confidence interval, 1.52-10.63) were risk indicators for nursing home.

CONCLUSIONS

Among the indices of the Cumulative Illness Rating Scale, comorbidity index is the sole independent determinant of both motor Functional Independence Measure scores and discharge destination in hip fracture patients. This suggests to specifically evaluate this index to identify the patients who may be admitted to a rehabilitation program.

Predicting Pre-planting Risk of Stagonospora nodorum blotch in Winter Wheat Using Machine Learning Models

Pre-planting factors have been associated with the late-season severity of Stagonospora nodorum blotch (SNB), caused by the fungal pathogen Parastagonospora nodorum, in winter wheat (Triticum aestivum). The relative importance of these factors in the risk of SNB has not been determined and this knowledge can facilitate disease management decisions prior to planting of the wheat crop. In this study, we examined the performance of multiple regression (MR) and three machine learning algorithms namely artificial neural networks, categorical and regression trees, and random forests (RF), in predicting the pre-planting risk of SNB in wheat. Pre-planting factors tested as potential predictor variables were cultivar resistance, latitude, longitude, previous crop, seeding rate, seed treatment, tillage type, and wheat residue. Disease severity assessed at the end of the growing season was used as the response variable. The models were developed using 431 disease cases (unique combinations of predictors) collected from 2012 to 2014 and these cases were randomly divided into training, validation, and test datasets. Models were evaluated based on the regression of observed against predicted severity values of SNB, sensitivity-specificity ROC analysis, and the Kappa statistic. A strong relationship was observed between late-season severity of SNB and specific pre-planting factors in which latitude, longitude, wheat residue, and cultivar resistance were the most important predictors. The MR model explained 33% of variability in the data, while machine learning models explained 47 to 79% of the total variability. Similarly, the MR model correctly classified 74% of the disease cases, while machine learning models correctly classified 81 to 83% of these cases. Results show that the RF algorithm, which explained 79% of the variability within the data, was the most accurate in predicting the risk of SNB, with an accuracy rate of 93%. The RF algorithm could allow early assessment of the risk of SNB, facilitating sound disease management decisions prior to planting of wheat.

Development of a web-based liver cancer prediction model for type II diabetes patients by using an artificial neural network

BACKGROUND

Diabetes mellitus is associated with an increased risk of liver cancer, and these two diseases are among the most common and important causes of morbidity and mortality in Taiwan.

PURPOSE

To use data mining techniques to develop a model for predicting the development of liver cancer within 6 years of diagnosis with type II diabetes.

METHODS

Data were obtained from the National Health Insurance Research Database (NHIRD) of Taiwan, which covers approximately 22 million people. In this study, we selected patients who were newly diagnosed with type II diabetes during the 2000-2003 periods, with no prior cancer diagnosis. We then used encrypted personal ID to perform data linkage with the cancer registry database to identify whether these patients were diagnosed with liver cancer. Finally, we identified 2060 cases and assigned them to a case group (patients diagnosed with liver cancer after diabetes) and a control group (patients with diabetes but no liver cancer). The risk factors were identified from the literature review and physicians' suggestion, then, chi-square test was conducted on each independent variable (or potential risk factor) for a comparison between patients with liver cancer and those without, those found to be significant were selected as the factors. We subsequently performed data training and testing to construct artificial neural network (ANN) and logistic regression (LR) prediction models. The dataset was randomly divided into 2 groups: a training group and a test group. The training group consisted of 1442 cases (70% of the entire dataset), and the prediction model was developed on the basis of the training group. The remaining 30% (618 cases) were assigned to the test group for model validation.

RESULTS

The following 10 variables were used to develop the ANN and LR models: sex, age, alcoholic cirrhosis, nonalcoholic cirrhosis, alcoholic hepatitis, viral hepatitis, other types of chronic hepatitis, alcoholic fatty liver disease, other types of fatty liver disease, and hyperlipidemia. The performance of the ANN was superior to that of LR, according to the sensitivity (0.757), specificity (0.755), and the area under the receiver operating characteristic curve (0.873). After developing the optimal prediction model, we base on this model to construct a web-based application system for liver cancer prediction, which can provide support to physicians during consults with diabetes patients.

CONCLUSION

In the original dataset (n=2060), 33% of diabetes patients were diagnosed with liver cancer (n=515). After using 70% of the original data to training the model and other 30% for testing, the sensitivity and specificity of our model were 0.757 and 0.755, respectively; this means that 75.7% of diabetes patients can be predicted correctly to receive a future liver cancer diagnosis, and 75.5% can be predicted correctly to not be diagnosed with liver cancer. These results reveal that this model can be used as effective predictors of liver cancer for diabetes patients, after discussion with physicians; they also agreed that model can assist physicians to advise potential liver cancer patients and also helpful to decrease the future cost incurred upon cancer treatment.

Predictive modeling in pediatric traumatic brain injury using machine learning

BACKGROUND

Pediatric traumatic brain injury (TBI) constitutes a significant burden and diagnostic challenge in the emergency department (ED). While large North American research networks have derived clinical prediction rules for the head injured child, these may not be generalizable to practices in countries with traditionally low rates of computed tomography (CT). We aim to study predictors for moderate to severe TBI in our ED population aged < 16 years.

METHODS

This was a retrospective case-control study based on data from a prospective surveillance head injury database. Cases were included if patients presented from 2006 to 2014, with moderate to severe TBI. Controls were age-matched head injured children from the registry, obtained in a 4 control: 1 case ratio. These children remained well on diagnosis and follow up. Demographics, history, and physical examination findings were analyzed and patients followed up for the clinical course and outcome measures of death and neurosurgical intervention. To predict moderate to severe TBI, we built a machine learning (ML) model and a multivariable logistic regression model and compared their performances by means of Receiver Operating Characteristic (ROC) analysis.

RESULTS

There were 39 cases and 156 age-matched controls. The following 4 predictors remained statistically significant after multivariable analysis: Involvement in road traffic accident, a history of loss of consciousness, vomiting and signs of base of skull fracture. The logistic regression model was created with these 4 variables while the ML model was built with 3 extra variables, namely the presence of seizure, confusion and clinical signs of skull fracture. At the optimal cutoff scores, the ML method improved upon the logistic regression method with respect to the area under the ROC curve (0.98 vs 0.93), sensitivity (94.9% vs 82.1%), specificity (97.4% vs 92.3%), PPV (90.2% vs 72.7%), and NPV (98.7% vs 95.4%).

CONCLUSIONS

In this study, we demonstrated the feasibility of using machine learning as a tool to predict moderate to severe TBI. If validated on a large scale, the ML method has the potential not only to guide discretionary use of CT, but also a more careful selection of head injured children who warrant closer monitoring in the hospital.

Artificial neural network models for predicting 1-year mortality in elderly patients with intertrochanteric fractures in China

The mortality rate of older patients with intertrochanteric fractures has been increasing with the aging of populations in China. The purpose of this study was: 1) to develop an artificial neural network (ANN) using clinical information to predict the 1-year mortality of elderly patients with intertrochanteric fractures, and 2) to compare the ANN's predictive ability with that of logistic regression models. The ANN model was tested against actual outcomes of an intertrochanteric femoral fracture database in China. The ANN model was generated with eight clinical inputs and a single output. ANN's performance was compared with a logistic regression model created with the same inputs in terms of accuracy, sensitivity, specificity, and discriminability. The study population was composed of 2150 patients (679 males and 1471 females): 1432 in the training group and 718 new patients in the testing group. The ANN model that had eight neurons in the hidden layer had the highest accuracies among the four ANN models: 92.46 and 85.79% in both training and testing datasets, respectively. The areas under the receiver operating characteristic curves of the automatically selected ANN model for both datasets were 0.901 (95%CI=0.814-0.988) and 0.869 (95%CI=0.748-0.990), higher than the 0.745 (95%CI=0.612-0.879) and 0.728 (95%CI=0.595-0.862) of the logistic regression model. The ANN model can be used for predicting 1-year mortality in elderly patients with intertrochanteric fractures. It outperformed a logistic regression on multiple performance measures when given the same variables.

Comparison of prediction model for cardiovascular autonomic dysfunction using artificial neural network and logistic regression analysis

Background

This study aimed to develop the artificial neural network (ANN) and multivariable logistic regression (LR) analyses for prediction modeling of cardiovascular autonomic (CA) dysfunction in the general population, and compare the prediction models using the two approaches.

Methods and Materials

We analyzed a previous dataset based on a Chinese population sample consisting of 2,092 individuals aged 30–80 years. The prediction models were derived from an exploratory set using ANN and LR analysis, and were tested in the validation set. Performances of these prediction models were then compared.

Results

Univariate analysis indicated that 14 risk factors showed statistically significant association with the prevalence of CA dysfunction (P<0.05). The mean area under the receiver-operating curve was 0.758 (95% CI 0.724–0.793) for LR and 0.762 (95% CI 0.732–0.793) for ANN analysis, but noninferiority result was found (P<0.001). The similar results were found in comparisons of sensitivity, specificity, and predictive values in the prediction models between the LR and ANN analyses.

Conclusion

The prediction models for CA dysfunction were developed using ANN and LR. ANN and LR are two effective tools for developing prediction models based on our dataset.

Artificial neural network models for prediction of cardiovascular autonomic dysfunction in general Chinese population

BACKGROUND

The present study aimed to develop an artificial neural network (ANN) based prediction model for cardiovascular autonomic (CA) dysfunction in the general population.

METHODS

We analyzed a previous dataset based on a population sample consisted of 2,092 individuals aged 30-80 years. The prediction models were derived from an exploratory set using ANN analysis. Performances of these prediction models were evaluated in the validation set.

RESULTS

Univariate analysis indicated that 14 risk factors showed statistically significant association with CA dysfunction (P < 0.05). The mean area under the receiver-operating curve was 0.762 (95% CI 0.732-0.793) for prediction model developed using ANN analysis. The mean sensitivity, specificity, positive and negative predictive values were similar in the prediction models was 0.751, 0.665, 0.330 and 0.924, respectively. All HL statistics were less than 15.0.

CONCLUSION

ANN is an effective tool for developing prediction models with high value for predicting CA dysfunction among the general population.

Evaluation of two treatment outcome prediction models for restoration of visual fields in patients with postchiasmatic visual pathway lesions

Visual functions of patients with visual field defects after acquired brain injury affecting the primary visual pathway can be improved by means of vision restoration training. Since the extent of the restored visual field varies between patients, the prediction of treatment outcome and its visualization may help patients to decide for or against participating in therapies aimed at vision restoration. For this purpose, two treatment outcome prediction models were established based on either self-organizing maps (SOMs) or categorical regression (CR) to predict visual field change after intervention by several features that were hypothesized to be associated with vision restoration. Prediction was calculated for visual field changes recorded with High Resolution Perimetry (HRP). Both models revealed a similar predictive quality with the CR model being slightly more beneficial. Predictive quality of the SOM model improved when using only a small number of features that exhibited a higher association with treatment outcome than the remaining features, i.e. neighborhood activity and homogeneity within the surrounding 5° visual field of a given position, together with its residual function and distance to the scotoma border. Although both models serve their purpose, these were not able to outperform a primitive prediction rule that attests the importance of areas of residual vision, i.e. regions with partial visual field function, for vision restoration.

Recognition of morphometric vertebral fractures by artificial neural networks: analysis from GISMO Lombardia Database

BACKGROUND

It is known that bone mineral density (BMD) predicts the fracture's risk only partially and the severity and number of vertebral fractures are predictive of subsequent osteoporotic fractures (OF). Spinal deformity index (SDI) integrates the severity and number of morphometric vertebral fractures. Nowadays, there is interest in developing algorithms that use traditional statistics for predicting OF. Some studies suggest their poor sensitivity. Artificial Neural Networks (ANNs) could represent an alternative. So far, no study investigated ANNs ability in predicting OF and SDI. The aim of the present study is to compare ANNs and Logistic Regression (LR) in recognising, on the basis of osteoporotic risk-factors and other clinical information, patients with SDI≥1 and SDI≥5 from those with SDI = 0.

METHODOLOGY

We compared ANNs prognostic performance with that of LR in identifying SDI≥1/SDI≥5 in 372 women with postmenopausal-osteoporosis (SDI≥1, n = 176; SDI = 0, n = 196; SDI≥5, n = 51), using 45 variables (44 clinical parameters plus BMD). ANNs were allowed to choose relevant input data automatically (TWIST-system-Semeion). Among 45 variables, 17 and 25 were selected by TWIST-system-Semeion, in SDI≥1 vs SDI = 0 (first) and SDI≥5 vs SDI = 0 (second) analysis. In the first analysis sensitivity of LR and ANNs was 35.8% and 72.5%, specificity 76.5% and 78.5% and accuracy 56.2% and 75.5%, respectively. In the second analysis, sensitivity of LR and ANNs was 37.3% and 74.8%, specificity 90.3% and 87.8%, and accuracy 63.8% and 81.3%, respectively.

CONCLUSIONS

ANNs showed a better performance in identifying both SDI≥1 and SDI≥5, with a higher sensitivity, suggesting its promising role in the development of algorithm for predicting OF.

Comparison of artificial neural network and logistic regression models for predicting mortality in elderly patients with hip fracture

PURPOSE

Older patients with hip fracture have a mortality rate one year after surgery of 20-30%. The purpose of this study is to establish a predictive model to assess the outcome of surgical treatment in older patients with hip fracture.

METHODS

A database of information from 286 consecutive cases of surgery for hip fracture from the Department of Orthopedics, National Taiwan University Hospital Yun-Lin Branch, was utilised for model building and testing. Both logistic regression and artificial neural network (ANN) models were developed. Cases were randomly assigned to training and testing datasets. A testing dataset was utilised to test the accuracy of both models (n=89).

RESULTS

The areas under the receiver operator characteristic curves of both models were utilised to compare predictability and accuracy. The logistic regression training and testing datasets had an area of 0.938 (95% CI: 0.904, 0.972) and 0.784 (95% CI: 0.669, 0.899), respectively, below the 0.998 (95% CI: 0.995, 1.000) and 0.949 (95% CI: 0.857, 1.000) of the final ANN model.

CONCLUSION

Overall, ANNs have higher predictive ability than logistic regression, perhaps because they are not affected by interactions between factors. They may assist in complex decision making in the clinical setting.

Using machine learning algorithms to guide rehabilitation planning for home care clients

Background

Targeting older clients for rehabilitation is a clinical challenge and a research priority. We investigate the potential of machine learning algorithms – Support Vector Machine (SVM) and K-Nearest Neighbors (KNN) – to guide rehabilitation planning for home care clients.

Methods

This study is a secondary analysis of data on 24,724 longer-term clients from eight home care programs in Ontario. Data were collected with the RAI-HC assessment system, in which the Activities of Daily Living Clinical Assessment Protocol (ADLCAP) is used to identify clients with rehabilitation potential. For study purposes, a client is defined as having rehabilitation potential if there was: i) improvement in ADL functioning, or ii) discharge home. SVM and KNN results are compared with those obtained using the ADLCAP. For comparison, the machine learning algorithms use the same functional and health status indicators as the ADLCAP.

Results

The KNN and SVM algorithms achieved similar substantially improved performance over the ADLCAP, although false positive and false negative rates were still fairly high (FP > .18, FN > .34 versus FP > .29, FN. > .58 for ADLCAP). Results are used to suggest potential revisions to the ADLCAP.

Conclusion

Machine learning algorithms achieved superior predictions than the current protocol. Machine learning results are less readily interpretable, but can also be used to guide development of improved clinical protocols.

Comparison of the levels of accuracy of an artificial neural network model and a logistic regression model for the diagnosis of acute appendicitis

An accurate diagnosis of acute appendicitis in the early stage is often difficult, and decision support tools to improve such a diagnosis might be required. This study compared the levels of accuracy of artificial neural network models and logistic regression models for the diagnosis of acute appendicitis. Data from 169 patients presenting with acute abdomen were used for the analyses. Nine variables were used for the evaluation of the accuracy of the two models. The constructed models were validated by the ".632+ bootstrap method". The levels of accuracy of the two models for diagnosis were compared by error rate and areas under receiver operating characteristic curves. The artificial neural network models provided more accurate results than did the logistic regression models for both indices, especially when categorical variables or normalized variables were used. The most accurate diagnosis was obtained by the artificial neural network model using normalized variables.

Comparing the predictive value of neural network models to logistic regression models on the risk of death for small-cell lung cancer patients

Cancer is one of the leading causes of mortality in the developed world, and prognostic assessment of cancer patients is indispensable in medical care. Medical researchers are accustomed to using regression models to predict patient outcomes. Neural networks have been proposed as an alternative with great potential. Nonetheless, empirical evidence remains lacking to support the application of this technique as the appropriate method to investigate cancer prognosis. Utilizing data on patients from two National Cancer Institute of Canada clinical trials, we compared predictive accuracy of neural network models and logistic regression models on risk of death of limited-stage small-cell lung cancer patients. Our results suggest that neural network and logistic regression models have similar predictive accuracy. The distributions of individual predicted probabilities are very similar. On occasion, however, the prediction pairs are quite different, suggesting that they do not always give the same interpretations of the same variables.

The K-nearest neighbor algorithm predicted rehabilitation potential better than current Clinical Assessment Protocol

OBJECTIVE

There may be great potential for using computer-modeling techniques and machine-learning algorithms in clinical decision making, if these can be shown to produce results superior to clinical protocols currently in use. We aim to explore the potential to use an automatic, data-driven, machine-learning algorithm in clinical decision making.

STUDY DESIGN AND SETTING

Using a database containing comprehensive health assessment information (the interRAI-HC) on home care clients (N=24,724) from eight community-care regions in Ontario, Canada, we compare the performance of the K-nearest neighbor (KNN) algorithm and a Clinical Assessment Protocol (the "ADLCAP") currently used to predict rehabilitation potential. For our purposes, we define a patient as having rehabilitation potential if the patient had functional improvement or remained at home over a follow-up period of approximately 1 year.

RESULTS

The KNN algorithm has a lower false positive rate in all but one of the eight regions in the sample, and lower false negative rates in all regions. Compared using likelihood ratio statistics, KNN is uniformly more informative than the ADLCAP.

CONCLUSION

This article illustrates the potential for a machine-learning algorithm to enhance clinical decision making.

Developing a prognostic model for 90-day mortality after liver transplantation based on pretransplant recipient factors

BACKGROUND

Current statistical prognostic models for mortality after liver transplantation do not have good discriminatory ability. Furthermore, the methodology used to develop these models is often flawed. The objective of this paper is to develop a prognostic model for 90-day mortality after liver transplantation based on pretransplant recipient factors, employing a rigorous model development method.

METHODS

We used data on 4,829 patient that were prospectively collected for the UK & Ireland Liver Transplant Audit. Switching regression was employed to impute missing values combined with a bootstrapping approach for variable selection.

RESULTS

In all, 452 patients (9.4%) died within 90 days of their transplantation. The final prognostic model was well calibrated and discriminated moderately well between patients who did and who did not die (c-statistic 0.65, 95% CI [0.63, 0.68]). Although discrimination was not excellent overall, the results showed that those patients with a "low" chance of dying within 90 days of their transplant and those with a "high" chance of dying could be differentiated from patients with a "intermediate" chance.

CONCLUSIONS

Our model can provide transplant candidates with predictions of their early posttransplantation prospects before any donor information is known, which is essential information for patients with end-stage liver disease for whom liver transplantation is a treatment option.

Prediction of target range of intact parathyroid hormone in hemodialysis patients with artificial neural network

The application of artificial neural network (ANN) to predict outcome and explore potential relationships among clinical data is increasing being used in many clinical scenarios. The aim of this study was to validate whether an ANN is a useful tool for predicting the target range of plasma intact parathyroid hormone (iPTH) concentration in hemodialysis patients. An ANN was constructed with input variables collected retrospectively from an internal validation group (n = 129) of hemodialysis patients. Plasma iPTH was the dichotomous outcome variable, either target group (150 ng/L300 ng/L). After internal validation, the ANN was prospectively tested in an external validation group (n = 32) of hemodialysis patients. The final ANN was a multilayer perceptron network with six predictors including age, diabetes, hypertension, and blood biochemistries (hemoglobin, albumin, calcium). The externally validated ANN provided excellent discrimination as appraised by area under the receiver operating characteristic curve (0.83 +/- 0.11, p = 0.003). The Hosmer-Lemeshow statistic was 5.02 (p= 0.08 > 0.05) which represented a good-fit calibration. These results suggest that an ANN, which is based on limited clinical data, is able to accurately forecast the target range of plasma iPTH concentration in hemodialysis patients.