Performance comparison of multi-label learning algorithms on clinical data for chronic diseases

Non-experimental rapid identification of lower respiratory tract infections in patients with chronic obstructive pulmonary disease using multi-label learning

BACKGROUND AND OBJECTIVE

Microbiological culture is a standard diagnostic test that takes a long time to identify lower respiratory tract infections (LRTI) in patients with chronic obstructive pulmonary disease (COPD). This study entailed the development of an interactive decision-support system using multi-label machine learning. It is designed to assist clinical medical staff in the rapid and simultaneous diagnosis of various infections in these patients.

METHODS

Clinical health record data were collected from inpatients with COPD suspected of having a LRTI. Two major categories of multi-label learning frameworks were integrated with various machine learning algorithms to create 23 predictive models to identify four categories of infection: fungal, gram-negative bacterial, gram-positive bacterial, and multidrug-resistant organism infections. The predictive power of the individual models was tested. Subsequently, the model with the highest comprehensive performance was selected and integrated with SHAP technology to construct a decision support system.

RESULTS

Three-thousand-eight-hundred-one subjects participated in this study. LP-RF recorded the highest overall performance, with a Hamming loss of 0.158 (95 %CI: 0.157-0.159) and a samples-precision of 0.894 (95 %CI: 0.891-0.896). The developed diagnostic decision support system generates predicted probability output for each infection category in a specific patient and displays the interpreted output results.

CONCLUSION

The developed multi-label decision support system enables effective prediction of four categories of infections in patients with a history of COPD, and has the potential to curb the overuse of antimicrobial drugs. This system is highly explainable and interactive, providing real-time support in the simultaneous diagnosis of multiple infection categories.

Comparing machine learning algorithms for multimorbidity prediction: An example from the Elsa-Brasil study

BACKGROUND

Multimorbidity is a worldwide concern related to greater disability, worse quality of life, and mortality. The early prediction is crucial for preventive strategies design and integrative medical practice. However, knowledge about how to predict multimorbidity is limited, possibly due to the complexity involved in predicting multiple chronic diseases.

METHODS

In this study, we present the use of a machine learning approach to build cost-effective multimorbidity prediction models. Based on predictors easily obtainable in clinical practice (sociodemographic, clinical, family disease history and lifestyle), we build and compared the performance of seven multilabel classifiers (multivariate random forest, and classifier chain, binary relevance and binary dependence, with random forest and support vector machine as base classifiers), using a sample of 15105 participants from the Brazilian Longitudinal Study of Adult Health (ELSA-Brasil). We developed a web application for the building and use of prediction models.

RESULTS

Classifier chain with random forest as base classifier performed better (accuracy = 0.34, subset accuracy = 0.15, and Hamming Loss = 0.16). For different feature sets, random forest based classifiers outperformed those based on support vector machine. BMI, blood pressure, sex, and age were the features most relevant to multimorbidity prediction.

CONCLUSIONS

Our results support the choice of random forest based classifiers for multimorbidity prediction.

Identification and Prediction of Chronic Diseases Using Machine Learning Approach

Nowadays, humans face various diseases due to the current environmental condition and their living habits. The identification and prediction of such diseases at their earlier stages are much important, so as to prevent the extremity of it. It is difficult for doctors to manually identify the diseases accurately most of the time. The goal of this paper is to identify and predict the patients with more common chronic illnesses. This could be achieved by using a cutting-edge machine learning technique to ensure that this categorization reliably identifies persons with chronic diseases. The prediction of diseases is also a challenging task. Hence, data mining plays a critical role in disease prediction. The proposed system offers a broad disease prognosis based on patient's symptoms by using the machine learning algorithms such as convolutional neural network (CNN) for automatic feature extraction and disease prediction and K-nearest neighbor (KNN) for distance calculation to find the exact match in the data set and the final disease prediction outcome. A collection of disease symptoms has been performed for the preparation of the data set along with the person's living habits, and details related to doctor consultations are taken into account in this general disease prediction. Finally, a comparative study of the proposed system with various algorithms such as Naïve Bayes, decision tree, and logistic regression has been demonstrated in this paper.

Numerical Feature Transformation-based Sequence Generation Model for Multi-disease Diagnosis

The goal of computer-aided diagnosis is to predict patient’s diseases based on patient’s clinical data. The development of deep learning technology provides new help for clinical diagnosis. In this paper, we propose a new sequence generation model for multi-disease diagnosis prediction based on numerical feature transformation. Our model simultaneously uses patient’s laboratory test results and clinical text as input to diagnose and predict the disease that the patient may have. According to medical knowledge, our model can transform numerical features into descriptive text features, thereby enriching the semantic information of clinical texts. Besides, our model uses attention-based sequence generation methods to achieve the diagnosis of multiple diseases and better utilizes the correlation information between multiple diseases. We evaluate our model’s performance on a dataset of respiratory diseases from the real world, and experimental results show that our model’s accuracy reaches 42.75%, and the [Formula: see text] score reaches 65.65%, which is better than many other methods. It is suitable for the accurate diagnosis of multiple diseases.

Identification of social determinants of health using multi-label classification of electronic health record clinical notes

OBJECTIVES

Social determinants of health (SDH), key contributors to health, are rarely systematically measured and collected in the electronic health record (EHR). We investigate how to leverage clinical notes using novel applications of multi-label learning (MLL) to classify SDH in mental health and substance use disorder patients who frequent the emergency department.

METHODS AND MATERIALS

We labeled a gold-standard corpus of EHR clinical note sentences (N = 4063) with 6 identified SDH-related domains recommended by the Institute of Medicine for inclusion in the EHR. We then trained 5 classification models: linear-Support Vector Machine, K-Nearest Neighbors, Random Forest, XGBoost, and bidirectional Long Short-Term Memory (BI-LSTM). We adopted 5 common evaluation measures: accuracy, average precision-recall (AP), area under the curve receiver operating characteristic (AUC-ROC), Hamming loss, and log loss to compare the performance of different methods for MLL classification using the F1 score as the primary evaluation metric.

RESULTS

Our results suggested that, overall, BI-LSTM outperformed the other classification models in terms of AUC-ROC (93.9), AP (0.76), and Hamming loss (0.12). The AUC-ROC values of MLL models of SDH related domains varied between (0.59-1.0). We found that 44.6% of our study population (N = 1119) had at least one positive documentation of SDH.

DISCUSSION AND CONCLUSION

The proposed approach of training an MLL model on an SDH rich data source can produce a high performing classifier using only unstructured clinical notes. We also provide evidence that model performance is associated with lexical diversity by health professionals and the auto-generation of clinical note sentences to document SDH.

Estimating the class prior for positive and unlabelled data via logistic regression

In the paper, we revisit the problem of class prior probability estimation with positive and unlabelled data gathered in a single-sample scenario. The task is important as it is known that in positive unlabelled setting, a classifier can be successfully learned if the class prior is available. We show that without additional assumptions, class prior probability is not identifiable and thus the existing non-parametric estimators are necessarily biased in general if extra assumptions are not imposed. The magnitude of their bias is also investigated. The problem becomes identifiable when the probabilistic structure satisfies mild semi-parametric assumptions. Consequently, we propose a method based on a logistic fit and a concave minorization of its (non-concave) log-likelihood. The experiments conducted on artificial and benchmark datasets as well as on a large clinical database MIMIC indicate that the estimation errors for the proposed method are usually lower than for its competitors and that it is robust against departures from logistic settings.

Multi-disease Predictive Analytics: A Clinical Knowledge-aware Approach

Multi-Disease Predictive Analytics (MDPA) models simultaneously predict the risks of multiple diseases in patients and are valuable in early diagnoses. Patients tend to have multiple diseases simultaneously or develop multiple complications over time, and MDPA models can learn and effectively utilize such correlations between diseases. Data from large-scale Electronic Health Records (EHR) can be used through Multi-Label Learning (MLL) methods to develop MDPA models. However, data-driven approaches for MDPA face the challenge of data imbalance, because rare diseases tend to have much less data than common diseases. Insufficient data for rare diseases makes it difficult to leverage correlations with other diseases. These correlations are studied and recorded in biomedical literature but are rarely utilized in predictive analytics. This article presents a novel method called Knowledge-Aware Approach (KAA) that learns clinical correlations from the rapidly growing body of clinical knowledge. KAA can be combined with any data-driven MLL model for MDPA to refine the predictions of the model. Our extensive experiments, on real EHR data, show that the use of KAA improves the predictive performance of commonly used MDPA models, particularly for rare diseases. KAA is also found to be superior to existing general approaches of combining clinical knowledge with data-driven models. Further, a counterfactual analysis shows the efficacy of KAA in improving physicians’ ability to prescribe preventive treatments.

A classification model for prediction of clinical severity level using qSOFA medical score

Purpose

The purpose of this study is to develop an efficient prediction model using vital signs and standard medical score systems, which predicts the clinical severity level of the patient in advance based on the quick sequential organ failure assessment (qSOFA) medical score method.

Design/methodology/approach

To predict the clinical severity level of the patient in advance, the authors have formulated a training dataset that is constructed based on the qSOFA medical score method. Further, along with the multiple vital signs, different standard medical scores and their correlation features are used to build and improve the accuracy of the prediction model. It is made sure that the constructed training set is suitable for the severity level prediction because the formulated dataset has different clusters each corresponding to different severity levels according to qSOFA score.

Findings

From the experimental result, it is found that the inclusion of the standard medical scores and their correlation along with multiple vital signs improves the accuracy of the clinical severity level prediction model. In addition, the authors showed that the training dataset formulated from the temporal data (which includes vital signs and medical scores) based on the qSOFA medical scoring system has the clusters which correspond to each severity level in qSOFA score. Finally, it is found that RAndom k-labELsets multi-label classification performs better prediction of severity level compared to neural network-based multi-label classification.

Originality/value

This paper helps in identifying patient' clinical status.

Multi-label classification methods for improving comorbidities identification

The medical diagnostic process may be supported by computational classification techniques. In many cases, patients are affected by multiple illnesses, and more than one classification label is required to improve medical decision-making. In this paper, we consider a multi-perspective classification problem for medical diagnostics, where cases are described by labels from separate sets. We attempt to improve the identification of comorbidities using multi-label classification techniques. Several investigated methods, which provide label dependencies, are analysed and evaluated. The methods' performances are verified by experiments conducted on four sets of medical data from subject patients. The results were evaluated using several metrics and were statistically verified. We compare the effects of the techniques that do and do not consider label correlations. We demonstrate that multi-label classification methods from the first group outperform the techniques from the second one.

RETRACTED: Diagnosis labeling with disease-specific characteristics mining

This article has been retracted: please see Elsevier Policy on Article Withdrawal (https://www.elsevier.com/about/our-business/policies/article-withdrawal). This article has been retracted at the request of the authors; serious errors had been introduced within the data set presented and, as such, the authors have decided to retract the paper. Error summary is outlined below: 1. The dataset was divided into training set, validation set and testing set. The training set and validation set were used to tune the parameter. According to the authors this is an overfitting and can mislead the readers. 2. The compared baselines are not strong enough for fair comparison. The code was not requested from the original authors, but algorithms have been implemented. 3. The bag of words model is not trained suitably. The model was trained with the used dataset and the performance has been evaluated on this dataset. Authors think this is an overfitting and makes the comparison untrustworthy.

A clinical decision-making mechanism for context-aware and patient-specific remote monitoring systems using the correlations of multiple vital signs

BACKGROUND AND OBJECTIVES

In home-based context-aware monitoring patient's real-time data of multiple vital signs (e.g. heart rate, blood pressure) are continuously generated from wearable sensors. The changes in such vital parameters are highly correlated. They are also patient-centric and can be either recurrent or can fluctuate. The objective of this study is to develop an intelligent method for personalized monitoring and clinical decision support through early estimation of patient-specific vital sign values, and prediction of anomalies using the interrelation among multiple vital signs.

METHODS

In this paper, multi-label classification algorithms are applied in classifier design to forecast these values and related abnormalities. We proposed a completely new approach of patient-specific vital sign prediction system using their correlations. The developed technique can guide healthcare professionals to make accurate clinical decisions. Moreover, our model can support many patients with various clinical conditions concurrently by utilizing the power of cloud computing technology. The developed method also reduces the rate of false predictions in remote monitoring centres.

RESULTS

In the experimental settings, the statistical features and correlations of six vital signs are formulated as multi-label classification problem. Eight multi-label classification algorithms along with three fundamental machine learning algorithms are used and tested on a public dataset of 85 patients. Different multi-label classification evaluation measures such as Hamming score, F1-micro average, and accuracy are used for interpreting the prediction performance of patient-specific situation classifications. We achieved 90-95% Hamming score values across 24 classifier combinations for 85 different patients used in our experiment. The results are compared with single-label classifiers and without considering the correlations among the vitals. The comparisons show that multi-label method is the best technique for this problem domain.

CONCLUSIONS

The evaluation results reveal that multi-label classification techniques using the correlations among multiple vitals are effective ways for early estimation of future values of those vitals. In context-aware remote monitoring this process can greatly help the doctors in quick diagnostic decision making.