Supervised machine learning in the mass spectrometry laboratory: A tutorial

As the demand for laboratory testing by mass spectrometry increases, so does the need for automated methods for data analysis. Clinical mass spectrometry (MS) data is particularly well-suited for machine learning (ML) methods, which deal nicely with structured and discrete data elements. The alignment of these two fields offers a promising synergy that can be used to optimize workflows, improve result quality, and enhance our understanding of high-dimensional datasets and their inherent relationship with disease. In recent years, there has been an increasing number of publications that examine the capabilities of ML-based software in the context of chromatography and MS. However, given the historically distant nature between the fields of clinical chemistry and computer science, there is an opportunity to improve technological literacy of ML-based software within the clinical laboratory scientist community. To this end, we present a basic overview of ML and a tutorial of an ML-based experiment using a previously published MS dataset. The purpose of this paper is to describe the fundamental principles of supervised ML, outline the steps that are classically involved in an ML-based experiment, and discuss the purpose of good ML practice in the context of a binary MS classification problem.

Automatic emotion recognition in healthcare data using supervised machine learning

Human feelings are fundamental to perceive the conduct and state of mind of an individual. A healthy emotional state is one significant highlight to improve personal satisfaction. On the other hand, bad emotional health can prompt social or psychological well-being issues. Recognizing or detecting feelings in online health care data gives important and helpful information regarding the emotional state of patients. To recognize or detection of patient's emotion against a specific disease using text from online sources is a challenging task. In this paper, we propose a method for the automatic detection of patient's emotions in healthcare data using supervised machine learning approaches. For this purpose, we created a new dataset named EmoHD, comprising of 4,202 text samples against eight disease classes and six emotion classes, gathered from different online resources. We used six different supervised machine learning models based on different feature engineering techniques. We also performed a detailed comparison of the chosen six machine learning algorithms using different feature vectors on our dataset. We achieved the highest 87% accuracy using MultiLayer Perceptron as compared to other state of the art models. Moreover, we use the emotional guidance scale to show that there is a link between negative emotion and psychological health issues. Our proposed work will be helpful to automatically detect a patient's emotion during disease and to avoid extreme acts like suicide, mental disorders, or psychological health issues. The implementation details are made publicly available at the given link: https://bit.ly/2NQeGET.

Using supervised machine learning approach to predict treatment outcomes of vedolizumab in ulcerative colitis patients

With recent advances in machine learning, we demonstrated the use of supervised machine learning to optimize the prediction of treatment outcomes of vedolizumab through iterative optimization using VARSITY and VISIBLE 1 data in patients with moderate-to-severe ulcerative colitis. The analysis was carried out using elastic net regularized regression following a 2-stage training process. The model performance was assessed through AUROC, specificity, sensitivity, and accuracy. The generalizable predictive patterns suggest that easily obtained baseline and medical history variables may be able to predict therapeutic response to vedolizumab with clinically meaningful accuracy, implying a potential for individualized prescription of vedolizumab.

Digging into the behaviour of an active hunting predator: arctic fox prey caching events revealed by accelerometry

BACKGROUND

Biologging now allows detailed recording of animal movement, thus informing behavioural ecology in ways unthinkable just a few years ago. In particular, combining GPS and accelerometry allows spatially explicit tracking of various behaviours, including predation events in large terrestrial mammalian predators. Specifically, identification of location clusters resulting from prey handling allows efficient location of killing events. For small predators with short prey handling times, however, identifying predation events through technology remains unresolved. We propose that a promising avenue emerges when specific foraging behaviours generate diagnostic acceleration patterns. One such example is the caching behaviour of the arctic fox (Vulpes lagopus), an active hunting predator strongly relying on food storage when living in proximity to bird colonies.

METHODS

We equipped 16 Arctic foxes from Bylot Island (Nunavut, Canada) with GPS and accelerometers, yielding 23 fox-summers of movement data. Accelerometers recorded tri-axial acceleration at 50 Hz while we obtained a sample of simultaneous video recordings of fox behaviour. Multiple supervised machine learning algorithms were tested to classify accelerometry data into 4 behaviours: motionless, running, walking and digging, the latter being associated with food caching. Finally, we assessed the spatio-temporal concordance of fox digging and greater snow goose (Anser caerulescens antlanticus) nesting, to test the ecological relevance of our behavioural classification in a well-known study system dominated by top-down trophic interactions.

RESULTS

The random forest model yielded the best behavioural classification, with accuracies for each behaviour over 96%. Overall, arctic foxes spent 49% of the time motionless, 34% running, 9% walking, and 8% digging. The probability of digging increased with goose nest density and this result held during both goose egg incubation and brooding periods.

CONCLUSIONS

Accelerometry combined with GPS allowed us to track across space and time a critical foraging behaviour from a small active hunting predator, informing on spatio-temporal distribution of predation risk in an Arctic vertebrate community. Our study opens new possibilities for assessing the foraging behaviour of terrestrial predators, a key step to disentangle the subtle mechanisms structuring many predator-prey interactions and trophic networks.

Predicting the relationship between PFAS component signatures in water and non-water phases through mathematical transformation: Application to machine learning classification

Per- and polyfluoroalkyl substances (PFAS) are widespread in the environment, as a result of decades of use across a range of applications. While PFAS contamination often enters the environment in the aqueous phase, PFAS is regularly detected in a range of different phases, including soils, sediments and biota. Although PFAS at a given site may originate from the same sources, the compositions observed in different phases are nearly always different, a fact that can complicate source allocation efforts. This paper presents a quantitative method for prediction of the relative composition of PFAS in different phases for components for which differences in behavior are primarily driven by hydrophobicity. The derived equations suggest that under these conditions, the relative compositions in different phases in contact with water should be independent of overall affinity for the phase, and as such should be the same for all non-water phases. This result is illustrated with data from individual samples, as well as from site-wide evaluations for a range of different phases. The results of the work provide a useful tool to reconcile PFAS composition differences in different phases, and provide a baseline for recognizing cases where hydrophobicity is not the primary driver of differences in distribution between phases. Furthermore, the results may be useful in forensic applications for classification of PFAS across different phases. The use of the resulting equations to transform water data to train a supervised learning algorithm for forensic analysis of PFAS in non-water phases is illustrated.

MS2DeepScore: a novel deep learning similarity measure to compare tandem mass spectra

Mass spectrometry data is one of the key sources of information in many workflows in medicine and across the life sciences. Mass fragmentation spectra are generally considered to be characteristic signatures of the chemical compound they originate from, yet the chemical structure itself usually cannot be easily deduced from the spectrum. Often, spectral similarity measures are used as a proxy for structural similarity but this approach is strongly limited by a generally poor correlation between both metrics. Here, we propose MS2DeepScore: a novel Siamese neural network to predict the structural similarity between two chemical structures solely based on their MS/MS fragmentation spectra. Using a cleaned dataset of > 100,000 mass spectra of about 15,000 unique known compounds, we trained MS2DeepScore to predict structural similarity scores for spectrum pairs with high accuracy. In addition, sampling different model varieties through Monte-Carlo Dropout is used to further improve the predictions and assess the model's prediction uncertainty. On 3600 spectra of 500 unseen compounds, MS2DeepScore is able to identify highly-reliable structural matches and to predict Tanimoto scores for pairs of molecules based on their fragment spectra with a root mean squared error of about 0.15. Furthermore, the prediction uncertainty estimate can be used to select a subset of predictions with a root mean squared error of about 0.1. Furthermore, we demonstrate that MS2DeepScore outperforms classical spectral similarity measures in retrieving chemically related compound pairs from large mass spectral datasets, thereby illustrating its potential for spectral library matching. Finally, MS2DeepScore can also be used to create chemically meaningful mass spectral embeddings that could be used to cluster large numbers of spectra. Added to the recently introduced unsupervised Spec2Vec metric, we believe that machine learning-supported mass spectral similarity measures have great potential for a range of metabolomics data processing pipelines.

A pilot study for investigating the feasibility of supervised machine learning approaches for the classification of pedestrians struck by vehicles

This research focuses on the application of Artificial Intelligence (AI) methodologies to the problem of classifying vehicles involved in lethal pedestrian collisions. Specifically, the vehicle type is predicted on the basis of traumatic injury suffered by casualties, exploiting machine learning algorithms. In the present study, AI-assisted diagnosis was shown to have correct prediction about 70% of the time. In pedestrians struck by trucks, more severe injuries were appreciated in the facial skeleton, lungs, major airways, liver, and spleen as well as in the sternum/clavicle/rib complex, whereas the lower extremities were more affected by fractures in pedestrians struck by cars. Although the distinction of the striking vehicle should develop beyond autopsy evidence alone, the presented approach which is novel in the realm of forensic science, is shown to be effective in building automated decision support systems. Outcomes from this system can provide valuable information after the execution of autoptic examinations supporting the forensic investigation. Preliminary results from the application of machine learning algorithms with real-world datasets seem to highlight the efficacy of the proposed approach, which could be used for further studies concerning this topic.

Single-center versus multi-center biparametric MRI radiomics approach for clinically significant peripheral zone prostate cancer

Objectives

To investigate a previously developed radiomics-based biparametric magnetic resonance imaging (bpMRI) approach for discrimination of clinically significant peripheral zone prostate cancer (PZ csPCa) using multi-center, multi-vendor (McMv) and single-center, single-vendor (ScSv) datasets.

Methods

This study’s starting point was a previously developed ScSv algorithm for PZ csPCa whose performance was demonstrated in a single-center dataset. A McMv dataset was collected, and 262 PZ PCa lesions (9 centers, 2 vendors) were selected to identically develop a multi-center algorithm. The single-center algorithm was then applied to the multi-center dataset (single–multi-validation), and the McMv algorithm was applied to both the multi-center dataset (multi–multi-validation) and the previously used single-center dataset (multi–single-validation). The areas under the curve (AUCs) of the validations were compared using bootstrapping.

Results

Previously the single–single validation achieved an AUC of 0.82 (95% CI 0.71–0.92), a significant performance reduction of 27.2% compared to the single–multi-validation AUC of 0.59 (95% CI 0.51–0.68). The new multi-center model achieved a multi–multi-validation AUC of 0.75 (95% CI 0.64–0.84). Compared to the multi–single-validation AUC of 0.66 (95% CI 0.56–0.75), the performance did not decrease significantly (p value: 0.114). Bootstrapped comparison showed similar single-center performances and a significantly different multi-center performance (p values: 0.03, 0.012).

Conclusions

A single-center trained radiomics-based bpMRI model does not generalize to multi-center data. Multi-center trained radiomics-based bpMRI models do generalize, have equal single-center performance and perform better on multi-center data.

Supplementary Information

The online version contains supplementary material available at 10.1186/s13244-021-01099-y.

Metabolic Subtyping of Adrenal Tumors: Prospective Multi-Center Cohort Study in Korea

BACKGROUND

Conventional diagnostic approaches for adrenal tumors require multi-step processes, including imaging studies and dynamic hormone tests. Therefore, this study aimed to discriminate adrenal tumors from a single blood sample based on the combination of liquid chromatography-mass spectrometry (LC-MS) and machine learning algorithms in serum profiling of adrenal steroids.

METHODS

The LC-MS-based steroid profiling was applied to serum samples obtained from patients with nonfunctioning adenoma (NFA, n=73), Cushing's syndrome (CS, n=30), and primary aldosteronism (PA, n=40) in a prospective multicenter study of adrenal disease. The decision tree (DT), random forest (RF), and extreme gradient boost (XGBoost) were performed to categorize the subtypes of adrenal tumors.

RESULTS

The CS group showed higher serum levels of 11-deoxycortisol than the NFA group, and increased levels of tetrahydrocortisone (THE), 20α-dihydrocortisol, and 6β-hydroxycortisol were found in the PA group. However, the CS group showed lower levels of dehydroepiandrosterone (DHEA) and its sulfate derivative (DHEA-S) than both the NFA and PA groups. Patients with PA expressed higher serum 18-hydroxycortisol and DHEA but lower THE than NFA patients. The balanced accuracies of DT, RF, and XGBoost for classifying each type were 78%, 96%, and 97%, respectively. In receiver operating characteristics (ROC) analysis for CS, XGBoost, and RF showed a significantly greater diagnostic power than the DT. However, in ROC analysis for PA, only RF exhibited better diagnostic performance than DT.

CONCLUSION

The combination of LC-MS-based steroid profiling with machine learning algorithms could be a promising one-step diagnostic approach for the classification of adrenal tumor subtypes.

Automated biomarker candidate discovery in imaging mass spectrometry data through spatially localized Shapley additive explanations

The search for molecular species that are differentially expressed between biological states is an important step towards discovering promising biomarker candidates. In imaging mass spectrometry (IMS), performing this search manually is often impractical due to the large size and high-dimensionality of IMS datasets. Instead, we propose an interpretable machine learning workflow that automatically identifies biomarker candidates by their mass-to-charge ratios, and that quantitatively estimates their relevance to recognizing a given biological class using Shapley additive explanations (SHAP). The task of biomarker candidate discovery is translated into a feature ranking problem: given a classification model that assigns pixels to different biological classes on the basis of their mass spectra, the molecular species that the model uses as features are ranked in descending order of relative predictive importance such that the top-ranking features have a higher likelihood of being useful biomarkers. Besides providing the user with an experiment-wide measure of a molecular species' biomarker potential, our workflow delivers spatially localized explanations of the classification model's decision-making process in the form of a novel representation called SHAP maps. SHAP maps deliver insight into the spatial specificity of biomarker candidates by highlighting in which regions of the tissue sample each feature provides discriminative information and in which regions it does not. SHAP maps also enable one to determine whether the relationship between a biomarker candidate and a biological state of interest is correlative or anticorrelative. Our automated approach to estimating a molecular species' potential for characterizing a user-provided biological class, combined with the untargeted and multiplexed nature of IMS, allows for the rapid screening of thousands of molecular species and the obtention of a broader biomarker candidate shortlist than would be possible through targeted manual assessment. Our biomarker candidate discovery workflow is demonstrated on mouse-pup and rat kidney case studies.

A new hybrid record linkage process to make epidemiological databases interoperable: application to the GEMO and GENEPSO studies involving BRCA1 and BRCA2 mutation carriers

BACKGROUND

Linking independent sources of data describing the same individuals enable innovative epidemiological and health studies but require a robust record linkage approach. We describe a hybrid record linkage process to link databases from two independent ongoing French national studies, GEMO (Genetic Modifiers of BRCA1 and BRCA2), which focuses on the identification of genetic factors modifying cancer risk of BRCA1 and BRCA2 mutation carriers, and GENEPSO (prospective cohort of BRCAx mutation carriers), which focuses on environmental and lifestyle risk factors.

METHODS

To identify as many as possible of the individuals participating in the two studies but not registered by a shared identifier, we combined probabilistic record linkage (PRL) and supervised machine learning (ML). This approach (named "PRL + ML") combined together the candidate matches identified by both approaches. We built the ML model using the gold standard on a first version of the two databases as a training dataset. This gold standard was obtained from PRL-derived matches verified by an exhaustive manual review. Results The Random Forest (RF) algorithm showed a highest recall (0.985) among six widely used ML algorithms: RF, Bagged trees, AdaBoost, Support Vector Machine, Neural Network. Therefore, RF was selected to build the ML model since our goal was to identify the maximum number of true matches. Our combined linkage PRL + ML showed a higher recall (range 0.988-0.992) than either PRL (range 0.916-0.991) or ML (0.981) alone. It identified 1995 individuals participating in both GEMO (6375 participants) and GENEPSO (4925 participants).

CONCLUSIONS

Our hybrid linkage process represents an efficient tool for linking GEMO and GENEPSO. It may be generalizable to other epidemiological studies involving other databases and registries.

Determining Follow-Up Imaging Study Using Radiology Reports

Radiology reports often contain follow-up imaging recommendations. Failure to comply with these recommendations in a timely manner can lead to delayed treatment, poor patient outcomes, complications, unnecessary testing, lost revenue, and legal liability. The objective of this study was to develop a scalable approach to automatically identify the completion of a follow-up imaging study recommended by a radiologist in a preceding report. We selected imaging-reports containing 559 follow-up imaging recommendations and all subsequent reports from a multi-hospital academic practice. Three radiologists identified appropriate follow-up examinations among the subsequent reports for the same patient, if any, to establish a ground-truth dataset. We then trained an Extremely Randomized Trees that uses recommendation attributes, study meta-data and text similarity of the radiology reports to determine the most likely follow-up examination for a preceding recommendation. Pairwise inter-annotator F-score ranged from 0.853 to 0.868; the corresponding F-score of the classifier in identifying follow-up exams was 0.807. Our study describes a methodology to automatically determine the most likely follow-up exam after a follow-up imaging recommendation. The accuracy of the algorithm suggests that automated methods can be integrated into a follow-up management application to improve adherence to follow-up imaging recommendations. Radiology administrators could use such a system to monitor follow-up compliance rates and proactively send reminders to primary care providers and/or patients to improve adherence.

Extraction and assessment of diagnosis-relevant features for heart murmur classification

This paper presents a heart murmur detection and multi-class classification approach via machine learning. We extracted heart sound and murmur features that are of diagnostic importance and developed additional 16 features that are not perceivable by human ears but are valuable to improve murmur classification accuracy. We examined and compared the classification performance of supervised machine learning with k-nearest neighbor (KNN) and support vector machine (SVM) algorithms. We put together a test repertoire having more than 450 heart sound and murmur episodes to evaluate the performance of murmur classification using cross-validation of 80-20 and 90-10 splits. As clearly demonstrated in our evaluation, the specific set of features chosen in our study resulted in accurate classification consistently exceeding 90% for both classifiers.

[Keratoconus detection and classification from parameters of the Corvis®ST : A study based on algorithms of machine learning]

BACKGROUND AND OBJECTIVE

In the last decades increasingly more systems of artificial intelligence have been established in medicine, which identify diseases or pathologies or discriminate them from complimentary diseases. Up to now the Corvis®ST (Corneal Visualization Scheimpflug Technology, Corvis®ST, Oculus, Wetzlar, Germany) yielded a binary index for classifying keratoconus but did not enable staging. The purpose of this study was to develop a prediction model, which mimics the topographic keratoconus classification index (TKC) of the Pentacam high resolution (HR, Oculus) with measurement parameters extracted from the Corvis®ST.

PATIENTS AND METHODS

In this study 60 measurements from normal subjects (TKC 0) and 379 eyes with keratoconus (TKC 1-4) were recruited. After measurement with the Pentacam HR (target parameter TKC) a measurement with the Corvis®ST device was performed. From this device 6 dynamic response parameters were extracted, which were included in the Corvis biomechanical index (CBI) provided by the Corvis®ST (ARTh, SP-A1, DA ratio 1 mm, DA ratio 2 mm, A1 velocity, max. deformation amplitude). In addition to the TKC as the target, the binarized TKC (1: TKC 1-4, 0: TKC 0) was modelled. The performance of the model was validated with accuracy as an indicator for correct classification made by the algorithm. Misclassifications in the modelling were penalized by the number of stages of deviation between the modelled and measured TKC values.

RESULTS

A total of 24 different models of supervised machine learning from 6 different families were tested. For modelling of the TKC stages 0-4, the algorithm based on a support vector machine (SVM) with linear kernel showed the best performance with an accuracy of 65.1% correct classifications. For modelling of binarized TKC, a decision tree with a coarse resolution showed a superior performance with an accuracy of 95.2% correct classifications followed by the SVM with linear or quadratic kernel and a nearest neighborhood classifier with cubic kernel (94.5% each).

CONCLUSION

This study aimed to show the principle of supervised machine learning applied to a set-up for the modelled classification of keratoconus staging. Preprocessed measurement data extracted from the Corvis®ST device were used to mimic the TKC provided by the Pentacam device with a series of different algorithms of machine learning.

Supervised machine learning-based prediction for in-hospital pressure injury development using electronic health records: A retrospective observational cohort study in a university hospital in Japan

BACKGROUND

In hospitals, nurses are responsible for pressure injury risk assessment using several kinds of risk assessment scales. However, their predictive validity is insufficient to initiate targeted preventive strategy for each patient. The use of electronic health records with machine learning technique is a promising strategy to provide automated clinical decision-making aid.

OBJECTIVE

The purpose of this study was to construct a predictive model for pressure injury development which included feature variables that can be collected on the first day of hospitalization by nurses who routinely input the data to electronic health records.

DESIGN

Retrospective observational cohort study.

SETTING

This study was conducted at a university hospital in Japan.

PARTICIPANTS

This study used electronic health records, which include entry/discharge records, basic nursing records, and pressure injury management documents (N = 75,353).

METHODS

The outcome measure was the pressure injuries which developed outside of an operation theatre and frequently appeared on the specific body parts at high risk of pressure injury development. We utilized four major classifiers: logistic regression, random forest, linear support vector machine, and extreme gradient boosting (XGBoost) with 5-fold cross-validation technique. The area under the receiver operating characteristic curve (AUC) was used for evaluating predictive performance.

RESULTS

The proportion of hospital-acquired pressure injuries was 0.52%. The receiver operating characteristic curves revealed the best predictive performance for the XGBoost model, achieving the highest sensitivity of 0.78±0.03 and AUC of 0.80±0.02 amongst four types of classifiers. Variables related to difficulty in activities of daily living, anorexia, and respiratory or cardiac disorders were extracted as important features.

CONCLUSIONS

Our findings suggest that routinely collected health data by nurses on the first day of patient admission have the potential to help determine high-risk patients for pressure injury development. Tweetable abstract: Machine learning models on routinely collected electronic health records data successfully predict pressure injury development during hospitalization.

FUNDING

This work was supported by a JSPS KAKENHI Grant-in-Aid for Exploratory Research (16K15865).

Reliable or not? An automated classification of webpages about early childhood vaccination using supervised machine learning

OBJECTIVE

To investigate the applicability of supervised machine learning (SML) to classify health-related webpages as 'reliable' or 'unreliable' in an automated way.

METHODS

We collected the textual content of 468 different Dutch webpages about early childhood vaccination. Webpages were manually coded as 'reliable' or 'unreliable' based on their alignment with evidence-based vaccination guidelines. Four SML models were trained on part of the data, whereas the remaining data was used for model testing.

RESULTS

All models appeared to be successful in the automated identification of unreliable (F1 scores: 0.54-0.86) and reliable information (F1 scores: 0.82-0.91). Typical words for unreliable information are 'dr', 'immune system', and 'vaccine damage', whereas 'measles', 'child', and 'immunization rate', were frequent in reliable information. Our best performing model was also successful in terms of out-of-sample prediction, tested on a dataset about HPV vaccination.

CONCLUSION

Automated classification of online content in terms of reliability, using basic classifiers, performs well and is particularly useful to identify reliable information.

PRACTICE IMPLICATIONS

The classifiers can be used as a starting point to develop more complex classifiers, but also warning tools which can help people evaluate the content they encounter online.

Metabolites as predictive biomarkers for Trypanosoma cruzi exposure in triatomine bugs

Trypanosoma cruzi, the causative agent of Chagas disease (American trypanosomiasis), colonizes the intestinal tract of triatomines. Triatomine bugs act as vectors in the life cycle of the parasite and transmit infective parasite stages to animals and humans. Contact of the vector with T. cruzi alters its intestinal microbial composition, which may also affect the associated metabolic patterns of the insect. Earlier studies suggest that the complexity of the triatomine fecal metabolome may play a role in vector competence for different T. cruzi strains. Using high-resolution mass spectrometry and supervised machine learning, we aimed to detect differences in the intestinal metabolome of the triatomine Rhodnius prolixus and predict whether the insect had been exposed to T. cruzi or not based solely upon their metabolic profile. We were able to predict the exposure status of R. prolixus to T. cruzi with accuracies of 93.6%, 94.2% and 91.8% using logistic regression, a random forest classifier and a gradient boosting machine model, respectively. We extracted the most important features in producing the models and identified the major metabolites which assist in positive classification. This work highlights the complex interactions between triatomine vector and parasite including effects on the metabolic signature of the insect.

Predictive and diagnosis models of stroke from hemodynamic signal monitoring

This work presents a novel and promising approach to the clinical management of acute stroke. Using machine learning techniques, our research has succeeded in developing accurate diagnosis and prediction real-time models from hemodynamic data. These models are able to diagnose stroke subtype with 30 min of monitoring, to predict the exitus during the first 3 h of monitoring, and to predict the stroke recurrence in just 15 min of monitoring. Patients with difficult access to a CT scan and all patients that arrive at the stroke unit of a specialized hospital will benefit from these positive results. The results obtained from the real-time developed models are the following: stroke diagnosis around 98% precision (97.8% sensitivity, 99.5% specificity), exitus prediction with 99.8% precision (99.8% Sens., 99.9% Spec.), and 98% precision predicting stroke recurrence (98% Sens., 99% Spec.). Graphical abstract depicting the complete process since a patient is monitored until the data collected is used to generate models.

OASIS +: leveraging machine learning to improve the prognostic accuracy of OASIS severity score for predicting in-hospital mortality

Background

Severity scores assess the acuity of critical illness by penalizing for the deviation of physiologic measurements from normal and aggregating these penalties (also called “weights” or “subscores”) into a final score (or probability) for quantifying the severity of critical illness (or the likelihood of in-hospital mortality). Although these simple additive models are human readable and interpretable, their predictive performance needs to be further improved.

Methods

We present OASIS +, a variant of the Oxford Acute Severity of Illness Score (OASIS) in which an ensemble of 200 decision trees is used to predict in-hospital mortality based on the 10 same clinical variables in OASIS.

Results

Using a test set of 9566 admissions extracted from the MIMIC-III database, we show that OASIS + outperforms nine previously developed severity scoring methods (including OASIS) in predicting in-hospital mortality. Furthermore, our results show that the supervised learning algorithms considered in our experiments demonstrated higher predictive performance when trained using the observed clinical variables as opposed to OASIS subscores.

Conclusions

Our results suggest that there is room for improving the prognostic accuracy of the OASIS severity scores by replacing the simple linear additive scoring function with more sophisticated non-linear machine learning models such as RF and XGB.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12911-021-01517-7.

Automatic Identification of Upper Extremity Rehabilitation Exercise Type and Dose Using Body-Worn Sensors and Machine Learning: A Pilot Study

BACKGROUND

Prior studies suggest that participation in rehabilitation exercises improves motor function poststroke; however, studies on optimal exercise dose and timing have been limited by the technical challenge of quantifying exercise activities over multiple days.

OBJECTIVES

The objectives of this study were to assess the feasibility of using body-worn sensors to track rehabilitation exercises in the inpatient setting and investigate which recording parameters and data analysis strategies are sufficient for accurately identifying and counting exercise repetitions.

METHODS

MC10 BioStampRC® sensors were used to measure accelerometer and gyroscope data from upper extremities of healthy controls (n = 13) and individuals with upper extremity weakness due to recent stroke (n = 13) while the subjects performed 3 preselected arm exercises. Sensor data were then labeled by exercise type and this labeled data set was used to train a machine learning classification algorithm for identifying exercise type. The machine learning algorithm and a peak-finding algorithm were used to count exercise repetitions in non-labeled data sets.

RESULTS

We achieved a repetition counting accuracy of 95.6% overall, and 95.0% in patients with upper extremity weakness due to stroke when using both accelerometer and gyroscope data. Accuracy was decreased when using fewer sensors or using accelerometer data alone.

CONCLUSIONS

Our exploratory study suggests that body-worn sensor systems are technically feasible, well tolerated in subjects with recent stroke, and may ultimately be useful for developing a system to measure total exercise "dose" in poststroke patients during clinical rehabilitation or clinical trials.

Combination of network and molecule structure accurately predicts competitive inhibitory interactions

Mining of metabolite-protein interaction networks facilitates the identification of design principles underlying the regulation of different cellular processes. However, identification and characterization of the regulatory role that metabolites play in interactions with proteins on a genome-scale level remains a pressing task. Based on availability of high-quality metabolite-protein interaction networks and genome-scale metabolic networks, here we propose a supervised machine learning approach, called CIRI that determines whether or not a metabolite is involved in a competitive inhibitory regulatory interaction with an enzyme. First, we show that CIRI outperforms the naive approach based on a structural similarity threshold for a putative competitive inhibitor and the substrates of a metabolic reaction. We also validate the performance of CIRI on several unseen data sets and databases of metabolite-protein interactions not used in the training, and demonstrate that the classifier can be effectively used to predict competitive inhibitory interactions. Finally, we show that CIRI can be employed to refine predictions about metabolite-protein interactions from a recently proposed PROMIS approach that employs metabolomics and proteomics profiles from size exclusion chromatography in E. coli to predict metabolite-protein interactions. Altogether, CIRI fills a gap in cataloguing metabolite-protein interactions and can be used in directing future machine learning efforts to categorize the regulatory type of these interactions.

Who is asked about alcohol consumption? A retrospective cohort study using a national repository of Electronic Medical Records

Documentation of alcohol use in electronic medical record (EMR) informs interventions to reduce alcohol-related morbidity and mortality. This retrospective cohort study explored EMR data from 960 primary care providers participating in the Canadian Primary Care Sentinel Surveillance Network to describe documentation of alcohol use (e.g. none, current or past use) in the EMR. Included providers represented 700,620 adult patients from across Canada with an encounter between 2015 and 2018. Bivariate comparisons characterized the patients with, and without, documentation of alcohol use. Multivariate generalized estimating equation models with logit function assessed patient and provider characteristics associated with (1) documentation of alcohol and (2) patients with heightened risk for alcohol-related problems. Forty percent of patients had alcohol use documentation in the EMR. Light alcohol consumption was recorded for 43.6% of these patients. Male patients (OR1.09, CI 1.07-1.12), who were older (OR1.26, CI 1.23-1.30), had more frequent visits to their provider (OR1.11, CI 1.09-1.13) and had hypertension (OR1.07, CI 1.06-1.09) or depression (OR1.07, CI 1.09-1.14) had higher odds of alcohol documentation. There were 4.7% of patients with a record indicating heightened risk for alcohol-related problems. Male patients (OR3.27 CI 3.14-3.4), patients with depression (OR2.01 CI1.93-2.1) and rural residency (OR1.35 CI1.29-1.42) was associated with risk for alcohol-related problems. Heavy alcohol consumption is associated with an increased risk of negative health outcomes, particularly for patients with certain chronic conditions. However, these patients do not have alcohol use consistently documented in the EMR. Strategies should be designed and implemented to support more consistent alcohol-screening among high-risk patients.

Mean parotid dose prediction model using machine learning regression method for intensity-modulated radiotherapy in head and neck cancer

Parotids are considered one of the major organs at risk in Head and Neck (HN) intensity-modulated radiotherapy (IMRT). Achieving proper target coverage with reduced mean parotid dose demands an elaborate time-consuming IMRT plan optimization. A parotid mean dose prediction model based on a machine-learning linear regression was developed and validated in this study. The model was developed using independent variables, such as parotid to PTV overlapping volume, dose coverage of the overlapping PTV, the ratio of overlapping parotid volume to total parotid volume, and volume of parotid overlapping with isotopically expanded PTV contours. The Pearson correlation coefficients between these independent variables and the mean parotid dose were calculated. Multicollinearity of the independent variables was checked by calculating the Variance Inflation Factor (VIF). All variables are having VIF less than ten were taken for the model. Fifty IMRT patient plans were used to develop the model. The mean parotid dose predicted by the model was in good agreement with the obtained mean parotid dose. The model is having a Root Mean Square Error (RMSE) of 2.89 Gy and an R-square of 0.7695. The model was successfully validated using the fivefold cross-validation method, resulting R-square value of 0.6179 and an RMSE of 2.93 Gy. The normality of the model's residuals was tested using Quartile-Quartile (Q-Q) plot and Shapiro Wilk test (p = 0.996, for null hypothesis ``residuals were normally distributed''). The data points in the Q-Q plot are falling approximately along the reference line. This model can be used in clinics to help the planner in the preplanning phase for efficient plan optimization.

Disease-Relevant Single Cell Photonic Signatures Identify S100β Stem Cells and their Myogenic Progeny in Vascular Lesions

A hallmark of subclinical atherosclerosis is the accumulation of vascular smooth muscle cell (SMC)-like cells leading to intimal thickening and lesion formation. While medial SMCs contribute to vascular lesions, the involvement of resident vascular stem cells (vSCs) remains unclear. We evaluated single cell photonics as a discriminator of cell phenotype in vitro before the presence of vSC within vascular lesions was assessed ex vivo using supervised machine learning and further validated using lineage tracing analysis. Using a novel lab-on-a-Disk(Load) platform, label-free single cell photonic emissions from normal and injured vessels ex vivo were interrogated and compared to freshly isolated aortic SMCs, cultured Movas SMCs, macrophages, B-cells, S100β⁺ mVSc, bone marrow derived mesenchymal stem cells (MSC) and their respective myogenic progeny across five broadband light wavelengths (λ465 - λ670 ± 20 nm). We found that profiles were of sufficient coverage, specificity, and quality to clearly distinguish medial SMCs from different vascular beds (carotid vs aorta), discriminate normal carotid medial SMCs from lesional SMC-like cells ex vivo following flow restriction, and identify SMC differentiation of a series of multipotent stem cells following treatment with transforming growth factor beta 1 (TGF- β1), the Notch ligand Jagged1, and Sonic Hedgehog using multivariate analysis, in part, due to photonic emissions from enhanced collagen III and elastin expression. Supervised machine learning supported genetic lineage tracing analysis of S100β⁺ vSCs and identified the presence of S100β⁺vSC-derived myogenic progeny within vascular lesions. We conclude disease-relevant photonic signatures may have predictive value for vascular disease.

MOD: A novel machine-learning optimal-filtering method for accurate and efficient detection of subthreshold synaptic events in vivo

BACKGROUND

To understand information coding in single neurons, it is necessary to analyze subthreshold synaptic events, action potentials (APs), and their interrelation in different behavioral states. However, detecting excitatory postsynaptic potentials (EPSPs) or currents (EPSCs) in behaving animals remains challenging, because of unfavorable signal-to-noise ratio, high frequency, fluctuating amplitude, and variable time course of synaptic events.

NEW METHOD

We developed a method for synaptic event detection, termed MOD (Machine-learning Optimal-filtering Detection-procedure), which combines concepts of supervised machine learning and optimal Wiener filtering. Experts were asked to manually score short epochs of data. The algorithm was trained to obtain the optimal filter coefficients of a Wiener filter and the optimal detection threshold. Scored and unscored data were then processed with the optimal filter, and events were detected as peaks above threshold.

RESULTS

We challenged MOD with EPSP traces in vivo in mice during spatial navigation and EPSC traces in vitro in slices under conditions of enhanced transmitter release. The area under the curve (AUC) of the receiver operating characteristics (ROC) curve was, on average, 0.894 for in vivo and 0.969 for in vitro data sets, indicating high detection accuracy and efficiency.

COMPARISON WITH EXISTING METHODS

When benchmarked using a (1 - AUC)^-1 metric, MOD outperformed previous methods (template-fit, deconvolution, and Bayesian methods) by an average factor of 3.13 for in vivo data sets, but showed comparable (template-fit, deconvolution) or higher (Bayesian) computational efficacy.

CONCLUSIONS

MOD may become an important new tool for large-scale, real-time analysis of synaptic activity.

Identification of the hub genes in gastric cancer through weighted gene co-expression network analysis

Background

Gastric cancer is one of the most lethal tumors and is characterized by poor prognosis and lack of effective diagnostic or therapeutic biomarkers. The aim of this study was to find hub genes serving as biomarkers in gastric cancer diagnosis and therapy.

Methods

GSE66229 from Gene Expression Omnibus (GEO) was used as training set. Genes bearing the top 25% standard deviations among all the samples in training set were performed to systematic weighted gene co-expression network analysis (WGCNA) to find candidate genes. Then, hub genes were further screened by using the “least absolute shrinkage and selection operator” (LASSO) logistic regression. Finally, hub genes were validated in the GSE54129 dataset from GEO by supervised learning method artificial neural network (ANN) algorithm.

Results

Twelve modules with strong preservation were identified by using WGCNA methods in training set. Of which, five modules significantly related to gastric cancer were selected as clinically significant modules, and 713 candidate genes were identified from these five modules. Then, ADIPOQ, ARHGAP39, ATAD3A, C1orf95, CWH43, GRIK3, INHBA, RDH12, SCNN1G, SIGLEC11 and LYVE1 were screened as the hub genes. These hub genes successfully differentiated the tumor samples from the healthy tissues in an independent testing set through artificial neural network algorithm with the area under the receiver operating characteristic curve at 0.946.

Conclusions

These hub genes bearing diagnostic and therapeutic values, and our results may provide a novel prospect for the diagnosis and treatment of gastric cancer in the future.

A machine learning approach to modeling PTSD and difficulties in emotion regulation

Despite evidence for the association between emotion regulation difficulties and posttraumatic stress disorder (PTSD), less is known about the specific emotion regulation abilities that are most relevant to PTSD severity. This study examined both item-level and subscale-level models of difficulties in emotion regulation in relation to PTSD severity using supervised machine learning in a sample of U.S. adults (N=570). Participants were recruited via Amazon's Mechanical Turk (MTurk) and completed self-report measures of emotion regulation difficulties and PTSD severity. We used five different machine learning algorithms separately to train each statistical model. Using ridge and elastic net regression results in the testing sample, emotion regulation predictor variables accounted for approximately 28% and 27% of the variance in PTSD severity in the item- and subscale-level models, respectively. In the item-level model, four predictor variables had notable relative importance values for PTSD severity. These items captured secondary emotional responding, experiencing emotions as out-of-control, difficulties modulating emotional arousal, and low emotional granularity. In the subscale-level model, lack of access to effective emotion regulation strategies, lack of emotional clarity, and emotional nonacceptance subscales had the highest relative importance to PTSD severity. Results from analyses modeling a probable diagnosis of PTSD based on DERS items and subscales are presented in supplemental findings. Findings have implications for developing more efficient, targeted emotion regulation interventions for PTSD.

Detecting depression using an ensemble classifier based on Quality of Life scales

Major depressive disorder (MDD) is an issue that affects 350 million people worldwide. Traditional approaches have been to identify depressive symptoms in datasets, but recently, research is beginning to explore the association between psychosocial factors such as those on the quality of life scale and mental well-being, which will lead to earlier diagnosis and prediction of MDD. In this research, an ensemble binary classifier is proposed to analyse health survey data against ground truth from the SF-20 Quality of Life scales. The classifier aims to improve the performance of machine learning techniques on large datasets and identify depressed cases based on associations between items on the QoL scale and mental illness by increasing predictive performance. On the experimental evaluation on the National Health and Nutrition Examination Survey (NHANES), the classifier demonstrated an F1 score of 0.976 in the prediction, without any incorrectly identified depression instances. Only about 4% of instances had been mistakenly classified into depressed cases, with a significant accuracy of 95.4% comparing to the result from PHQ-9 mental screen inventory. The presented ensemble binary classifier performed comparably better than each baseline algorithm in all measures and all experiments. We trained the ensemble model on the processed NHANES dataset, tested and evaluated the results of its performance against mental screen inventory and discussed the comparable predictions. Finally, we provided future research directions.

An attempt at estrus detection in cattle by continuous measurements of ventral tail base surface temperature with supervised machine learning

We aimed to determine the effectiveness of estrus detection based on continuous measurements of the ventral tail base surface temperature (ST) with supervised machine learning in cattle. ST data were obtained through 51 estrus cycles on 11 female cattle (six Holsteins and five Japanese Blacks) using the tail-attached sensor. Three estrus detection models were constructed with the training data (n = 17) using machine learning techniques (random forest, artificial neural network, and support vector machine) based on 13 features extracted from sensing data (indicative of estrus-associated ST changes). Estrus detection abilities of the three models on test data (n = 34) were not statistically different among models in terms of sensitivity and precision (range 50.0% to 58.8% and 60.6% to 73.1%, respectively). The relatively poor performance of the models might indicate the difficulty of separating estrus-associated ST changes from estrus-independent fluctuations in ST.

Pitfalls in training and validation of deep learning systems

The number of publications in endoscopic journals that present deep learning applications has risen tremendously over the past years. Deep learning has shown great promise for automated detection, diagnosis and quality improvement in endoscopy. However, the interdisciplinary nature of these works has undoubtedly made it more difficult to estimate their value and applicability. In this review, the pitfalls and common misconducts when training and validating deep learning systems are discussed and some practical guidelines are proposed that should be taken into account when acquiring data and handling it to ensure an unbiased system that will generalize for application in routine clinical practice. Finally, some considerations are presented to ensure correct validation and comparison of AI systems.

Development of machine learning models to predict RT-PCR results for severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) in patients with influenza-like symptoms using only basic clinical data

BACKGROUND

Reverse Transcription-Polymerase Chain Reaction (RT-PCR) for Severe Acute Respiratory Syndrome Coronavirus 2 (SARS-COV-2) diagnosis currently requires quite a long time span. A quicker and more efficient diagnostic tool in emergency departments could improve management during this global crisis. Our main goal was assessing the accuracy of artificial intelligence in predicting the results of RT-PCR for SARS-COV-2, using basic information at hand in all emergency departments.

METHODS

This is a retrospective study carried out between February 22, 2020 and March 16, 2020 in one of the main hospitals in Milan, Italy. We screened for eligibility all patients admitted with influenza-like symptoms tested for SARS-COV-2. Patients under 12 years old and patients in whom the leukocyte formula was not performed in the ED were excluded. Input data through artificial intelligence were made up of a combination of clinical, radiological and routine laboratory data upon hospital admission. Different Machine Learning algorithms available on WEKA data mining software and on Semeion Research Centre depository were trained using both the Training and Testing and the K-fold cross-validation protocol.

RESULTS

Among 199 patients subject to study (median [interquartile range] age 65 [46-78] years; 127 [63.8%] men), 124 [62.3%] resulted positive to SARS-COV-2. The best Machine Learning System reached an accuracy of 91.4% with 94.1% sensitivity and 88.7% specificity.

CONCLUSION

Our study suggests that properly trained artificial intelligence algorithms may be able to predict correct results in RT-PCR for SARS-COV-2, using basic clinical data. If confirmed, on a larger-scale study, this approach could have important clinical and organizational implications.

Supervised machine learning for diagnostic classification from large-scale neuroimaging datasets

There are growing concerns about the generalizability of machine learning classifiers in neuroimaging. In order to evaluate this aspect across relatively large heterogeneous populations, we investigated four disorders: Autism spectrum disorder (N = 988), Attention deficit hyperactivity disorder (N = 930), Post-traumatic stress disorder (N = 87) and Alzheimer's disease (N = 132). We applied 18 different machine learning classifiers (based on diverse principles) wherein the training/validation and the hold-out test data belonged to samples with the same diagnosis but differing in either the age range or the acquisition site. Our results indicate that overfitting can be a huge problem in heterogeneous datasets, especially with fewer samples, leading to inflated measures of accuracy that fail to generalize well to the general clinical population. Further, different classifiers tended to perform well on different datasets. In order to address this, we propose a consensus-classifier by combining the predictive power of all 18 classifiers. The consensus-classifier was less sensitive to unmatched training/validation and holdout test data. Finally, we combined feature importance scores obtained from all classifiers to infer the discriminative ability of connectivity features. The functional connectivity patterns thus identified were robust to the classification algorithm used, age and acquisition site differences, and had diagnostic predictive ability in addition to univariate statistically significant group differences between the groups. A MATLAB toolbox called Machine Learning in NeuroImaging (MALINI), which implements all the 18 different classifiers along with the consensus classifier is available from Lanka et al. (2019) The toolbox can also be found at the following URL: https://github.com/pradlanka/malini .

Machine Learning Electronic Health Record Identification of Patients with Rheumatoid Arthritis: Algorithm Pipeline Development and Validation Study

Background

Financial codes are often used to extract diagnoses from electronic health records. This approach is prone to false positives. Alternatively, queries are constructed, but these are highly center and language specific. A tantalizing alternative is the automatic identification of patients by employing machine learning on format-free text entries.

Objective

The aim of this study was to develop an easily implementable workflow that builds a machine learning algorithm capable of accurately identifying patients with rheumatoid arthritis from format-free text fields in electronic health records.

Methods

Two electronic health record data sets were employed: Leiden (n=3000) and Erlangen (n=4771). Using a portion of the Leiden data (n=2000), we compared 6 different machine learning methods and a naïve word-matching algorithm using 10-fold cross-validation. Performances were compared using the area under the receiver operating characteristic curve (AUROC) and the area under the precision recall curve (AUPRC), and F1 score was used as the primary criterion for selecting the best method to build a classifying algorithm. We selected the optimal threshold of positive predictive value for case identification based on the output of the best method in the training data. This validation workflow was subsequently applied to a portion of the Erlangen data (n=4293). For testing, the best performing methods were applied to remaining data (Leiden n=1000; Erlangen n=478) for an unbiased evaluation.

Results

For the Leiden data set, the word-matching algorithm demonstrated mixed performance (AUROC 0.90; AUPRC 0.33; F1 score 0.55), and 4 methods significantly outperformed word-matching, with support vector machines performing best (AUROC 0.98; AUPRC 0.88; F1 score 0.83). Applying this support vector machine classifier to the test data resulted in a similarly high performance (F1 score 0.81; positive predictive value [PPV] 0.94), and with this method, we could identify 2873 patients with rheumatoid arthritis in less than 7 seconds out of the complete collection of 23,300 patients in the Leiden electronic health record system. For the Erlangen data set, gradient boosting performed best (AUROC 0.94; AUPRC 0.85; F1 score 0.82) in the training set, and applied to the test data, resulted once again in good results (F1 score 0.67; PPV 0.97).

Conclusions

We demonstrate that machine learning methods can extract the records of patients with rheumatoid arthritis from electronic health record data with high precision, allowing research on very large populations for limited costs. Our approach is language and center independent and could be applied to any type of diagnosis. We have developed our pipeline into a universally applicable and easy-to-implement workflow to equip centers with their own high-performing algorithm. This allows the creation of observational studies of unprecedented size covering different countries for low cost from already available data in electronic health record systems.

Artificial intelligence facilitates decision-making in the treatment of lumbar disc herniations

PURPOSE

Apart from patients with severe neurological deficits, it is not clear whether surgical or conservative treatment of lumbar disc herniations is superior for the individual patient. We investigated whether deep learning techniques can predict the outcome of patients with lumbar disc herniation after 6 months of treatment.

METHODS

The data of 60 patients were used to train and test a deep learning algorithm with the aim to achieve an accurate prediction of the ODI 6 months after surgery or the start of conservative therapy. We developed an algorithm that predicts the ODI of 6 randomly selected test patients in tenfold cross-validation.

RESULTS

A 100% accurate prediction of an ODI range could be achieved by dividing the ODI scale into 12% sections. A maximum absolute difference of only 3.4% between individually predicted and actual ODI after 6 months of a given therapy was achieved with our most powerful model. The application of artificial intelligence as shown in this work also allowed to compare the actual patient values after 6 months with the prediction for the alternative therapy, showing deviations up to 18.8%.

CONCLUSION

Deep learning in the supervised form applied here can identify patients at an early stage who would benefit from conservative therapy, and on the contrary avoid painful and unnecessary delays for patients who would profit from surgical therapy. In addition, this approach can be used in many other areas of medicine as an effective tool for decision-making when choosing between opposing treatment options, despite small patient groups.

Using supervised machine learning classifiers to estimate likelihood of participating in clinical trials of a de-identified version of ResearchMatch

Introduction:

Lack of participation in clinical trials (CTs) is a major barrier for the evaluation of new pharmaceuticals and devices. Here we report the results of the analysis of a dataset from ResearchMatch, an online clinical registry, using supervised machine learning approaches and a deep learning approach to discover characteristics of individuals more likely to show an interest in participating in CTs.

Methods:

We trained six supervised machine learning classifiers (Logistic Regression (LR), Decision Tree (DT), Gaussian Naïve Bayes (GNB), K-Nearest Neighbor Classifier (KNC), Adaboost Classifier (ABC) and a Random Forest Classifier (RFC)), as well as a deep learning method, Convolutional Neural Network (CNN), using a dataset of 841,377 instances and 20 features, including demographic data, geographic constraints, medical conditions and ResearchMatch visit history. Our outcome variable consisted of responses showing specific participant interest when presented with specific clinical trial opportunity invitations (‘yes’ or ‘no’). Furthermore, we created four subsets from this dataset based on top self-reported medical conditions and gender, which were separately analysed.

Results:

The deep learning model outperformed the machine learning classifiers, achieving an area under the curve (AUC) of 0.8105.

Conclusions:

The results show sufficient evidence that there are meaningful correlations amongst predictor variables and outcome variable in the datasets analysed using the supervised machine learning classifiers. These approaches show promise in identifying individuals who may be more likely to participate when offered an opportunity for a clinical trial.

The detection of hospitalized patients at risk of testing positive to multi-drug resistant bacteria using MOCA-I, a rule-based "white-box" classification algorithm for medical data

BACKGROUND

Multi-drug resistant (MDR) bacteria are a major health concern. In this retrospective study, a rule-based classification algorithm, MOCA-I (Multi-Objective Classification Algorithm for Imbalanced data) is used to identify hospitalized patients at risk of testing positive for multidrug-resistant (MDR) bacteria, including Methicillin-resistant Staphylococcus aureus (MRSA), before or during their stay.

METHODS

Applied to a data set of 48,945 hospital stays (including known cases of carriage) with up to 16,325 attributes per stay, MOCA-I generated alert rules for risk of carriage or infection. A risk score was then computed from each stay according to the triggered rules.Recall and precision curves were plotted.

RESULTS

The classification can be focused on specifically detecting high risk of having a positive test, or identifying large numbers of at-risk patients by modulating the risk score cut-off level. For a risk score above 0.85,recall (sensitivity) is 62 % with 69 % precision (confidence) for MDR bacteria, recall is 58 % with 88 % precision for MRSA. In addition, MOCA-I identifies 38 and 21 cases of previously unknown MDR and MRSA respectively.

CONCLUSIONS

MOCA-I generates medically pertinent alert rules. This classification algorithm can be used to detect patients with high risk of testing positive to MDR bacteria (including MRSA). Classification can be modulated by appropriately setting the risk score cut-off level to favor specific detection of small numbers of patients at very high risk or identification of large numbers of patients at risk. MOCA-I can thus contribute to more adapted treatments and preventive measures from admission, depending on the clinical setting or management strategy.

Machine learning for predicting long-term kidney allograft survival: a scoping review

Supervised machine learning (ML) is a class of algorithms that "learn" from existing input-output pairs, which is gaining popularity in pattern recognition for classification and prediction problems. In this scoping review, we examined the use of supervised ML algorithms for the prediction of long-term allograft survival in kidney transplant recipients. Data sources included PubMed, the Cumulative Index to Nursing and Allied Health Literature, and the Institute for Electrical and Electronics Engineers (IEEE) Xplore libraries from inception to November 2019. We screened titles and abstracts and potentially eligible full-text reports to select studies and subsequently abstracted the data. Eleven studies were identified. Decision trees were the most commonly used method (n = 8), followed by artificial neural networks (ANN) (n = 4) and Bayesian belief networks (n = 2). The area under receiver operating curve (AUC) was the most common measure of discrimination (n = 7), followed by sensitivity (n = 5) and specificity (n = 4). Model calibration examining the reliability in risk prediction was performed using either the Pearson r or the Hosmer-Lemeshow test in four studies. One study showed that logistic regression had comparable performance to ANN, while another study demonstrated that ANN performed better in terms of sensitivity, specificity, and accuracy, as compared with a Cox proportional hazards model. We synthesized the evidence related to the comparison of ML techniques with traditional statistical approaches for prediction of long-term allograft survival in patients with a kidney transplant. The methodological and reporting quality of included studies was poor. Our study also demonstrated mixed results in terms of the predictive potential of the models.

Monitoring functional status using a wearable real-time locating technology

Sensor technologies enable real-time, continuous, and objective monitoring of activity and functioning in later life. In long-term care, timely assessment of functional status is needed to prevent falls and other acute events. However, the electronic forms and paper and pencil tools currently used are time-consuming and conducted too infrequently (e.g., every 6 months) to provide the sensitivity and specificity required. Staff are also unable to detect subtle changes in functioning through observation alone. The purpose of this paper is to discuss the use of a wearable real-time locating system that utilizes ultra wideband radio technology to continuously and objectively measure activity and aspects of functional status. This paper discusses the associated conceptualization and development of the scoring algorithms, raw data transformation, use of traditional paper and pencil tools and electronic health record data to validate sensor data, and other tips for those interested in this type of wearable sensor technology.

Advanced procedures for skull sex estimation using sexually dimorphic morphometric features

This paper introduces an automated method for estimating sex from cranial sex diagnostic traits by extracting and evaluating specialized morphometric features from the glabella, the supraorbital ridge, the occipital protuberance, and the mastoid process. The proposed method was developed and evaluated using two European population samples, a Czech sample comprising 170 crania reconstructed from anonymized CT scans and a Greek sample of 156 crania from the Athens Collection. It is based on a fully automatic algorithm applied on 3D models for extracting sex diagnostic morphometric features which are further processed by computer vision and machine learning algorithms. Classification accuracy was evaluated in a population specific and a population generic 2-way cross-validation scheme. Population-specific accuracy for individual morphometric features ranged from 78.5 to 96.7%, whereas population generic correct classification ranged from 71.7 to 90.8%. Combining all sex diagnostic traits in multi-feature sex estimation yielded correct classification performance in excess of 91% for the entire sample, whereas the sex of about three fourths of the sample could be determined with 100% accuracy according to posterior probability estimates. The proposed method provides an efficient and reliable way to estimate sex from cranial remains, and it offers significant advantages over existing methods. The proposed method can be readily implemented with the skullanalyzer computer program and the estimate_sex.m GNU Octave function, which are freely available under a suitable license.

Quantitative enhancement thresholds and machine learning algorithms for the evaluation of renal lesions using single-phase split-filter dual-energy CT

PURPOSE

To establish thresholds for contrast enhancement-based attenuation (CM) and iodine concentration (IOD) for the quantitative evaluation of enhancement in renal lesions on single-phase split-filter dual-energy CT (tbDECT) and combine measurements in a machine learning algorithm to potentially improve performance.

MATERIAL

126 patients with incidental renal cysts (both hypo- and hyperdense cysts) or high suspicion for renal cell carcinoma (312 total lesions) undergoing abdominal, portal venous phase tbDECT were initially included in this retrospective study. Gold standard was pathological confirmation or follow-up imaging (MRI or multiphasic CT). CM, IOD, and ROI size were recorded. Thresholds for CM and IOD were identified using Youden-Index of the empirical ROC curves. Decision tree (DTC) and random forest classifier (RFC) were trained. Sensitivities, specificities, and AUCs were compared using McNemar and DeLong test.

RESULTS

The final study cohort comprised 40 enhancing and 113 non-enhancing renal lesions. Optimal thresholds for quantitative iodine measurements and contrast enhancement-based attenuation were 1.0 ± 0.0 mg/ml and 23.6 ± 0.3 HU, respectively. Single DECT parameters (IOD, CM) showed similar overall performance with an AUC of 0.894 and 0.858 (p = 0.541) (sensitivity 90 and 80%, specificity 88 and 92%, respectively). While overall performance for the DTC (AUC 0.944) was higher than RFC (AUC 0.886), this difference (p = 0.409) and comparison to CM (p = 0.243) and IOD (p = 0.353) was not statistically significant.

CONCLUSIONS

Enhancement in incidental renal lesions on single-phase tbDECT can be classified with up to 87.5% sensitivity and 94.6% specificity. Algorithms combining DECT parameters did not increase overall performance.

The interplay between multisensory integration and perceptual decision making

Facing perceptual uncertainty, the brain combines information from different senses to make optimal perceptual decisions and to guide behavior. However, decision making has been investigated mostly in unimodal contexts. Thus, how the brain integrates multisensory information during decision making is still unclear. Two opposing, but not mutually exclusive, scenarios are plausible: either the brain thoroughly combines the signals from different modalities before starting to build a supramodal decision, or unimodal signals are integrated during decision formation. To answer this question, we devised a paradigm mimicking naturalistic situations where human participants were exposed to continuous cacophonous audiovisual inputs containing an unpredictable signal cue in one or two modalities and had to perform a signal detection task or a cue categorization task. First, model-based analyses of behavioral data indicated that multisensory integration takes place alongside perceptual decision making. Next, using supervised machine learning on concurrently recorded EEG, we identified neural signatures of two processing stages: sensory encoding and decision formation. Generalization analyses across experimental conditions and time revealed that multisensory cues were processed faster during both stages. We further established that acceleration of neural dynamics during sensory encoding and decision formation was directly linked to multisensory integration. Our results were consistent across both signal detection and categorization tasks. Taken together, the results revealed a continuous dynamic interplay between multisensory integration and decision making processes (mixed scenario), with integration of multimodal information taking place both during sensory encoding as well as decision formation.

Identification of elders at higher risk for fall with statewide electronic health records and a machine learning algorithm

OBJECTIVE

Predicting the risk of falls in advance can benefit the quality of care and potentially reduce mortality and morbidity in the older population. The aim of this study was to construct and validate an electronic health record-based fall risk predictive tool to identify elders at a higher risk of falls.

METHODS

The one-year fall prediction model was developed using the machine-learning-based algorithm, XGBoost, and tested on an independent validation cohort. The data were collected from electronic health records (EHR) of Maine from 2016 to 2018, comprising 265,225 older patients (≥65 years of age).

RESULTS

This model attained a validated C-statistic of 0.807, where 50 % of the identified high-risk true positives were confirmed to fall during the first 94 days of next year. The model also captured in advance 58.01 % and 54.93 % of falls that happened within the first 30 and 30-60 days of next year. The identified high-risk patients of fall showed conditions of severe disease comorbidities, an enrichment of fall-increasing cardiovascular and mental medication prescriptions and increased historical clinical utilization, revealing the complexity of the underlying fall etiology. The XGBoost algorithm captured 157 impactful predictors into the final predictive model, where cognitive disorders, abnormalities of gait and balance, Parkinson's disease, fall history and osteoporosis were identified as the top-5 strongest predictors of the future fall event.

CONCLUSIONS

By using the EHR data, this risk assessment tool attained an improved discriminative ability and can be immediately deployed in the health system to provide automatic early warnings to older adults with increased fall risk and identify their personalized risk factors to facilitate customized fall interventions.

Accurate Inference of Tree Topologies from Multiple Sequence Alignments Using Deep Learning

Reconstructing the phylogenetic relationships between species is one of the most formidable tasks in evolutionary biology. Multiple methods exist to reconstruct phylogenetic trees, each with their own strengths and weaknesses. Both simulation and empirical studies have identified several "zones" of parameter space where accuracy of some methods can plummet, even for four-taxon trees. Further, some methods can have undesirable statistical properties such as statistical inconsistency and/or the tendency to be positively misleading (i.e. assert strong support for the incorrect tree topology). Recently, deep learning techniques have made inroads on a number of both new and longstanding problems in biological research. In this study, we designed a deep convolutional neural network (CNN) to infer quartet topologies from multiple sequence alignments. This CNN can readily be trained to make inferences using both gapped and ungapped data. We show that our approach is highly accurate on simulated data, often outperforming traditional methods, and is remarkably robust to bias-inducing regions of parameter space such as the Felsenstein zone and the Farris zone. We also demonstrate that the confidence scores produced by our CNN can more accurately assess support for the chosen topology than bootstrap and posterior probability scores from traditional methods. Although numerous practical challenges remain, these findings suggest that the deep learning approaches such as ours have the potential to produce more accurate phylogenetic inferences.

MALINI (Machine Learning in NeuroImaging): A MATLAB toolbox for aiding clinical diagnostics using resting-state fMRI data

Resting-state functional Magnetic Resonance Imaging (rs-fMRI) has been extensively used for diagnostic classification because it does not require task compliance and is easier to pool data from multiple imaging sites, thereby increasing the sample size. A MATLAB-based toolbox called Machine Learning in NeuroImaging (MALINI) for feature extraction and disease classification is presented. The MALINI toolbox extracts functional and effective connectivity features from preprocessed rs-fMRI data and performs classification between healthy and disease groups using any of 18 popular and widely used machine learning algorithms that are based on diverse principles. A consensus classifier combining the power of multiple classifiers is also presented. The utility of the toolbox is illustrated by accompanying data consisting of resting-state functional connectivity features from healthy controls and subjects with various brain-based disorders: autism spectrum disorder from autism brain imaging data exchange (ABIDE), Alzheimer's disease and mild cognitive impairment from Alzheimer's disease neuroimaging initiative (ADNI), attention deficit hyperactivity disorder from ADHD-200, and post-traumatic stress disorder and post-concussion syndrome acquired in-house. Results of classification performed on the above datasets can be obtained from the main article titled “Supervised machine learning for diagnostic classification from large-scale neuroimaging datasets” [1]. The data was divided into homogeneous and heterogeneous splits, such that 80% could be used for training, model building and cross-validation, while the remaining 20% of the data could be used as a hold-out independent test data for replication of the classification performance, to ensure the robustness of the classifiers to population variance in image acquisition site and age of the sample.

ImaGene: a convolutional neural network to quantify natural selection from genomic data

BACKGROUND

The genetic bases of many complex phenotypes are still largely unknown, mostly due to the polygenic nature of the traits and the small effect of each associated mutation. An alternative approach to classic association studies to determining such genetic bases is an evolutionary framework. As sites targeted by natural selection are likely to harbor important functionalities for the carrier, the identification of selection signatures in the genome has the potential to unveil the genetic mechanisms underpinning human phenotypes. Popular methods of detecting such signals rely on compressing genomic information into summary statistics, resulting in the loss of information. Furthermore, few methods are able to quantify the strength of selection. Here we explored the use of deep learning in evolutionary biology and implemented a program, called ImaGene, to apply convolutional neural networks on population genomic data for the detection and quantification of natural selection.

RESULTS

ImaGene enables genomic information from multiple individuals to be represented as abstract images. Each image is created by stacking aligned genomic data and encoding distinct alleles into separate colors. To detect and quantify signatures of positive selection, ImaGene implements a convolutional neural network which is trained using simulations. We show how the method implemented in ImaGene can be affected by data manipulation and learning strategies. In particular, we show how sorting images by row and column leads to accurate predictions. We also demonstrate how the misspecification of the correct demographic model for producing training data can influence the quantification of positive selection. We finally illustrate an approach to estimate the selection coefficient, a continuous variable, using multiclass classification techniques.

CONCLUSIONS

While the use of deep learning in evolutionary genomics is in its infancy, here we demonstrated its potential to detect informative patterns from large-scale genomic data. We implemented methods to process genomic data for deep learning in a user-friendly program called ImaGene. The joint inference of the evolutionary history of mutations and their functional impact will facilitate mapping studies and provide novel insights into the molecular mechanisms associated with human phenotypes.

Detection of malaria parasites in dried human blood spots using mid-infrared spectroscopy and logistic regression analysis

BACKGROUND

Epidemiological surveys of malaria currently rely on microscopy, polymerase chain reaction assays (PCR) or rapid diagnostic test kits for Plasmodium infections (RDTs). This study investigated whether mid-infrared (MIR) spectroscopy coupled with supervised machine learning could constitute an alternative method for rapid malaria screening, directly from dried human blood spots.

METHODS

Filter papers containing dried blood spots (DBS) were obtained from a cross-sectional malaria survey in 12 wards in southeastern Tanzania in 2018/19. The DBS were scanned using attenuated total reflection-Fourier Transform Infrared (ATR-FTIR) spectrometer to obtain high-resolution MIR spectra in the range 4000 cm-1 to 500 cm-1. The spectra were cleaned to compensate for atmospheric water vapour and CO2 interference bands and used to train different classification algorithms to distinguish between malaria-positive and malaria-negative DBS papers based on PCR test results as reference. The analysis considered 296 individuals, including 123 PCR-confirmed malaria positives and 173 negatives. Model training was done using 80% of the dataset, after which the best-fitting model was optimized by bootstrapping of 80/20 train/test-stratified splits. The trained models were evaluated by predicting Plasmodium falciparum positivity in the 20% validation set of DBS.

RESULTS

Logistic regression was the best-performing model. Considering PCR as reference, the models attained overall accuracies of 92% for predicting P. falciparum infections (specificity = 91.7%; sensitivity = 92.8%) and 85% for predicting mixed infections of P. falciparum and Plasmodium ovale (specificity = 85%, sensitivity = 85%) in the field-collected specimen.

CONCLUSION

These results demonstrate that mid-infrared spectroscopy coupled with supervised machine learning (MIR-ML) could be used to screen for malaria parasites in human DBS. The approach could have potential for rapid and high-throughput screening of Plasmodium in both non-clinical settings (e.g., field surveys) and clinical settings (diagnosis to aid case management). However, before the approach can be used, we need additional field validation in other study sites with different parasite populations, and in-depth evaluation of the biological basis of the MIR signals. Improving the classification algorithms, and model training on larger datasets could also improve specificity and sensitivity. The MIR-ML spectroscopy system is physically robust, low-cost, and requires minimum maintenance.

Tissue diagnosis during colorectal cancer surgery using optical sensing: an in vivo study

Background

In colorectal cancer surgery there is a delicate balance between complete removal of the tumor and sparing as much healthy tissue as possible. Especially in rectal cancer, intraoperative tissue recognition could be of great benefit in preventing positive resection margins and sparing as much healthy tissue as possible. To better guide the surgeon, we evaluated the accuracy of diffuse reflectance spectroscopy (DRS) for tissue characterization during colorectal cancer surgery and determined the added value of DRS when compared to clinical judgement.

Methods

DRS spectra were obtained from fat, healthy colorectal wall and tumor tissue during colorectal cancer surgery and results were compared to histopathology examination of the measurement locations. All spectra were first normalized at 800 nm, thereafter two support vector machines (SVM) were trained using a tenfold cross-validation. With the first SVM fat was separated from healthy colorectal wall and tumor tissue, the second SVM distinguished healthy colorectal wall from tumor tissue.

Results

Patients were included based on preoperative imaging, indicating advanced local stage colorectal cancer. Based on the measurement results of 32 patients, the classification resulted in a mean accuracy for fat, healthy colorectal wall and tumor of 0.92, 0.89 and 0.95 respectively. If the classification threshold was adjusted such that no false negatives were allowed, the percentage of false positive measurement locations by DRS was 25% compared to 69% by clinical judgement.

Conclusion

This study shows the potential of DRS for the use of tissue classification during colorectal cancer surgery. Especially the low false positive rate obtained for a false negative rate of zero shows the added value for the surgeons.

Trail registration This trail was performed under approval from the internal review board committee (Dutch Trail Register NTR5315), registered on 04/13/2015, https://www.trialregister.nl/trial/5175.

Utilizing supervised machine learning to identify microglia and astrocytes in situ: implications for large-scale image analysis and quantification

BACKGROUND

The evaluation of histological tissue samples plays a crucial role in deciphering preclinical disease and injury mechanisms. High-resolution images can be obtained quickly however data acquisition are often bottlenecked by manual analysis methodologies.

NEW METHOD

We describe and validate a pipeline for a novel machine learning-based analytical method, using the Opera High-Content Screening system and Harmony software, allowing for detailed image analysis of cellular markers in histological samples.

RESULTS

To validate the machine learning pipeline, analyses of single proteins in mouse brain sections were utilized. To demonstrate adaptability of the pipeline for multiple cell types and epitopes, the percent brain coverage of microglial cells, identified by ionized calcium binding adaptors molecule 1 (Iba1), and of astrocytes, by glial fibrillary acidic protein (GFAP) demonstrated no significant differences between automated and manual analyses protocols. Further to examine the robustness of this protocol for multiple proteins simultaneously labeling of rat brain sections were utilized; co-localization of astrocytic endfeet on blood vessels, using aquaporin-4 and tomato lectin respectively, were efficiently identified and quantified by the novel pipeline and were not significantly different between the two analyses protocols. Comparison with Existing Methods: The automated platform maintained the sensitivity and accuracy of manual analysis, while accomplishing the analyses in 1/200^th of the time.

CONCLUSIONS

We demonstrate the benefits and potential of adapting an automated high-throughput machine-learning analytical approach for the analysis ofin situ tissue samples, show effectiveness across different animal models, while reducing analysis time and increasing productivity.

Prediction of early colorectal cancer metastasis by machine learning using digital slide images

BACKGROUND AND OBJECTIVES

Prediction of lymph node metastasis (LNM) for early colorectal cancer (CRC) is critical for determining treatment strategies after endoscopic resection. Some histologic parameters for predicting LNM have been established, but evaluator error and inter-observer disagreement are unsolved issues. Here we describe an LNM prediction algorithm for submucosal invasive (T1) CRC based on machine learning.

METHODS

We conducted a retrospective single-institution study of 397 T1 CRCs. Several morphologic parameters were extracted from whole slide images of cytokeratin immunohistochemistry using Image J. A random forest algorithm for a training dataset (n = 277) was executed and used to predict LNM for the test dataset (n = 120). The results were compared with conventional histologic evaluation of hematoxylin-eosin staining.

RESULTS

Machine learning showed better LNM predictive ability than the conventional method on some datasets. Cross validation revealed no significant difference between the methods. Machine learning resulted in fewer false-negative cases than the conventional method.

CONCLUSIONS

Machine learning on whole slide images is a potential alternative for determining treatment strategies for T1 CRC.