From concept drift to model degradation: An overview on performance-aware drift detectors

Temporally boosting neural network for improving dynamic prediction of PM2.5 concentration with changing and unbalanced distribution

Increasing medical research evidence suggests that even low PM2.5 concentrations may trigger significant health issues. Hence, an accurate prediction of PM2.5 holds immense significance in securing public health safety. However, current data-drive predictive methods exhibit seasonal model performance decline and difficulties in predicting extremely high values. Those issues may stem from neglecting two crucial features in PM2.5 data streams, i.e., concept drift and imbalanced distribution. In this study, we validate this hypothesis by conducting an in-depth analysis of the characteristics of the PM2.5 data stream and the prediction errors of three mainstream models trained on this PM2.5 data stream, i.e., random forest, convolutional neural network and transformer. Based on the identified types of concept drift and the patterns of imbalanced distribution, we introduce the Temporally boosting neural network (Temp-boost), a novel ensemble learning method designed to enhance predictive accuracy by integrating static and dynamic models. Static models, which are trained on balanced historical datasets, typically receive infrequent updates. Conversely, dynamic models are trained on newly arrived data and undergo more frequent updates. We evaluated the performance of Temp-boost and the three mentioned models in predicting gridded PM2.5 concentrations across the North China Plain in 2019. Compared to the three models, the Temp-boost shows improved prediction accuracy for different seasons, with notable enhancements in high-pollution levels. Specifically, for pollution levels above lightly polluted, the Temp-boost effectively reduces the average MAE by 13.22 μgm-3, RMSE by 13.32 μgm-3 , with reductions peaking MAE at 26.45 μgm-3,RMSE at 25.76 μgm-3 in more severe case.

Explanation and elaboration of MedinAI: guidelines for reporting artificial intelligence studies in medicines, pharmacotherapy, and pharmaceutical services

The increasing adoption of artificial intelligence (AI) in medicines, pharmacotherapy, and pharmaceutical services necessitates clear guidance on reporting standards. While the MedinAI Statement (Bottacin in Int J Clin Pharm, https://doi.org/10.1007/s11096-025-01905-3, 2025) provides core guidelines for reporting AI studies in these fields, detailed explanations and practical examples are crucial for optimal implementation. This companion document was developed to offer comprehensive guidance and real-world examples for each guideline item. The document elaborates on all 14 items and 78 sub-items across four domains: core, ethical considerations in medication and pharmacotherapy, medicines as products, and services related to medicines and pharmacotherapy. Through clear, actionable guidance and diverse examples, this document enhances MedinAI's utility, enabling researchers and stakeholders to improve the quality and transparency of AI research reporting across various contexts, study designs, and development stages.

A review of urban heat island mapping approaches with a special emphasis on the Indian region

With an estimated population of about 6 billion living in urban areas by 2050, controlling the rate of urbanization is one of the major challenges governments face in the twenty-first century. Increasing urbanization is problematic since it causes heat accumulation within urban areas due to the enhanced anthropogenic activities. With developing countries like India aiming for their economic growth, combating urban heat islands (UHIs) is a critical step in improving the quality of life. Efficient mitigation of UHI effects requires accurate mapping and monitoring, which can be enhanced through the use of machine learning and deep learning algorithms. This paper provides a critical review of UHI mapping approaches employed globally, with a particular emphasis on the Indian region along with a focus on AI-based methods. Key issues, challenges, and future research directions are also discussed.

Intra- and inter-channel deep convolutional neural network with dynamic label smoothing for multichannel biosignal analysis

Efficient processing of multichannel biosignals has significant application values in the fields of healthcare and human-machine interaction. Although previous research has achieved high recognition performance with deep convolutional neural networks, several key challenges still remain: (1) Effective extraction of spatial and temporal features from the multichannel biosignals. (2) Appropriate trade-off between performance and complexity for improving applicability in real-life situations given that traditional machine learning and 2D-based CNN approaches often involve excessive preprocessing steps or model parameters; and (3) Generalization ability of neural networks to compensate for domain difference and to reduce overfitting during training process. To address challenges 1 and 2, we propose a 1D-based deep intra and inter channel (I2C) convolution neural network. The I2C convolutional block is introduced to replace the standard convolutional layer, further extending it to several state-of-the-art modules, with the intent of extracting more effective features from multichannel biosignals with fewer parameters. To address challenge 3, we integrate a branch model into the main model to perform dynamic label smoothing, enabling the model to learn domain difference and improve its generalization ability. Experiments were conducted on three public multichannel biosignals databases, namely ISRUC-S3, HEF and Ninapro-DB1. The results suggest that the proposed method exhibits significant competitive advantages in accuracy, complexity, and generalization ability.

Predicting Inpatient Admissions From Emergency Department Triage Using Machine Learning: A Systematic Review

This study aimed to evaluate the quality of evidence for using machine learning models to predict inpatient admissions from emergency department triage data, ultimately aiming to improve patient flow management. A comprehensive literature search was conducted according to the PRISMA guidelines across 5 databases, PubMed, Embase, Web of Science, Scopus, and CINAHL, on August 1, 2024, for English-language studies published between August 1, 2014, and August 1, 2024. This yielded 700 articles, of which 66 were screened in full, and 31 met the inclusion and exclusion criteria. Model quality was assessed using the PROBAST appraisal tool and a modified TRIPOD+AI framework, alongside reported model performance metrics. Seven studies demonstrated rigorous methodology and promising in silico performance, with an area under the receiver operating characteristic ranging from 0.81 to 0.93. However, further performance analysis was limited by heterogeneity in model development and an unclear-to-high risk of bias and applicability concerns in the remaining 24 articles, as evaluated by the PROBAST tool. The current literature demonstrates a good degree of in silico accuracy in predicting inpatient admission from triage data alone. Future research should emphasize transparent model development and reporting, temporal validation, concept drift analysis, exploration of emerging artificial intelligence techniques, and analysis of real-world patient flow metrics to comprehensively assess the usefulness of these models.

Derivation and validation of a clinical predictive model for longer duration diarrhea among pediatric patients in Kenya using machine learning algorithms

BACKGROUND

Despite the adverse health outcomes associated with longer duration diarrhea (LDD), there are currently no clinical decision tools for timely identification and better management of children with increased risk. This study utilizes machine learning (ML) to derive and validate a predictive model for LDD among children presenting with diarrhea to health facilities.

METHODS

LDD was defined as a diarrhea episode lasting ≥ 7 days. We used 7 ML algorithms to build prognostic models for the prediction of LDD among children < 5 years using de-identified data from Vaccine Impact on Diarrhea in Africa study (N = 1,482) in model development and data from Enterics for Global Health Shigella study (N = 682) in temporal validation of the champion model. Features included demographic, medical history and clinical examination data collected at enrolment in both studies. We conducted split-sampling and employed K-fold cross-validation with over-sampling technique in the model development. Moreover, critical predictors of LDD and their impact on prediction were obtained using an explainable model agnostic approach. The champion model was determined based on the area under the curve (AUC) metric. Model calibrations were assessed using Brier, Spiegelhalter's z-test and its accompanying p-value.

RESULTS

There was a significant difference in prevalence of LDD between the development and temporal validation cohorts (478 [32.3%] vs 69 [10.1%]; p < 0.001). The following variables were associated with LDD in decreasing order: pre-enrolment diarrhea days (55.1%), modified Vesikari score(18.2%), age group (10.7%), vomit days (8.8%), respiratory rate (6.5%), vomiting (6.4%), vomit frequency (6.2%), rotavirus vaccination (6.1%), skin pinch (2.4%) and stool frequency (2.4%). While all models showed good prediction capability, the random forest model achieved the best performance (AUC [95% Confidence Interval]: 83.0 [78.6-87.5] and 71.0 [62.5-79.4]) on the development and temporal validation datasets, respectively. While the random forest model showed slight deviations from perfect calibration, these deviations were not statistically significant (Brier score = 0.17, Spiegelhalter p-value = 0.219).

CONCLUSIONS

Our study suggests ML derived algorithms could be used to rapidly identify children at increased risk of LDD. Integrating ML derived models into clinical decision-making may allow clinicians to target these children with closer observation and enhanced management.

MoCab: A framework for the deployment of machine learning models across health information systems

BACKGROUND AND OBJECTIVE

Machine learning models are vital for enhancing healthcare services. However, integrating them into health information systems (HISs) introduces challenges beyond clinical decision making, such as interoperability and diverse electronic health records (EHR) formats. We proposed Model Cabinet Architecture (MoCab), a framework designed to leverage fast healthcare interoperability resources (FHIR) as the standard for data storage and retrieval when deploying machine learning models across various HISs, addressing the challenges highlighted by platforms such as EPOCH®, ePRISM®, KETOS, and others.

METHODS

The MoCab architecture is designed to streamline predictive modeling in healthcare through a structured framework incorporating several specialized parts. The Data Service Center manages patient data retrieval from FHIR servers. These data are then processed by the Knowledge Model Center, where they are formatted and fed into predictive models. The Model Retraining Center is crucial in continuously updating these models to maintain accuracy in dynamic clinical environments. The framework further incorporates Clinical Decision Support (CDS) Hooks for issuing clinical alerts. It uses Substitutable Medical Apps Reusable Technologies (SMART) on FHIR to develop applications for displaying alerts, prediction results, and patient records.

RESULTS

The MoCab framework was demonstrated using three types of predictive models: a scoring model (qCSI), a machine learning model (NSTI), and a deep learning model (SPC), applied to synthetic data that mimic a major EHR system. The implementations showed how MoCab integrates predictive models with health data for clinical decision support, utilizing CDS Hooks and SMART on FHIR for seamless HIS integration. The demonstration confirmed the practical utility of MoCab in supporting clinical decision making, validated by its application in various healthcare settings.

CONCLUSIONS

We demonstrate MoCab's potential in promoting the interoperability of machine learning models and enhancing its utility across various EHRs. Despite facing challenges like FHIR adoption, MoCab addresses key challenges in adapting machine learning models within healthcare settings, paving the way for further enhancements and broader adoption.

Enhancing Emergency Department Management: A Data-Driven Approach to Detect and Predict Surge Persistence

The prediction of patient attendance in emergency departments (ED) is crucial for effective healthcare planning and resource allocation. This paper proposes an early warning system that can detect emerging trends in ED attendance, offering timely alerts for proactive operational planning. Over 13 years of historical ED attendance data (from January 2010 till December 2022) with 1,700,887 data points were used to develop and validate: (1) a Seasonal Autoregressive Integrated Moving Average with eXogenous factors (SARIMAX) forecasting model; (2) an Exponentially Weighted Moving Average (EWMA) surge prediction model, and (3) a trend persistence prediction model. Drift detection was achieved with the EWMA control chart, and the slopes of a kernel-regressed ED attendance curve were used to train various machine learning (ML) models to predict trend persistence. The EWMA control chart effectively detected significant COVID-19 events in Singapore. The surge prediction model generated preemptive signals on changes in the trends of ED attendance over the COVID-19 pandemic period from January 2020 until December 2022. The persistence of novel trends was further estimated using the trend persistence model, with a mean absolute error of 7.54 (95% CI: 6.77-8.79) days. This study advanced emergency healthcare management by introducing a proactive surge detection framework, which is vital for bolstering the preparedness and agility of emergency departments amid unforeseen health crises.

Unsupervised Domain Adaptation for Inter-Session Re-Calibration of Ultrasound-Based HMIs

Human-Machine Interfaces (HMIs) have gained popularity as they allow for an effortless and natural interaction between the user and the machine by processing information gathered from a single or multiple sensing modalities and transcribing user intentions to the desired actions. Their operability depends on frequent periodic re-calibration using newly acquired data due to their adaptation needs in dynamic environments, where test-time data continuously change in unforeseen ways, a cause that significantly contributes to their abandonment and remains unexplored by the Ultrasound-based (US-based) HMI community. In this work, we conduct a thorough investigation of Unsupervised Domain Adaptation (UDA) algorithms for the re-calibration of US-based HMIs during within-day sessions, which utilize unlabeled data for re-calibration. Our experimentation led us to the proposal of a CNN-based architecture for simultaneous wrist rotation angle and finger gesture prediction that achieves comparable performance with the state-of-the-art while featuring 87.92% less trainable parameters. According to our findings, DANN (a Domain-Adversarial training algorithm), with proper initialization, offers an average 24.99% classification accuracy performance enhancement when compared to no re-calibration setting. However, our results suggest that in cases where the experimental setup and the UDA configuration may differ, observed enhancements would be rather small or even unnoticeable.

Susceptibility of AutoML mortality prediction algorithms to model drift caused by the COVID pandemic

BACKGROUND

Concept drift and covariate shift lead to a degradation of machine learning (ML) models. The objective of our study was to characterize sudden data drift as caused by the COVID pandemic. Furthermore, we investigated the suitability of certain methods in model training to prevent model degradation caused by data drift.

METHODS

We trained different ML models with the H2O AutoML method on a dataset comprising 102,666 cases of surgical patients collected in the years 2014-2019 to predict postoperative mortality using preoperatively available data. Models applied were Generalized Linear Model with regularization, Default Random Forest, Gradient Boosting Machine, eXtreme Gradient Boosting, Deep Learning and Stacked Ensembles comprising all base models. Further, we modified the original models by applying three different methods when training on the original pre-pandemic dataset: (Rahmani K, et al, Int J Med Inform 173:104930, 2023) we weighted older data weaker, (Morger A, et al, Sci Rep 12:7244, 2022) used only the most recent data for model training and (Dilmegani C, 2023) performed a z-transformation of the numerical input parameters. Afterwards, we tested model performance on a pre-pandemic and an in-pandemic data set not used in the training process, and analysed common features.

RESULTS

The models produced showed excellent areas under receiver-operating characteristic and acceptable precision-recall curves when tested on a dataset from January-March 2020, but significant degradation when tested on a dataset collected in the first wave of the COVID pandemic from April-May 2020. When comparing the probability distributions of the input parameters, significant differences between pre-pandemic and in-pandemic data were found. The endpoint of our models, in-hospital mortality after surgery, did not differ significantly between pre- and in-pandemic data and was about 1% in each case. However, the models varied considerably in the composition of their input parameters. None of our applied modifications prevented a loss of performance, although very different models emerged from it, using a large variety of parameters.

CONCLUSIONS

Our results show that none of our tested easy-to-implement measures in model training can prevent deterioration in the case of sudden external events. Therefore, we conclude that, in the presence of concept drift and covariate shift, close monitoring and critical review of model predictions are necessary.

LSTMDD: an optimized LSTM-based drift detector for concept drift in dynamic cloud computing

This study aims to investigate the problem of concept drift in cloud computing and emphasizes the importance of early detection for enabling optimum resource utilization and offering an effective solution. The analysis includes synthetic and real-world cloud datasets, stressing the need for appropriate drift detectors tailored to the cloud domain. A modified version of Long Short-Term Memory (LSTM) called the LSTM Drift Detector (LSTMDD) is proposed and compared with other top drift detection techniques using prediction error as the primary evaluation metric. LSTMDD is optimized to improve performance in detecting anomalies in non-Gaussian distributed cloud environments. The experiments show that LSTMDD outperforms other methods for gradual and sudden drift in the cloud domain. The findings suggest that machine learning techniques such as LSTMDD could be a promising approach to addressing the problem of concept drift in cloud computing, leading to more efficient resource allocation and improved performance.

ELM-KL-LSTM: a robust and general incremental learning method for efficient classification of time series data

Efficiently analyzing and classifying dynamically changing time series data remains a challenge. The main issue lies in the significant differences in feature distribution that occur between old and new datasets generated constantly due to varying degrees of concept drift, anomalous data, erroneous data, high noise, and other factors. Taking into account the need to balance accuracy and efficiency when the distribution of the dataset changes, we proposed a new robust, generalized incremental learning (IL) model ELM-KL-LSTM. Extreme learning machine (ELM) is used as a lightweight pre-processing model which is updated using the new designed evaluation metrics based on Kullback-Leibler (KL) divergence values to measure the difference in feature distribution within sliding windows. Finally, we implemented efficient processing and classification analysis of dynamically changing time series data based on ELM lightweight pre-processing model, model update strategy and long short-term memory networks (LSTM) classification model. We conducted extensive experiments and comparation analysis based on the proposed method and benchmark methods in several different real application scenarios. Experimental results show that, compared with the benchmark methods, the proposed method exhibits good robustness and generalization in a number of different real-world application scenarios, and can successfully perform model updates and efficient classification analysis of incremental data with varying degrees improvement of classification accuracy. This provides and extends a new means for efficient analysis of dynamically changing time-series data.

Data drift in medical machine learning: implications and potential remedies

Data drift refers to differences between the data used in training a machine learning (ML) model and that applied to the model in real-world operation. Medical ML systems can be exposed to various forms of data drift, including differences between the data sampled for training and used in clinical operation, differences between medical practices or context of use between training and clinical use, and time-related changes in patient populations, disease patterns, and data acquisition, to name a few. In this article, we first review the terminology used in ML literature related to data drift, define distinct types of drift, and discuss in detail potential causes within the context of medical applications with an emphasis on medical imaging. We then review the recent literature regarding the effects of data drift on medical ML systems, which overwhelmingly show that data drift can be a major cause for performance deterioration. We then discuss methods for monitoring data drift and mitigating its effects with an emphasis on pre- and post-deployment techniques. Some of the potential methods for drift detection and issues around model retraining when drift is detected are included. Based on our review, we find that data drift is a major concern in medical ML deployment and that more research is needed so that ML models can identify drift early, incorporate effective mitigation strategies and resist performance decay.

Digital Transformation of Cancer Care in the Era of Big Data, Artificial Intelligence and Data-Driven Interventions: Navigating the Field

OBJECTIVES

To navigate the field of digital cancer care and define and discuss key aspects and applications of big data analytics, artificial intelligence (AI), and data-driven interventions.

DATA SOURCES

Peer-reviewed scientific publications and expert opinion.

CONCLUSION

The digital transformation of cancer care, enabled by big data analytics, AI, and data-driven interventions, presents a significant opportunity to revolutionize the field. An increased understanding of the lifecycle and ethics of data-driven interventions will enhance development of innovative and applicable products to advance digital cancer care services.

IMPLICATIONS FOR NURSING PRACTICE

As digital technologies become integrated into cancer care, nurse practitioners and scientists will be required to increase their knowledge and skills to effectively use these tools to the patient's benefit. An enhanced understanding of the core concepts of AI and big data, confident use of digital health platforms, and ability to interpret the outputs of data-driven interventions are key competencies. Nurses in oncology will play a crucial role in patient education around big data and AI, with a focus on addressing any arising questions, concerns, or misconceptions to foster trust in these technologies. Successful integration of data-driven innovations into oncology nursing practice will empower practitioners to deliver more personalized, effective, and evidence-based care.

Anomaly Detection and Concept Drift Adaptation for Dynamic Systems: A General Method with Practical Implementation Using an Industrial Collaborative Robot

Industrial collaborative robots (cobots) are known for their ability to operate in dynamic environments to perform many different tasks (since they can be easily reprogrammed). Due to their features, they are largely used in flexible manufacturing processes. Since fault diagnosis methods are generally applied to systems where the working conditions are bounded, problems arise when defining condition monitoring architecture, in terms of setting absolute criteria for fault analysis and interpreting the meanings of detected values since working conditions may vary. The same cobot can be easily programmed to accomplish more than three or four tasks in a single working day. The extreme versatility of their use complicates the definition of strategies for detecting abnormal behavior. This is because any variation in working conditions can result in a different distribution of the acquired data stream. This phenomenon can be viewed as concept drift (CD). CD is defined as the change in data distribution that occurs in dynamically changing and nonstationary systems. Therefore, in this work, we propose an unsupervised anomaly detection (UAD) method that is capable of operating under CD. This solution aims to identify data changes coming from different working conditions (the concept drift) or a system degradation (failure) and, at the same time, can distinguish between the two cases. Additionally, once a concept drift is detected, the model can be adapted to the new conditions, thereby avoiding misinterpretation of the data. This paper concludes with a proof of concept (POC) that tests the proposed method on an industrial collaborative robot.

R. N, B. R. S, Lahza H, Lahza HFM. A survey on detecting healthcare concept drift in AI/ML models from a finance perspective

Data is incredibly significant in today's digital age because data represents facts and numbers from our regular life transactions. Data is no longer arriving in a static form; it is now arriving in a streaming fashion. Data streams are the arrival of limitless, continuous, and rapid data. The healthcare industry is a major generator of data streams. Processing data streams is extremely complex due to factors such as volume, pace, and variety. Data stream classification is difficult owing to idea drift. Concept drift occurs in supervised learning when the statistical properties of the target variable that the model predicts change unexpectedly. We focused on solving various forms of concept drift problems in healthcare data streams in this research, and we outlined the existing statistical and machine learning methodologies for dealing with concept drift. It also emphasizes the use of deep learning algorithms for concept drift detection and describes the various healthcare datasets utilized for concept drift detection in data stream categorization.