Prediction and analysis of train arrival delay based on XGBoost and Bayesian optimization

Enhanced nitrogen prediction and mechanistic process analysis in high-salinity wastewater treatment using interpretable machine learning approach

This study introduces an interpretable machine learning framework to predict nitrogen removal in membrane bioreactor (MBR) treating high-salinity wastewater. By integrating Shapley additive explanations (SHAP) with Categorical Boosting (CatBoost), we address the critical gap in linking predictive accuracy to operational decision-making for saline systems. CatBoost achieved the best performance, with an coefficient of determination (R2) of 0.88 and root mean square error (RMSE) of 4.27 for the effluent ammonia nitrogen (NH4+-Nout), and an R2 of 0.91 and RMSE of 4.35 for the effluent total nitrogen (TNout). SHAP analysis uniquely revealed salinity's dual role in inhibiting nitrifying enzymes and disrupting carbon metabolism, with dissolved oxygen, pH and chemical oxygen demand removal efficiency as key regulators. Temperature and carbon-to-nitrogen ratio further modulated total nitrogen dynamics through electron donor availability and microbial activity. The proposed SHAP-CatBoost model in high salinity MBR combines predictive modelling with mechanical process control.

Explainability Enhanced Machine Learning Model for Classifying Intellectual Disability and Attention-Deficit/Hyperactivity Disorder With Psychological Test Reports

BACKGROUND

Psychological test reports are essential in assessing intellectual functioning, aiding in diagnosing and treating intellectual disability (ID) and attention-deficit/hyperactivity disorder (ADHD). However, these reports can have several problems because they are diverse, unstructured, subjective, and involve human errors. Additionally, physicians often do not read the entire report, and the number of reports is lower than that of diagnoses.

METHODS

We developed explainable predictive models for classifying IDs and ADHDs based on written reports to address these issues. The reports of 1,475 patients with IDs and ADHDs who underwent intelligence tests were used for the models. These models were developed by analyzing reports using natural language processing (NLP) and incorporating the physician's diagnosis for each report. We selected n-gram features from the models' results by extracting important features using SHapley Additive exPlanations and permutation importance to make the models explainable. Developing the n-gram feature-based original text search system compensated for the lack of human readability caused by NLP and enabled the reconstruction of human-readable texts from the selected n-gram features.

RESULTS

The maximum model accuracy was 0.92, and the 80 human-readable texts were restored from four models.

CONCLUSION

The results showed that the models could accurately classify IDs and ADHDs, even with a few reports. The models were also able to explain their predictions. The explainability-enhanced model can help physicians understand the classification process of IDs and ADHDs and provide evidence-based insights.

Optimizing chickpea yield prediction under wilt disease through synergistic integration of biophysical and image parameters using machine learning models

Crop health assessment and early yield predictions are highly crucial under biotic stress conditions for crop management and market planning by farmers and policy planners. The objective of this study was, therefore, to assess the impact of different levels of wilt disease on the biophysical parameters of chickpea and developing machine learning (ML) models for early yield prediction. Field experiments were carried out over three years at the Indian Agricultural Research Institute research farm in New Delhi. Thermal and visible images were collected alongside the measurement of crop biophysical parameters, including leaf area index (LAI), photosynthesis, transpiration rate, stomatal conductance, relative leaf water content (RWC), membrane stability index (MSI), and NDVI, for 85 chickpea genotypes with varying levels of wilt resistance. ML models were developed for early yield prediction by combining visible and thermal image indices with biophysical parameters. The results showed that the canopy temperatures were directly correlated with increasing levels of wilt severity. Crop photosynthesis, stomatal conductance, transpiration, LAI, RWC, MSI, and NDVI dropped significantly with increasing levels of wilt severity. Yield reductions of 44-69% were observed in susceptible genotypes. Machine learning models were able to give accurate early yield predictions. The accuracy of the models increases as we move closer to the harvest. Ranking of the model's performances indicated that XGB is the best model to predict chickpea yield under wilt conditions. NDVI was identified as most important variable for yield prediction. The findings of the study quantified the impacts of wilt on important crop biophysical parameters and highlighted the suitability of ML models in early yield prediction under different levels of disease severity.

Salmon Consumption Behavior Prediction Based on Bayesian Optimization and Explainable Artificial Intelligence

Predicting seafood consumption behavior is essential for fishing companies to adjust their production plans and marketing strategies. To achieve accurate predictions, this paper introduces a model for forecasting seafood consumption behavior based on an interpretable machine learning algorithm. Additionally, the Shapley Additive exPlanation (SHAP) model and the Accumulated Local Effects (ALE) plot were integrated to provide a detailed analysis of the factors influencing Shanghai residents' intentions to purchase salmon. In this study, we constructed nine regression prediction models, including ANN, Decision Tree, GBDT, Random Forest, AdaBoost, XGBoost, LightGBM, CatBoost, and NGBoost, to predict the consumers' intentions to purchase salmon and to compare their predictive performance. In addition, Bayesian optimization algorithm is used to optimize the hyperparameters of the optimal regression prediction model to improve the model prediction accuracy. Finally, the SHAP model was used to analyze the key factors and interactions affecting the consumers' willingness to purchase salmon, and the Accumulated Local Effects plot was used to show the specific prediction patterns of different influences on salmon consumption. The results of the study show that salmon farming safety and ease of cooking have significant nonlinear effects on salmon consumption; the BO-CatBoost nonlinear regression prediction model demonstrates superior performance compared to the benchmark model, with the test set exhibiting RMSE, MSE, MAE, R2 and TIC values of 0.155, 0.024, 0.097, 0.902, and 0.313, respectively. This study can provide technical support for suppliers in the salmon value chain and help their decision-making to adjust their corporate production plan and marketing activities.

Temporal Autoregressive Matrix Factorization for High-Dimensional Time Series Prediction of OSS

Open-source software (OSS) plays an increasingly significant role in modern software development tendency, so accurate prediction of the future development of OSS has become an essential topic. The behavioral data of different open-source software are closely related to their development prospects. However, most of these behavioral data are typical high-dimensional time series data streams with noise and missing values. Hence, accurate prediction on such cluttered data requires the model to be highly scalable, which is not a property of traditional time series prediction models. To this end, we propose a temporal autoregressive matrix factorization (TAMF) framework that supports data-driven temporal learning and prediction. Specifically, we first construct a trend and period autoregressive model to extract trend and period features from OSS behavioral data, and then combine the regression model with a graph-based matrix factorization (MF) to complete the missing values by exploiting the correlations among the time series data. Finally, use the trained regression model to make predictions on the target data. This scheme ensures that TAMF can be applied to different types of high-dimensional time series data and thus has high versatility. We selected ten real developer behavior data from GitHub for case analysis. The experimental results show that TAMF has good scalability and prediction accuracy.

Rice yield prediction through integration of biophysical parameters with SAR and optical remote sensing data using machine learning models

In an era marked by growing global population and climate variability, ensuring food security has become a paramount concern. Rice, being a staple crop for billions of people, requires accurate and timely yield prediction to ensure global food security. This study was undertaken across two rice crop seasons in the Udham Singh Nagar district of Uttarakhand state to predict rice yield at 45, 60 and 90 days after transplanting (DAT) through machine learning (ML) models, utilizing a combination of optical and Synthetic Aperture Radar (SAR) data in conjunction with crop biophysical parameters. Results revealed that the ML models were able to provide relatively accurate early yield estimates. For summer rice, eXtreme gradient boosting (XGB) was the best-performing model at all three stages (45, 60, and 90 DAT), while for kharif rice, the best-performing models at 45, 60, and 90 DAT were XGB, Neural network (NNET), and Cubist, respectively. The combined ranking of ML models showed that prediction accuracy improved as the prediction date approaches harvest, and the best prediction of yield was observed at 90 DAT for both summer and kharif rice. Overall rankings indicate that for summer rice, the top three models were XGB, NNET, and Support vector regression, while for kharif rice, these were Cubist, NNET, and Random Forest, respectively. The findings of this study offer valuable insights into the potential of the combined use of remote sensing and biophysical parameters using ML models, which enhances food security planning and resource management by enabling more informed decision-making by stakeholders such as farmers, policy planners as well as researchers.

The shift in the spatiotemporal relationship between supply and demand of ecosystem services and its drivers in China

In China, over 65% of human activities are concentrated in cities, resulting in a conflict between the supply and demand of ecosystem services (ESs). To alleviate this problem, many cities have adopted eco-friendly development modes, however, the effectiveness of these models in reducing ESs supply-demand conflicts has not been comprehensively reviewed, and the human and natural drivers behind these relationship shifts remain unclear. To bridge this gap, this study analyzed the shifts in the relationships between supply and demand of ESs across China from 2010 to 2020 at a city level, as well as identified the human and natural drivers behind them. Firstly, the InVEST models were integrated with socioeconomic data to evaluate the supply and demand distribution for three pivotal ESs: water yield (WY), habitat quality (HQ), and soil retention (SR). Then, a four-quadrant diagram approach was proposed to enhance the analysis of their spatiotemporal relationships. Furthermore, random forest models were employed to examine the drivers of the shifts in these relationships. The results showed that WY and SR services witnessed growth until 2015, and then receded, while HQ saw a modest decline from 2010 to 2020. Spatial synergies in the supply and demand of ESs were primarily observed in the southern cities, with a significant northward extension by 2020. From a temporal perspective, the percentage of cities achieving coordination in WY and SR services increased from 32.6% to 57.3%, respectively, in the 2010-2015 period to 42.4% and 63.3% between 2015 and 2020, meanwhile, HQ service conflicts diminished from 58.7% to 53.5%. The changes in socioeconomic and land use factors contributed to 64.3%, 36.1%, and 33.3% of the shifts in the supply-demand relationship for HQ, WY, and SR services, respectively. Our analysis highlights the potential of human-driven ecological management to enhance the balance of this relationship. It can support the design of city-specific policies that foster a balance between ecological processes and socio-economic development.

Prediction of Out-of-Hospital Cardiac Arrest Survival Outcomes Using a Hybrid Agnostic Explanation TabNet Model

Survival after out-of-hospital cardiac arrest (OHCA) is contingent on time-sensitive interventions taken by onlookers, emergency call operators, first responders, emergency medical services (EMS) personnel, and hospital healthcare staff. By building integrated cardiac resuscitation systems of care, measurement systems, and techniques for assuring the correct execution of evidence-based treatments by bystanders, EMS professionals, and hospital employees, survival results can be improved. To aid in OHCA prognosis and treatment, we develop a hybrid agnostic explanation TabNet (HAE-TabNet) model to predict OHCA patient survival. According to the results, the HAE-TabNet model has an “Area under the receiver operating characteristic curve value” (ROC AUC) score of 0.9934 (95% confidence interval 0.9933–0.9935), which outperformed other machine learning models in the previous study, such as XGBoost, k-nearest neighbors, random forest, decision trees, and logistic regression. In order to achieve model prediction explainability for a non-expert in the artificial intelligence field, we combined the HAE-TabNet model with a LIME-based explainable model. This HAE-TabNet model may assist medical professionals in the prognosis and treatment of OHCA patients effectively.

m1A-pred: Prediction of Modified 1-methyladenosine Sites in RNA Sequences through Artificial Intelligence

BACKGROUND

The process of nucleotides modification or methyl groups addition to nucleotides is known as post-transcriptional modification (PTM). 1-methyladenosine (m1A) is a type of PTM formed by adding a methyl group to the nitrogen at the 1st position of the adenosine base. Many human disorders are associated with m1A, which is widely found in ribosomal RNA and transfer RNA.

OBJECTIVE

The conventional methods such as mass spectrometry and site-directed mutagenesis proved to be laborious and burdensome. Systematic identification of modified sites from RNA sequences is gaining much attention nowadays. Consequently, an extreme gradient boost predictor, m1A-Pred, is developed in this study for the prediction of modified m1A sites.

METHODS

The current study involves the extraction of position and composition-based properties within nucleotide sequences. The extraction of features helps in the development of the features vector. Statistical moments were endorsed for dimensionality reduction in the obtained features.

RESULTS

Through a series of experiments using different computational models and evaluation methods, it was revealed that the proposed predictor, m1A-pred, proved to be the most robust and accurate model for the identification of modified sites.

AVAILABILITY AND IMPLEMENTATION

To enhance the research on m¹A sites, a friendly server was also developed, which was the final phase of this research.

Explainable diabetes classification using hybrid Bayesian-optimized TabNet architecture

Diabetes is a deadly chronic disease that occurs when the pancreas is not able to produce ample insulin or when the body cannot use insulin effectively. If undetected, it may lead to a host of health complications. Hence, accurate and explainable early-stage detection of diabetes is essential for the proper administration of treatment options in leading a healthy and productive life. For this, we developed an interpretable TabNet model tuned via Bayesian optimization (BO). To achieve model-specific interpretability, the attention mechanism of TabNet architecture was used, which offered the local and global model explanations on the influence of the attributes on the outcomes. The model was further explained locally and globally using more robust model-agnostic LIME and SHAP eXplainable Artificial Intelligence (XAI) tools. The proposed model outperformed all benchmarked models by obtaining high accuracy of 92.2% and 99.4% using the Pima Indians diabetes dataset (PIDD) and the early-stage diabetes risk prediction dataset (ESDRPD), respectively. Based on the XAI results, it was clear that the most influential attribute for diabetes classification using PIDD and ESDRPD were Insulin and Polyuria, respectively. The feature importance values registered for insulin was 0.301 (PIDD) and for polyuria 0.206 was registered (ESDRPD). The high accuracy and ancillary interpretability of our objective model is expected to increase end-users trust and confidence in early-stage detection of diabetes.

Machine Learning Enabled 3D Body Measurement Estimation Using Hybrid Feature Selection and Bayesian Search

The 3D body scan technology has recently innovated the way of measuring human bodies and generated a large volume of body measurements. However, one inherent issue that plagues the use of the resultant database is the missing data usually caused by using automatic data extractions from the 3D body scans. Tedious extra efforts have to be made to manually fill the missing data for various applications. To tackle this problem, this paper proposes a machine learning (ML)-based approach for 3D body measurement estimation while considering the measurement (feature) importance. The proposed approach selects the most critical features to reduce the algorithm input and to improve the ML method performance. In addition, a Bayesian search is further used in fine-tuning the hyperparameters to minimize the mean square error. Two distinct ML methods, i.e., Random Forest and XGBoost, are used and tested on a real-world dataset that contains 3D body scans of 212 participants in the Kansas-Missouri area of the United States. The results show the effectiveness of the proposed methods with roughly 3% of Mean Absolute Percentage Errors in estimating the missing data. The two ML methods with the proposed hybrid feature selection and the Baysian search are comprehensively compared. The comparative results suggest that the Random Forest method performs better than the XGBoost counterpart in filling missing 3D body measurements.

Vibration prediction and analysis of strip rolling mill based on XGBoost and Bayesian optimization

The stable operation of strip rolling mill is the key factor to ensure the stability of product quality. The design capability of existing domestic imported and self-developed strip rolling mills cannot be fully developed, and the frequent occurrence of mill vibration and operation instability problems seriously restrict the equipment capacity and the production of high-end strip products. The vibration prediction analysis method for hot strip mill based on eXtreme gradient boosting (XGBoost) and Bayesian optimization (BO) is proposed. First, an XGBoost prediction model is developed based on a self-built data set to construct a complex functional relationship between process parameters and rolling mill vibration. Second, the important hyperparameters and parameters of XGBoost are optimized using Bayesian optimization algorithm to improve the prediction accuracy, computational efficiency, and stability of the model. Third, a comprehensive comparison is made between the prediction model in this paper and other well-known machine learning benchmark models. Finally, the prediction results of the model are interpreted using the SHapley Additive exPlanations (SHAP) method. The proposed model outperforms existing models in terms of prediction accuracy, computational speed and stability. At the same time, the degree of influence of each feature on rolling mill vibration is also obtained.

First satellite-based regional hourly NO2 estimations using a space-time ensemble learning model: A case study for Beijing-Tianjin-Hebei Region, China

Surface Nitrogen dioxide (NO₂) concentrations have been generated with satellite retrievals using multiple statistical algorithms. However, they are often given at coarse frequencies ("snapshot", daily or even longer), limiting their applications in epidemiological studies and assessing the evolution of NO₂ pollution. This study investigated the potential applicability of Himawari-8 derived hourly fine particulate matter concentrations in producing hourly NO₂ concentrations by constructing a space-time ensemble model. The Beijing-Tianjin-Hebei (BTH) region, one of the serious pollution regions in China, is the study region chosen. The proposed model performs well in estimating hourly NO₂ concentration with a high cross-validation (CV) coefficient of determination (R² = 0.81) and low CV root-mean-square (RMSE = 9.71 μg/m³), mean prediction errors (MPE = 6.33 μg/m³), and relative prediction errors (RPE = 22.5%). On daily, monthly, seasonal, and annual time scales, CV R² increases to 0.89, 0.93, 0.97, and 0.99, respectively. The annual mean model estimated NO₂ concentration over BTH region is 28.2 ± 6.5 μg/m³, with relatively higher NO₂ concentrations are seen in southern and southeastern BTH. Winter experiences the most severe NO₂ concentrations, followed by autumn, spring, and summer. Surface NO₂ concentrations are higher (lower) in the morning (afternoon) and tend to decrease gradually with time. The model generally captures the hourly evolution of NO₂ concentrations for the severe pollution episode but shows some underestimations. The annual mean NO₂ concentrations were 2.8% lower on the weekend than on weekdays. In addition, the weekend effects of NO₂ concentrations are larger at rush hour and lower in the noon. The hourly NO₂ products derived from proposed approach are potentially useful for improving our understanding of the source, evolution, and transportation behavior of NO₂ pollution episodes and for exposure- and health-related research. The proposed approach also enriches the potential applications of geostationary satellites (e.g., Himawari-8).

Industry 4.0 Technologies Applied to the Rail Transportation Industry: A Systematic Review

BACKGROUND

Industry 4.0 technologies have been widely used in the railway industry, focusing mainly on maintenance and control tasks necessary in the railway infrastructure. Given the great potential that these technologies offer, the scientific community has come to use them in varied ways to solve a wide range of problems such as train failures, train station security, rail system control and communication in hard-to-reach areas, among others. For this reason, this paper aims to answer the following research questions: what are the main issues in the railway transport industry, what are the technologic strategies that are currently being used to solve these issues and what are the technologies from industry 4.0 that are used in the railway transport industry to solve the aforementioned issues?

METHODS

This study adopts a systematic literature review approach. We searched the Science Direct and Web of Science database inception from January 2017 to November 2021. Studies published in conferences or journals written in English or Spanish were included for initial process evaluation. The initial included papers were analyzed by authors and selected based on whether they helped answer the proposed research questions or not.

RESULTS

Of the recovered 515 articles, 109 were eligible, from which we could identify three main application domains in the railway industry: monitoring, decision and planification techniques, and communication and security. Regarding industry 4.0 technologies, we identified 9 different technologies applied in reviewed studies: Artificial Intelligence (AI), Internet of Things (IoT), Cloud Computing, Big Data, Cybersecurity, Modelling and Simulation, Smart Decision Support Systems (SDSS), Computer Vision and Virtual Reality (VR). This study is, to our knowledge, one of the first to show how industry 4.0 technologies are currently being used to tackle railway industry problems and current application trends in the scientific community, which is highly useful for the development of future studies and more advanced solutions.

FUNDING

Colombian national organizations Minciencias and the Mining-Energy Planning Unit.

Hybrid feature selection-based machine learning Classification system for the prediction of injury severity in single and multiple-vehicle accidents

To undertake a reliable analysis of injury severity in road traffic accidents, a complete understanding of important attributes is essential. As a result of the shift from traditional statistical parametric procedures to computer-aided methods, machine learning approaches have become an important aspect in predicting the severity of road traffic injuries. The paper presents a hybrid feature selection-based machine learning classification approach for detecting significant attributes and predicting injury severity in single and multiple-vehicle accidents. To begin, we employed a Random Forests (RF) classifier in conjunction with an intrinsic wrapper-based feature selection approach called the Boruta Algorithm (BA) to find the relevant important attributes that determine injury severity. The influential attributes were then fed into a set of four classifiers to accurately predict injury severity (Naive Bayes (NB), K-Nearest Neighbor (K-NN), Binary Logistic Regression (BLR), and Extreme Gradient Boosting (XGBoost)). According to BA's experimental investigation, the vehicle type was the most influential factor, followed by the month of the year, the driver's age, and the alignment of the road segment. The driver's gender, the presence of a median, and the presence of a shoulder were all found to be unimportant. According to classifier performance measures, XGBoost surpasses the other classifiers in terms of prediction performance. Using the specified attributes, the accuracy, Cohen's Kappa, F1-Measure, and AUC-ROC values of the XGBoost were 82.10%, 0.607, 0.776, and 0.880 for single vehicle accidents and 79.52%, 0.569, 0.752, and 0.86 for multiple-vehicle accidents, respectively.