RDDL: A systematic ensemble pipeline tool that streamlines balancing training schemes to reduce the effects of data imbalance in rare-disease-related deep-learning applications

Identifying lowly prevalent diseases, or rare diseases, in their early stages is key to disease treatment in the medical field. Deep learning techniques now provide promising tools for this purpose. Nevertheless, the low prevalence of rare diseases entangles the proper application of deep networks for disease identification due to the severe class-imbalance issue. In the past decades, some balancing methods have been studied to handle the data-imbalance issue. The bad news is that it is verified that none of these methods guarantees superior performance to others. This performance variation causes the need to formulate a systematic pipeline with a comprehensive software tool for enhancing deep-learning applications in rare disease identification. We reviewed the existing balancing schemes and summarized a systematic deep ensemble pipeline with a constructed tool called RDDL for handling the data imbalance issue. Through two real case studies, we showed that rare disease identification could be boosted with this systematic RDDL pipeline tool by lessening the data imbalance problem during model training. The RDDL pipeline tool is available at https://github.com/cobisLab/RDDL/.

Tailored multi-organ segmentation with model adaptation and ensemble

Multi-organ segmentation, which identifies and separates different organs in medical images, is a fundamental task in medical image analysis. Recently, the immense success of deep learning motivated its wide adoption in multi-organ segmentation tasks. However, due to expensive labor costs and expertise, the availability of multi-organ annotations is usually limited and hence poses a challenge in obtaining sufficient training data for deep learning-based methods. In this paper, we aim to address this issue by combining off-the-shelf single-organ segmentation models to develop a multi-organ segmentation model on the target dataset, which helps get rid of the dependence on annotated data for multi-organ segmentation. To this end, we propose a novel dual-stage method that consists of a Model Adaptation stage and a Model Ensemble stage. The first stage enhances the generalization of each off-the-shelf segmentation model on the target domain, while the second stage distills and integrates knowledge from multiple adapted single-organ segmentation models. Extensive experiments on four abdomen datasets demonstrate that our proposed method can effectively leverage off-the-shelf single-organ segmentation models to obtain a tailored model for multi-organ segmentation with high accuracy.

Bias-corrected climate change projections over the Upper Indus Basin using a multi-model ensemble

The study projects climate over the Upper Indus Basin (UIB), covering geographic areas in India, Pakistan, Afghanistan, and China, under the two Representative Concentration Pathways (RCPs), viz., RCP4.5 and RCP8.5 by the late twenty-first century using the best-fit climate model validated against the climate observations from eight meteorological stations. GFDL CM3 performed better than the other five evaluated climate models in simulating the climate of the UIB. The model bias was significantly reduced by the Aerts and Droogers statistical downscaling method, and the projections overall revealed a significant increase in temperature and a slight increase in precipitation across the UIB comprising of Jhelum, Chenab, and Indus sub-basins. According to RCP4.5 and RCP8.5, the temperature and precipitation in the Jhelum are projected to increase by 3 °C and 5.2 °C and 0.8% and 3.4% respectively by the late twenty-first century. The temperature and precipitation in the Chenab are projected to increase by 3.5 °C and 4.8 °C and  8% and 8.2% respectively by the late twenty-first century under the two scenarios. The temperature and precipitation in the Indus are projected to increase by 4.8 °C and 6.5 °C and 2.6% and 8.7% respectively by the late twenty-first century under RCP4.5 and RCP8.5 scenarios. The late twenty-first century projected climate would have significant impacts on various ecosystem services and products, irrigation and socio-hydrological regimes, and various dependent livelihoods. It is therefore hoped that the high-resolution climate projections would be useful for impact assessment studies to inform policymaking for climate action in the UIB.

Prediction of anticancer peptides based on an ensemble model of deep learning and machine learning using ordinal positional encoding

Anticancer peptides (ACPs) are the types of peptides that have been demonstrated to have anticancer activities. Using ACPs to prevent cancer could be a viable alternative to conventional cancer treatments because they are safer and display higher selectivity. Due to ACP identification being highly lab-limited, expensive and lengthy, a computational method is proposed to predict ACPs from sequence information in this study. The process includes the input of the peptide sequences, feature extraction in terms of ordinal encoding with positional information and handcrafted features, and finally feature selection. The whole model comprises of two modules, including deep learning and machine learning algorithms. The deep learning module contained two channels: bidirectional long short-term memory (BiLSTM) and convolutional neural network (CNN). Light Gradient Boosting Machine (LightGBM) was used in the machine learning module. Finally, this study voted the three models' classification results for the three paths resulting in the model ensemble layer. This study provides insights into ACP prediction utilizing a novel method and presented a promising performance. It used a benchmark dataset for further exploration and improvement compared with previous studies. Our final model has an accuracy of 0.7895, sensitivity of 0.8153 and specificity of 0.7676, and it was increased by at least 2% compared with the state-of-the-art studies in all metrics. Hence, this paper presents a novel method that can potentially predict ACPs more effectively and efficiently. The work and source codes are made available to the community of researchers and developers at https://github.com/khanhlee/acp-ope/.

Fast anther dehiscence status recognition system established by deep learning to screen heat tolerant cotton

BACKGROUND

From an economic perspective, cotton is one of the most important crops in the world. The fertility of male reproductive organs is a key determinant of cotton yield. Anther dehiscence or indehiscence directly determines the probability of fertilization in cotton. Thus, rapid and accurate identification of cotton anther dehiscence status is important for judging anther growth status and promoting genetic breeding research. The development of computer vision technology and the advent of big data have prompted the application of deep learning techniques to agricultural phenotype research. Therefore, two deep learning models (Faster R-CNN and YOLOv5) were proposed to detect the number and dehiscence status of anthers.

RESULT

The single-stage model based on YOLOv5 has higher recognition speed and the ability to deploy to the mobile end. Breeding researchers can apply this model to terminals to achieve a more intuitive understanding of cotton anther dehiscence status. Moreover, three improvement strategies are proposed for the Faster R-CNN model, where the improved model has higher detection accuracy than the YOLOv5 model. We have made three improvements to the Faster R-CNN model and after the ensemble of the three models and original Faster R-CNN model, R² of "open" reaches to 0.8765, R² of "close" reaches to 0.8539, R² of "all" reaches to 0.8481, higher than the prediction results of either model alone, which are completely able to replace the manual counting results. We can use this model to quickly extract the dehiscence rate of cotton anthers under high temperature (HT) conditions. In addition, the percentage of dehiscent anthers of 30 randomly selected cotton varieties were observed from the cotton population under normal conditions and HT conditions through the ensemble of the Faster R-CNN model and manual counting. The results show that HT decreased the percentage of dehiscent anthers in different cotton lines, consistent with the manual method.

CONCLUSIONS

Deep learning technology have been applied to cotton anther dehiscence status recognition instead of manual methods for the first time to quickly screen HT-tolerant cotton varieties. Deep learning can help to explore the key genetic improvement genes in the future, promoting cotton breeding and improvement.

Multi-stage ensemble-learning-based model fusion for surface ozone simulations: A focus on CMIP6 models

Accurately simulating the geographical distribution and temporal variability of global surface ozone has long been one of the principal components of chemistry-climate modelling. However, the simulation outcomes have been reported to vary significantly as a result of the complex mixture of uncertain factors that control the tropospheric ozone budget. Settling the cross-model discrepancies to achieve higher accuracy predictions of surface ozone is thus a task of priority, and methods that overcome structural biases in models going beyond naïve averaging of model simulations are urgently required. Building on the Coupled Model Intercomparison Project Phase 6 (CMIP6), we have transplanted a conventional ensemble learning approach, and also constructed an innovative 2-stage enhanced space-time Bayesian neural network to fuse an ensemble of 57 simulations together with a prescribed ozone dataset, both of which have realised outstanding performances (R² > 0.95, RMSE < 2.12 ppbv). The conventional ensemble learning approach is computationally cheaper and results in higher overall performance, but at the expense of oceanic ozone being overestimated and the learning process being uninterpretable. The Bayesian approach performs better in spatial generalisation and enables perceivable interpretability, but induces heavier computational burdens. Both of these multi-stage machine learning-based approaches provide frameworks for improving the fidelity of composition-climate model outputs for uses in future impact studies.

Real-time forecasting of epidemic trajectories using computational dynamic ensembles

Forecasting the trajectory of social dynamic processes, such as the spread of infectious diseases, poses significant challenges that call for methods that account for data and model uncertainty. Here we introduce an ensemble model for sequential forecasting that weights a set of plausible models and use a frequentist computational bootstrap approach to evaluate its uncertainty. We demonstrate the feasibility of our approach using simple dynamic differential-equation models and the trajectory of outbreak scenarios of the Ebola Forecasting Challenge. Specifically, we generate sequential short-term forecasts of epidemic outbreaks by combining phenomenological models that incorporate flexible epidemic growth scaling, namely the Generalized-Growth Model (GGM) and the Generalized Logistic Model (GLM). We rely on the root-mean-square error (RMSE) to quantify the quality of the models' fits during the calibration periods for weighting their contribution to the ensemble model while forecasting performance was evaluated using the RMSE of the forecasts. For a given forecasting horizon (1-4 weeks), we report the performance for each model as the percentage of the number of times each model outperforms the other models. The overall mean RMSE performance of the GLM and the GGM-GLM ensemble models outcompeted that of participant models of the Ebola Forecasting Challenge. We also found that the ensemble model provided more accurate forecasts with higher frequency than the GGM and GLM models, but its performance varied across forecasting horizons. For instance, across all of the Ebola Challenge Scenarios, the ensemble model outperformed the other models at horizons of 2 and 3 weeks while the GLM outperformed other models at horizons of 1 and 4 weeks.

Effects of changes in land use and climate on aquatic ecosystems: Coupling of models and decomposition of uncertainties

To analyse the potential future ecological state of estuaries located in the temperate climate (here exemplified with the Odense Fjord estuary, Denmark), we combined end-of-the-century climate change projections from four different climate models, four contrasting land use scenarios ("Agriculture for nature", "Extensive agriculture", "High-tech agriculture" and "Market driven agriculture") and two different eco-hydrological models. By decomposing the variance of the model-simulated output from all scenario and model combinations, we identified the key sources of uncertainties of these future projections. There was generally a decline in the ecological state of the estuary in scenarios with a warmer climate. Strikingly, even the most nature-friendly land use scenario, where a proportion of the intensive agricultural area was converted to forest, may not be enough to counteract the negative effects of a future warmer climate on the ecological state of the estuary. The different land use scenarios were the most significant sources of uncertainty in the projections of future ecological state, followed, in order, by eco-hydrological models and climate models, albeit all three sources caused high variability in the simulated outputs. Therefore, when projecting the future state of aquatic ecosystems in a global warming context, one should at the very least consider to evaluate an ensemble of land use scenarios (nutrient loads) but ideally also include multiple eco-hydrological models and climate change projections. Our study may set precedence for future attempts to predict and quantify uncertainties of model and model input ensembles, as this will likely be key elements in future tools for decision-making processes.