Bayesian machine learning ensemble approach to quantify model uncertainty in predicting groundwater storage change

A new strategy for groundwater level prediction using a hybrid deep learning model under Ecological Water Replenishment

Accurate prediction of the groundwater level (GWL) is crucial for sustainable groundwater resource management. Ecological water replenishment (EWR) involves artificially diverting water to replenish the ecological flow and water resources of both surface water and groundwater within the basin. However, fluctuations in GWLs during the EWR process exhibit high nonlinearity and complexity in their time series, making it challenging for single data-driven models to predict the trend of groundwater level changes under the backdrop of EWR. This study introduced a new GWL prediction strategy based on a hybrid deep learning model, STL-IWOA-GRU. It integrated the LOESS-based seasonal trend decomposition algorithm (STL), improved whale optimization algorithm (IWOA), and Gated recurrent unit (GRU). The aim was to accurately predict GWLs in the context of EWR. This study gathered GWL, precipitation, and surface runoff data from 21 monitoring wells in the Yongding River Basin (Beijing Section) over a period of 731 days. The research results demonstrate that the improvement strategy implemented for the IWOA enhances the convergence speed and global search capabilities of the algorithm. In the case analysis, evaluation metrics including the root mean square error (RMSE), mean absolute error (MAE), mean absolute percentage error (MAPE), and Nash-Sutcliffe efficiency (NSE) were employed. STL-IWOA-GRU exhibited commendable performance, with MAE achieving the best result, averaging at 0.266. When compared to other models such as Variance Mode Decomposition-Gated Recurrent Unit (VMD-GRU), Ant Lion Optimizer-Support Vector Machine (ALO-SVM), STL-Particle Swarm Optimization-GRU (STL-PSO-GRU), and STL-Sine Cosine Algorithm-GRU (STL-SCA-GRU), MAE was reduced by 18%, 26%, 11%, and 29%, respectively. This indicates that the model proposed in this study exhibited high prediction accuracy and robust versatility, making it a potent strategic choice for forecasting GWL changes in the context of EWR.

Uncertainty-based saltwater intrusion prediction using integrated Bayesian machine learning modeling (IBMLM) in a deep aquifer

Data-driven machine learning approaches are promising to substitute physically based groundwater numerical models and capture input-output relationships for reducing computational burden. But the performance and reliability are strongly influenced by different sources of uncertainty. Conventional researches generally rely on a stand-alone machine learning surrogate approach and fail to account for errors in model outputs resulting from structural deficiencies. To overcome this issue, this study proposes a flexible integrated Bayesian machine learning modeling (IBMLM) method to explicitly quantify uncertainties originating from structures and parameters of machine learning surrogate models. An Expectation-Maximization (EM) algorithm is combined with Bayesian model averaging (BMA) to find out maximum likelihood and construct posterior predictive distribution. Three machine learning approaches representing different model complexity are incorporated in the framework, including artificial neural network (ANN), support vector machine (SVM) and random forest (RF). The proposed IBMLM method is demonstrated in a field-scale real-world "1500-foot" sand aquifer, Baton Rouge, USA, where overexploitation caused serious saltwater intrusion (SWI) issues. This study adds to the understanding of how chloride concentration transport responds to multi-dimensional extraction-injection remediation strategies in a sophisticated saltwater intrusion model. Results show that most IBMLM exhibit r values above 0.98 and NSE values above 0.93, both slightly higher than individual machine learning, confirming that the IBMLM is well established to provide better model predictions than individual machine learning models, while maintaining the advantage of high computing efficiency. The IBMLM is found useful to predict saltwater intrusion without running the physically based numerical simulation model. We conclude that an explicit consideration of machine learning model structure uncertainty along with parameters improves accuracy and reliability of predictions, and also corrects uncertainty bounds. The applicability of the IBMLM framework can be extended in regions where a physical hydrogeologic model is difficult to build due to lack of subsurface information.

Faba bean and pea harvest index estimations using aerial-based multimodal data and machine learning algorithms

Early and high-throughput estimations of the crop harvest index (HI) are essential for crop breeding and field management in precision agriculture; however, traditional methods for measuring HI are time-consuming and labor-intensive. The development of unmanned aerial vehicles (UAVs) with onboard sensors offers an alternative strategy for crop HI research. In this study, we explored the potential of using low-cost, UAV-based multimodal data for HI estimation using red-green-blue (RGB), multispectral (MS), and thermal infrared (TIR) sensors at 4 growth stages to estimate faba bean (Vicia faba L.) and pea (Pisum sativum L.) HI values within the framework of ensemble learning. The average estimates of RGB (faba bean: coefficient of determination [R2] = 0.49, normalized root-mean-square error [NRMSE] = 15.78%; pea: R2 = 0.46, NRMSE = 20.08%) and MS (faba bean: R2 = 0.50, NRMSE = 15.16%; pea: R2 = 0.46, NRMSE = 19.43%) were superior to those of TIR (faba bean: R2 = 0.37, NRMSE = 16.47%; pea: R2 = 0.38, NRMSE = 19.71%), and the fusion of multisensor data exhibited a higher estimation accuracy than those obtained using each sensor individually. Ensemble Bayesian model averaging provided the most accurate estimations (faba bean: R2 = 0.64, NRMSE = 13.76%; pea: R2 = 0.74, NRMSE = 15.20%) for whole growth stage, and the estimation accuracy improved with advancing growth stage. These results indicate that the combination of low-cost, UAV-based multimodal data and machine learning algorithms can be used to estimate crop HI reliably, therefore highlighting a promising strategy and providing valuable insights for high spatial precision in agriculture, which can help breeders make early and efficient decisions.

An intelligent decision support system for groundwater supply management and electromechanical infrastructure controls

This study presents an intelligent Decision Support System (DSS) aimed at bridging the theoretical-practical gap in groundwater management. The ongoing demand for sophisticated systems capable of interpreting extensive data to inform sustainable groundwater decision-making underscores the critical nature of this research. To meet this challenge, telemetry data from six randomly selected wells were used to establish a comprehensive database of groundwater pumping parameters, including flow rate, pressure, and current intensity. Statistical analysis of these parameters led to the determination of threshold values for critical factors such as water pressure and electrical current. Additionally, a soft sensor was developed using a Random Forest (RF) machine learning algorithm, enabling real-time forecasting of key variables. This was achieved by continuously comparing live telemetry data to pump design specifications and results from regular field testing. The proposed machine learning model ensures robust empirical monitoring of well and pump health. Furthermore, expert operational knowledge from water management professionals, gathered through a Classical Delphi (CD) technique, was seamlessly integrated. This collective expertise culminated in a data-driven framework for sustainable groundwater facilities monitoring. In conclusion, this innovative DSS not only addresses the theory-application gap but also leverages the power of data analytics and expert knowledge to provide high-precision online insights, thereby optimizing groundwater management practices.

Bayesian model averaging by combining deep learning models to improve lake water level prediction

Water level (WL) is an essential indicator of lakes and sensitive to climate change. Fluctuations of lake WL may significantly affect water supply security and ecosystem stability. Accurate prediction of lake WL is, therefore, crucial for water resource management and eco-environmental protection. In this study, three deep learning (DL) models, including long short-term memory (LSTM), the gated recurrent unit (GRU), and the temporal convolutional network (TCN), were used to predict WLs at five stations of Poyang Lake for different forecast periods (1-day ahead, 3-day ahead, and 7-day ahead). The forecast results of the three DL models were synthesized through Bayesian model averaging (BMA) to improve prediction accuracy, and Monte Carlo sampling method was used to calculated the 90 % confidence intervals to analyze the model uncertainty. All the three DL models achieved satisfactory prediction accuracy. GRU performed best in most forecast scenarios, followed by TCN and LSTM. None of the models, however, consistently provided the optimal results in all forecast scenarios. Lake WL prediction accuracy of BMA had a further improvement in metrics of NSE and R2 in 80 % of the forecast scenarios and ranked at least top two in all forecast scenarios. The uncertainty analysis showed that the containing ration (CR) values were above 84 % while the relative bandwidth (RB) maintained reliable performance over the 7-day ahead prediction. The proposed framework in the present study can realize satisfactory WL forecast accuracy while avoiding complex comparison and selection of DL models, and it can also be easily applied to the prediction of other hydrological variables.

HCEs-Net: Hepatic cystic echinococcosis classification ensemble model based on tree-structured Parzen estimator and snap-shot approach

BACKGROUND

Hepatic cystic echinococcosis (HCE) still has a high misdiagnosis rate, and misdiagnosis may lead to wrong treatments seriously harmful for the patients. Precise diagnosis of HCE relies heavily on the experience of clinical experts with auxiliary diagnostic tools using medical images.

PURPOSE

This paper intends to improve the diagnostic accuracy for HCE by employing a method which combines deep learning with ensemble method.

METHODS

We proposed a method, namely HCEs-Net, for classification of five HCE subtypes using ultrasound images. It takes first the snap-shot strategy to obtain sub-models from the pre-trained VGG19, ResNet18, ViT-Base, and ConvNeXt-T models, then a stacking process to ensemble those sub-models. Afterwards, it uses the tree-structured Pazren estimator (TPE) to optimize the hyperparameters. The experiments were evaluated by the five-fold cross-validation process.

RESULTS

A total of 3083 abdominal ultrasound images from 972 patients covering five subtypes of HCE were utilized in this study. The experiments were conducted to predict the HCE subtype, and results of modeling performance evaluation were reported in terms of precision, recall, F1-score, and AUC. The stacking model based on three ConvNeXt-T sub-models showed the best performance, with precision 85.9%, recall 85.5%, F1-score 85.7%, and AUC 0.971 which are higher than the compared state-of-the-art models.

CONCLUSION

The stacking model of three ConvNeXt-T sub-models shows comparable or superior performance to the other methods, including VGG19, ResNet18 and ViT-Base. It has the potential to enhance clinical diagnosis for HCE.

Bayesian inference for survival prediction of childhood Leukemia

BACKGROUND

Childhood Leukemia is the most common type of cancer among children. Nearly 39% of cancer-induced childhood deaths are attributable to Leukemia. Nevertheless, early intervention has long been underdeveloped. Moreover, there are still a group of children succumbing to their cancer due to the cancer care resource disparity. Therefore, it calls for an accurate predictive approach to improve childhood Leukemia survival and mitigate these disparities. Existing survival predictions rely on a single best model, which fails to consider model uncertainties in predictions. Prediction from a single model is brittle, with model uncertainty neglected, and inaccurate prediction could lead to serious ethical and economic consequences.

METHODS

To address these challenges, we develop a Bayesian survival model to predict patient-specific survivals by taking model uncertainty into account. Specifically, we first develop a survival model predict time-varying survival probabilities. Second, we place different prior distributions over various model parameters and estimate their posterior distribution with full Bayesian inference. Third, we predict the patient-specific survival probabilities changing with respect to time by considering model uncertainty induced by posterior distribution.

RESULTS

Concordance index of the proposed model is 0.93. Moreover, the standardized survival probability of the censored group is higher than that of the deceased group.

CONCLUSIONS

Experimental results indicate that the proposed model is robust and accurate in predicting patient-specific survivals. It can also help clinicians track the contribution of multiple clinical attributes, thereby enabling well-informed intervention and timely medical care for childhood Leukemia.

Prediction of elevated groundwater fluoride across India using multi-model approach: insights on the influence of geologic and environmental factors

Elevated fluoride in groundwater is a severe problem in India due to its extensive occurrence and detrimental health impacts on the large population that thrives on groundwater. Although fluoride is primarily a geogenic pollutant, existing model-based studies lack the amalgamation of the influence of geologic factors, specifically tectonics, for identifying groundwater fluoride distribution. This drawback encourages the present study to investigate the association of the tectonic framework with fluoride in a multi-model approach. We have applied three machine learning models (random forest, boosted regression tree, and logistic regression) to predict elevated groundwater fluoride based on fluoride measurements across India. The random forest model outperformed other models with an accuracy of 93%. Tectonics was found to be one of the most important predictors alongside "depth to water table." Two major areas of high risk identified were the northwest parts and the south-southeast cratonic peninsular region. The random forest model also performed significantly well over the validation dataset. We estimate that nearly 257 million people are exposed to elevated fluoride risk in India. We endeavor that the findings of our study would be an effective tool for identifying the areas at risk of elevated fluoride and also assist in undertaking effective groundwater management strategies.

Groundwater level response identification by hybrid wavelet-machine learning conjunction models using meteorological data

Due to its heterogeneous and complex nature, groundwater modeling needs great effort to quantify the aquifer, a crucial tool for policymakers and hydrogeologists to understand the variations in groundwater levels (GWL). This study proposed a set of supervised machine learning (ML) models to delineate the GWL changes in the Zarand-Saveh complex aquifer in Iran using 15-year (2005-2020) monthly dataset. The wavelet transform (WT) procedure was also used to improve the GWL prediction ability of ML models for 3-month horizons using input datasets of precipitation, evapotranspiration, temperature, and GWL. The four well-accepted standalone ML methods, i.e., artificial neural network (ANN), adaptive neuro-fuzzy inference system (ANFIS), group method of data handling (GMDH), and least square support vector machine (LSSVM), were implemented and compared with the hybrid wavelet conjunction models. The methods were compared based on root mean square error (RMSE), mean absolute error (MAE), correlation coefficient (R), and Nash-Sutcliffe efficiency (NSE). Comparison outcomes showed that the hybrid wavelet-ML considerably improved the standalone model results. The wavelet transform-least square support vector machine (WT-LSSVM) model was superior to other standalone and hybrid wavelet-ML methods to predict GWL. The best GWL predictions were acquired from the WT-LSSVM model with input scenario 5 involving all influential variables, and this model produced RMSE, MAE, R, and NSE as 0.05, 0.04, 0.99, and 0.99 for 1 month ahead of GWL prediction, while the corresponding values were obtained as 0.18, 0.14, 0.95, and 0.90 for 3 months ahead of GWL prediction, respectively.

An interpretable stacking ensemble learning framework based on multi-dimensional data for real-time prediction of drug concentration: The example of olanzapine

Background and Aim: Therapeutic drug monitoring (TDM) has evolved over the years as an important tool for personalized medicine. Nevertheless, some limitations are associated with traditional TDM. Emerging data-driven model forecasting [e.g., through machine learning (ML)-based approaches] has been used for individualized therapy. This study proposes an interpretable stacking-based ML framework to predict concentrations in real time after olanzapine (OLZ) treatment.

Methods: The TDM-OLZ dataset, consisting of 2,142 OLZ measurements and 472 features, was formed by collecting electronic health records during the TDM of 927 patients who had received OLZ treatment. We compared the performance of ML algorithms by using 10-fold cross-validation and the mean absolute error (MAE). The optimal subset of features was analyzed by a random forest-based sequential forward feature selection method in the context of the top five heterogeneous regressors as base models to develop a stacked ensemble regressor, which was then optimized via the grid search method. Its predictions were explained by using local interpretable model-agnostic explanations (LIME) and partial dependence plots (PDPs).

Results: A state-of-the-art stacking ensemble learning framework that integrates optimized extra trees, XGBoost, random forest, bagging, and gradient-boosting regressors was developed for nine selected features [i.e., daily dose (OLZ), gender_male, age, valproic acid_yes, ALT, K, BW, MONO#, and time of blood sampling after first administration]. It outperformed other base regressors that were considered, with an MAE of 0.064, R-square value of 0.5355, mean squared error of 0.0089, mean relative error of 13%, and ideal rate (the percentages of predicted TDM within ± 30% of actual TDM) of 63.40%. Predictions at the individual level were illustrated by LIME plots, whereas the global interpretation of associations between features and outcomes was illustrated by PDPs.

Conclusion: This study highlights the feasibility of the real-time estimation of drug concentrations by using stacking-based ML strategies without losing interpretability, thus facilitating model-informed precision dosing.

Application of UAV Multisensor Data and Ensemble Approach for High-Throughput Estimation of Maize Phenotyping Traits

High-throughput estimation of phenotypic traits from UAV (unmanned aerial vehicle) images is helpful to improve the screening efficiency of breeding maize. Accurately estimating phenotyping traits of breeding maize at plot scale helps to promote gene mining for specific traits and provides a guarantee for accelerating the breeding of superior varieties. Constructing an efficient and accurate estimation model is the key to the application of UAV-based multiple sensors data. This study aims to apply the ensemble learning model to improve the feasibility and accuracy of estimating maize phenotypic traits using UAV-based red-green-blue (RGB) and multispectral sensors. The UAV images of four growth stages were obtained, respectively. The reflectance of visible light bands, canopy coverage, plant height (PH), and texture information were extracted from RGB images, and the vegetation indices were calculated from multispectral images. We compared and analyzed the estimation accuracy of single-type feature and multiple features for LAI (leaf area index), fresh weight (FW), and dry weight (DW) of maize. The basic models included ridge regression (RR), support vector machine (SVM), random forest (RF), Gaussian process (GP), and K-neighbor network (K-NN). The ensemble learning models included stacking and Bayesian model averaging (BMA). The results showed that the ensemble learning model improved the accuracy and stability of maize phenotypic traits estimation. Among the features extracted from UAV RGB images, the highest accuracy was obtained by the combination of spectrum, structure, and texture features. The model had the best accuracy constructed using all features of two sensors. The estimation accuracies of ensemble learning models, including stacking and BMA, were higher than those of the basic models. The coefficient of determination (R²) of the optimal validation results were 0.852, 0.888, and 0.929 for LAI, FW, and DW, respectively. Therefore, the combination of UAV-based multisource data and ensemble learning model could accurately estimate phenotyping traits of breeding maize at plot scale.

UAV-based multi-sensor data fusion and machine learning algorithm for yield prediction in wheat

Early prediction of grain yield helps scientists to make better breeding decisions for wheat. Use of machine learning (ML) methods for fusion of unmanned aerial vehicle (UAV)-based multi-sensor data can improve the prediction accuracy of crop yield. For this, five ML algorithms including Cubist, support vector machine (SVM), deep neural network (DNN), ridge regression (RR) and random forest (RF) were used for multi-sensor data fusion and ensemble learning for grain yield prediction in wheat. A set of thirty wheat cultivars and breeding lines were grown under three irrigation treatments i.e., light, moderate and high irrigation treatments to evaluate the yield prediction capabilities of a low-cost multi-sensor (RGB, multi-spectral and thermal infrared) UAV platform. Multi-sensor data fusion-based yield prediction showed higher accuracy compared to individual-sensor data in each ML model. The coefficient of determination (R ²) values for Cubist, SVM, DNN and RR models regarding grain yield prediction were observed from 0.527 to 0.670. Moreover, the results of ensemble learning through integrating the above models illustrated further increase in accuracy. The predictions of ensemble learning showed high R ² values up to 0.692, which was higher as compared to individual ML models across the multi-sensor data. Root mean square error (RMSE), residual prediction deviation (RPD) and ratio of prediction performance to inter-quartile range (RPIQ) were calculated to be 0.916 t ha^-1, 1.771 and 2.602, respectively. The results proved that low altitude UAV-based multi-sensor data can be used for early grain yield prediction using data fusion and an ensemble learning framework with high accuracy. This high-throughput phenotyping approach is valuable for improving the efficiency of selection in large breeding activities.

SUPPLEMENTARY INFORMATION

The online version contains supplementary material available at 10.1007/s11119-022-09938-8.

Machine learning-based estimation of riverine nutrient concentrations and associated uncertainties caused by sampling frequencies

Accurate and sufficient water quality data is essential for watershed management and sustainability. Machine learning models have shown great potentials for estimating water quality with the development of online sensors. However, accurate estimation is challenging because of uncertainties related to models used and data input. In this study, random forest (RF), support vector machine (SVM), and back-propagation neural network (BPNN) models are developed with three sampling frequency datasets (i.e., 4-hourly, daily, and weekly) and five conventional indicators (i.e., water temperature (WT), hydrogen ion concentration (pH), electrical conductivity (EC), dissolved oxygen (DO), and turbidity (TUR)) as surrogates to individually estimate riverine total phosphorus (TP), total nitrogen (TN), and ammonia nitrogen (NH₄⁺-N) in a small-scale coastal watershed. The results show that the RF model outperforms the SVM and BPNN machine learning models in terms of estimative performance, which explains much of the variation in TP (79 ± 1.3%), TN (84 ± 0.9%), and NH₄⁺-N (75 ± 1.3%), when using the 4-hourly sampling frequency dataset. The higher sampling frequency would help the RF obtain a significantly better performance for the three nutrient estimation measures (4-hourly > daily > weekly) for R² and NSE values. WT, EC, and TUR were the three key input indicators for nutrient estimations in RF. Our study highlights the importance of high-frequency data as input to machine learning model development. The RF model is shown to be viable for riverine nutrient estimation in small-scale watersheds of important local water security.

Assessing the utility of remote sensing data to accurately estimate changes in groundwater storage

Accurate and timely estimates of groundwater storage changes are critical to the sustainable management of aquifers worldwide, but are hindered by the lack of in-situ groundwater measurements in most regions. Hydrologic remote sensing measurements provide a potential pathway to quantify groundwater storage changes by closing the water balance, but the degree to which remote sensing data can accurately estimate groundwater storage changes is unclear. In this study, we quantified groundwater storage changes in California's Central Valley at two spatial scales for the period 2002 through 2020 using remote sensing data and an ensemble water balance method. To evaluate performance, we compared estimates of groundwater storage changes to three independent estimates: GRACE satellite data, groundwater wells and a groundwater flow model. Results suggest evapotranspiration has the highest uncertainty among water balance components, while precipitation has the lowest. We found that remote sensing-based groundwater storage estimates correlated well with independent estimates; annual trends during droughts fall within 15% of trends calculated using wells and groundwater models within the Central Valley. Remote sensing-based estimates also reliably estimated the long-term trend, seasonality, and rate of groundwater depletion during major drought events. Additionally, our study suggests that the proposed method estimate changes in groundwater at sub-annual latencies, which is not currently possible using other methods. The findings have implications for improving the understanding of aquifer dynamics and can inform regional water managers about the status of groundwater systems during droughts.