Simulating ensembles of source water quality using a K-nearest neighbor resampling approach

A data-driven approach for understanding the structure dependence of redox activity in humic substances

Humic substances (HS) can facilitate electron transfer during biogeochemical processes due to their redox properties, but the structure-redox activity relationships are still difficult to describe and poorly understood. Herein, the linear (Partial Least Squares regressions; PLS) and nonlinear (artificial neural network; ANN) models were applied to monitor the structure dependence of HS redox activities in terms of electron accepting (EAC), electron donating (EDC) and overall electron transfer capacities (ETC) using its physicochemical features as input variables. The PLS model exhibited a moderate ability with R² values of 0.60, 0.53 and 0.65 to evaluate EAC, EDC and ETC, respectively. The variable influence in the projection (VIP) scores of the PLS identified that the phenols, quinones and aromatic systems were particularly important for describing the redox activities of HS. Compared with the PLS model, the back-propagation ANN model achieved higher performance with R² values of 0.81, 0.65 and 0.78 for monitoring the EAC, EDC and ETC, respectively. Sensitivity analysis of the ANN separately identified that the EAC highly depended on quinones, aromatics and protein-like fluorophores, while the EDC depended on phenols, aromatics and humic-like fluorophores (or stable free radicals). Additionally, carboxylic groups were the best indicator for evaluating both the EAC and EDC. Good model performances were obtained from the selected features via the PLS and sensitivity analysis, further confirming the accuracy of describing the structure-redox activity relationships with these analyses. This study provides a potential approach for identifying the structure-activity relationships of HS and an efficient machine-learning model for predicting HS redox activities.

Prediction of water quality extremes with composite quantile regression neural network

Water quality extremes, which water quality models often struggle to predict, are a grave concern to water supply facilities. Most existing water quality models use mean error functions to maximize the predictability of water quality mean value. This paper describes a composite quantile regression neural network (CQRNN) model, which simultaneously estimates non-crossing regression quantiles by minimizing the composite quantile regression error function. This method can improve the prediction of extremes. This paper evaluates the performance of CQRNN for predicting extreme values of turbidity and total organic carbon (TOC) and compares with quantile regression (QR), linear regression (LR), and k-nearest neighbors (KNN) in an application to the Hetch Hetchy Regional Water System, which is the primary water supply for San Francisco, CA. CQRNN is superior to QR, LR, and KNN for predicting the mean trend and extremes of turbidity and TOC, especially for the non-Gaussian turbidity data. The performance of CQRNN is the most stable relative to other methods over different training sample sizes.

Can water composition and weather factors predict fecal indicator bacteria removal in retention ponds in variable weather conditions?

Retention ponds provide benefits including flood control, groundwater recharge, and water quality improvement, but changes in weather conditions could limit the effectiveness in improving microbial water quality metrics. The concentration of fecal indicator bacteria (FIB), which is used as regulatory standards to assess microbial water quality in retention ponds, could vary widely based on many factors including local weather and influent water chemistry and composition. In this critical review, we analyzed 7421 data collected from 19 retention ponds across North America listed in the International Stormwater BMP Database to examine if variable FIB removal in the field conditions can be predicted based on changes in these weather and water composition factors. Our analysis confirms that FIB removal in retention ponds is sensitive to weather conditions or seasons, but temperature and precipitation data may not describe the variable FIB removal. These weather conditions affect suspended solid and nutrient concentrations, which in turn could affect FIB concentration in the ponds. Removal of total suspended solids and total P only explained 5% and 12% of FIB removal data, respectively, and TN removal had no correlation with FIB removal. These results indicate that regression-based modeling with a single parameter as input has limited use to predict FIB removal due to the interactive nature of their effects on FIB removal. In contrast, machine learning algorithms such as the random forest method were able to predict 65% of the data. The overall analysis indicates that the machine learning model could play a critical role in predicting microbial water quality of surface waters under complex conditions where the variation of both water composition and weather conditions could deem regression-based modeling less effective.

Prediction of Nitrate and Phosphorus Concentrations Using Machine Learning Algorithms in Watersheds with Different Landuse

Rapid industrialization and population growth have elevated the concerns over water quality. Excessive nitrates and phosphates in the water system have an adverse effect on the aquatic ecosystem. In recent years, machine learning (ML) algorithms have been extensively employed to estimate water quality over traditional methods. In this study, the performance of nine different ML algorithms is evaluated to predict nitrate and phosphorus concentration for five different watersheds with different land-use practices. The land-use distribution affects the model performance for all methods. In urban watersheds, the regular and predictable nature of nitrate concentration from wastewater treatment plants results in more accurate estimates. For the nitrate prediction, ANN outperforms other ML models for the urban and agricultural watersheds, while RT-BO performs well for the forested Grand watershed. For the total phosphorus prediction, ensemble-BO and M-SVM outperform other ML models for the agricultural and forested watershed, while the ANN performs better than other ML models for the urban Cuyahoga watershed. In predicting phosphorus concentration, the model predictability is better for agricultural and forested watersheds. Regarding consistency, Bayesian optimized RT, ensemble, and GPR consistently yielded good performance for all watersheds. The methodology and results outlined in this study will assist policymakers in accurately predicting nitrate and phosphorus concentration which will be instrumental in drafting a proper plan to deal with the problem of water pollution.

Relative performance of different data mining techniques for nitrate concentration and load estimation in different type of watersheds

The increasing availability of water quality datasets has led to a greater focus on hydrologic and water quality analysis, thus requiring more efficient and accurate modelling methods. Data mining techniques have been increasingly used for water quality analysis and prediction of the concentration and load of nitrogen pollutants instead of more traditional simulation methods. In this study, we tested the multilayer perceptron (MLP), k-nearest neighbor (k-NN), random forest, and reduced error pruning tree (REPTree) methods, along with the traditional linear regression, to predict nitrate levels based on long-term data from six watersheds with different land-use practices in the midwestern United States. Both the concentration and load results indicated that REPTree had the best performance, with an R² of 0.61-0.85 and a relative absolute error of <75.8%. The different watershed types, however, influenced the performance of the data mining methods, where all four methods showed a higher accuracy for urban dominant watershed and lower accuracy for agricultural and forest watersheds. Out of these four methods, classification tree methods (REPTree and RF) performed better than cluster methods (MLP and k-NN) for agricultural and forested watersheds. Our results indicated that both the data structure based on the dominant land use and type of algorithmic method should be carefully considered for selecting a data mining method to predict nitrate concentration and load for a watershed.

Water quality assessment with emphasis in parameter optimisation using pattern recognition methods and genetic algorithm

A non-supervised (k-means) and a supervised (k-Nearest Neighbour in combination with genetic algorithm optimisation, k-NN/GA) pattern recognition algorithms were applied for evaluating and interpreting a large complex matrix of water quality (WQ) data collected during five years (2008, 2010-2013) in the Paute river basin (southern Ecuador). 21 physical, chemical and microbiological parameters collected at 80 different WQ sampling stations were examined. At first, the k-means algorithm was carried out to identify classes of sampling stations regarding their associated WQ status by considering three internal validation indexes, i.e., Silhouette coefficient, Davies-Bouldin and Caliński-Harabasz. As a result, two WQ classes were identified, representing low (C1) and high (C2) pollution. The k-NN/GA algorithm was applied on the available data to construct a classification model with the two WQ classes, previously defined by the k-means algorithm, as the dependent variables and the 21 physical, chemical and microbiological parameters being the independent ones. This algorithm led to a significant reduction of the multidimensional space of independent variables to only nine, which are likely to explain most of the structure of the two identified WQ classes. These parameters are, namely, electric conductivity, faecal coliforms, dissolved oxygen, chlorides, total hardness, nitrate, total alkalinity, biochemical oxygen demand and turbidity. Further, the land use cover of the study basin revealed a very good agreement with the WQ spatial distribution suggested by the k-means algorithm, confirming the credibility of the main results of the used WQ data mining approach.

Assessment of wastewater treatment facility compliance with decreasing ammonia discharge limits using a regression tree model

A regression tree-based diagnostic approach is developed to evaluate factors affecting US wastewater treatment plant compliance with ammonia discharge permit limits using Discharge Monthly Report (DMR) data from a sample of 106 municipal treatment plants for the period of 2004-2008. Predictor variables used to fit the regression tree are selected using random forests, and consist of the previous month's effluent ammonia, influent flow rates and plant capacity utilization. The tree models are first used to evaluate compliance with existing ammonia discharge standards at each facility and then applied assuming more stringent discharge limits, under consideration in many states. The model predicts that the ability to meet both current and future limits depends primarily on the previous month's treatment performance. With more stringent discharge limits predicted ammonia concentration relative to the discharge limit, increases. In-sample validation shows that the regression trees can provide a median classification accuracy of >70%. The regression tree model is validated using ammonia discharge data from an operating wastewater treatment plant and is able to accurately predict the observed ammonia discharge category approximately 80% of the time, indicating that the regression tree model can be applied to predict compliance for individual treatment plants providing practical guidance for utilities and regulators with an interest in controlling ammonia discharges. The proposed methodology is also used to demonstrate how to delineate reliable sources of demand and supply in a point source-to-point source nutrient credit trading scheme, as well as how planners and decision makers can set reasonable discharge limits in future.

Assessment the performance of classification methods in water quality studies, A case study in Karaj River

To show the performance of classification methods in water quality studies, linear discriminant, and Naïve Bayesian classification methods were applied at nine sampling stations with respect to four parameters including COD, nitrite, nitrate, and total coliforms (selected from ten water quality variables) in Karaj River, Iran. To fulfill the goals of this study, the sampling stations were first separated into two groups using cluster analysis. Rural wastewater was the main source of pollution in the first group, whereas the quality of water in the second group has been degraded mainly by organic and agricultural pollution. In order to have an independent group against which the performance of other classification methods is considered, three cross-validation methods including twofold, leave-one-out, and holdout methods were utilized to retain an independent test set. The results of cross-validation for the linear discriminant analysis show that, except for the leave-one-out method with 11.1 % misclassification error, the overall performance has been the same as that of the training data set. Therefore, it has outperformed compared with that of Naïve Bayesian classification method. However, even though in situations where the correlation coefficient among the parameters is low, the latest method can offer the same performance as that of linear discriminant analysis as well. A sensitivity analysis was implemented using ten water quality variables (pH, COD, EC, TDA, turbidity, nitrate, nitrite, sulfate, TC, and FC) to find the most important variables in the classification of Karaj River showing that turbidity, next to COD, pH, nitrate, and sulfate, have had the most contribution in this field.