1. Introduction
1.1 Background
Water management is essential in the Netherlands due to more than half of the country being below sea level (Dutch Water Authorities, n.d.). In order to maintain safety, there are 22 regional water authorities in the Netherlands who are experts on water level management, water quality management, dredging, wastewater treatment and adapting to climate change. Regional water authorities have existed in the Netherlands since the 13th century, meaning that they already exist for about 800 years. This early existence has everything to do with the geographical location of the Netherlands. In the 13th century, the water level was as much a problem as it is now and dykes, canals and ditches were needed to protect the Dutch people from floods. Nowadays, the regional water authorities do more than build dykes to keep the water out; they have also become experts on cleaning wastewater. Wastewater treatment is something that has grown in importance during the last centuries. Nature has the ability to cope with small amounts of wastewater and pollution, but is not able to deal with the amount of wastewater that we produce nowadays. Wastewater is collected from households, where one can think of showers, toilets, washing machines and dishwashers, and from businesses and industries. It includes, amongst others, substances such as human waste, food residues, oils, soaps and chemicals. Additionally, precipitation is considered wastewater for the purpose of this thesis. Even though precipitation is clean, the runoff is not. Harmful substances that wash off from roads, parking lots and rooftops can harm our surface waters as well. For this reason, we need to reduce the pollutants in wastewater before returning it to the surface waters (The USGS Water Science School, 2016).
Definition 1 (surface water): water on the surface of the planet such as a river, lake, wetland or ocean (Aquolex, n.d).
If we were not to clean our wastewater, this would have a potential harmful effect on ecosystems and our health. Clean water is critical to plants and animals that live in water, and is a great playground for us. When water is not properly cleaned, water can carry diseases. Since we live, work and play so close to water, harmful bacteria have to be removed to make water safe.
Every moment of the day, wastewater flows into the Dutch sewer system. In most parts of the Netherlands, we have a combined sewer system, where runoff from rain and wastewater from households, business and industries are collected in the same sewer. The collected water is then pumped towards the different wastewater plants for treatment. When arrived at a plant, it undergoes several treatment processes before it returns to the surface water. The water that comes into the wastewater plants is called influent, where the water that is returned to the surface waters is called effluent. During the treatment process, a lot of energy is used. Singh, Carliell-Marquet & Kansal (2012) investigated the energy use of wastewater plants and discovered that a major source of energy consumption during the treatment process is the pumping of influent and effluent (79%) and Kusiak et al. reported that wastewater plants in the United States use 4% of the nation’s electricity. With growing climate concerns, energy saving and energy efficiency have become a common development principle. This offers opportunities for investigating how the energy use of these
pumps can be constrained.
Regional water authority Vechtstromen is one of the largest regional water authorities in the Netherlands and has 23 wastewater plants in their possession, spread out over 225,000 hectares and serves a population of 800,000 (Vechtstromen, n.d.). Vechtstromen acknowledges the opportunities that lie in optimizing the energy use of the water pumps and has made this problem available as a thesis project for the master Data Science: Business and Governance from Tilburg University.
1.2 Research Objective
To reduce energy use by increasing efficiency, regional water authority Vechtstromen is interested in investigating the possibilities of buffering and gradual influent flow rate. In this thesis, we will talk about flow rate, which is defined as follows:
Definition 2 (flow rate): the volume of water flowing through a section per unit of time. (Aquolex, n.d.). Unit of time is set to hour for this thesis.
Currently, wastewater is treated the moment it arrives at the wastewater plants via the water pumps placed throughout Vechtstromen. These wastewater plants differ in size, function and in the data they collect. For this reason, this thesis will focus only on two wastewater plants that are equal in their function and data collection. These wastewater plants are located in Oldenzaal and Losser, Overijssel. For this reason, when discussing wastewater plants, only the aforementioned two will be considered unless specified differently.
Definition 3 (wastewater plants): the wastewater plants in Oldenzaal and Losser, both located in Overijssel, the Netherlands.
Treating the water as soon as it arrives at the wastewater plants may not be the most efficient strategy. In continuously operated wastewater treatment plants, it is often desirable that the flow through the plant should vary as little as possible for optimal performance (Lindqvist, Wik, Lumley, & Äijälä, 2005). To achieve as little variance as possible, the storing capacity of the wastewater plants itself and in the sewer, can be used as buffering facility. However, storing wastewater at the wastewater plants and in the sewer, could cause problems when an unforeseen amount of wastewater arrives at the plant. If the plants are not able to process the stored and incoming water promptly, the streets are likely to get flooded. For example, in August 2017 the regional water authority Vechtstromen had problems with floods in Hengelo due to heavy rain (Tubantia, 2017). Predicting the influent flow rate would therefore be helpful. When being able to predict the influent flow rate, Vechtstromen can make maximum use of the buffer capacity. When the incoming flow rate is on the lower side, the wastewater can be buffered to gradually treat it later at a constant rate. Additionally, the wastewater could be buffered up until the night and gradually processed then, since the contract of Vechtstromen with the energy supplier includes a cheaper night rate.
1.3 Scientific and societal relevance
This current study yields scientific as well as societal relevance. The scientific relevance comes from the opportunity to create clarity on modelling of influent flow rate of wastewater plants. In history, different methods and algorithms have been used to develop a predictive model for this cause. Additionally, different researches have used different input features. This current study will take the different input features into account and test different models and algorithms. This way, the current study can prove clarification and homogeneity on study results. The societal relevance comes from a possible reduction of taxes and responsible use of the earth’s natural resources. The regional water authorities are embedded in the democratic structure of the Netherlands, which means that they are empowered to collect taxes. In 2016, Dutch citizens paid a total of 2,7 billion euros for regional water management, which equals to 8% of the total tax burden (Dutch Water Authorities, n.d.). Regional water authorities are authorized to determine the height of the annual taxes per capita and, therefore, the regional water authorities producing less costs per year, could prove to be beneficial for Dutch citizens. Less costs might lead to lower taxes, leaving households with more means. Additionally, all countries, with the exception of The United States of America, have agreed to partake in the Paris Agreement that was drawn up in 2015. The purpose was to bring all nations into a common cause to undertake ambitious efforts to combat climate change. An important part of the agreement is to handle our natural resources responsible (United Nations, 2015). Hence, energy efficiency is very important and predicting the hourly influent flow rate could contribute to this cause.
1.4 Research question
As mentioned in the previous sections, the aim of this thesis is to investigate if the influent flow rate can be predicted to allow for buffering of wastewater. Hence, the main research question is defined as follows:
Main research question: To what extent can influent flow rate be predicted for the wastewater plants of Vechtstromen?
Four sub questions are created, which aid in answering the main research question. The first sub research question aims to investigate how the influent flow rate differs during the day, as well as during the week. Having insights on how the influent flow rates fluctuates can give a basic estimate on what flow rate to expect on average, and therefore gives a baseline model that is already a little smarter than an overall mean model. Additionally, it provides primary insights to Vechtstromen which they are lacking at the moment. Hence, the first sub research question reads as follows:
Sub RQ 1: What are the average influent flow rates per part of the day (defined as morning, afternoon, evening and night), per day of the week?
To investigate which features contribute in predicting influent flow rates, a second sub research question is formulated. Insights into which features contribute can help to determine what the most important factors are when concerning influent flow rate. Hence, the second sub research question reads as follows:
Sub RQ 2: Which features contribute the most in predicting influent flow rates for the wastewater plants?
From the moment it rains or water is used in a household, it takes an average 2-3 hours before the wastewater reaches the plants. For this reason, investigating the influence of the chosen time period on the prediction performance can prove to be interesting. Hence, the third sub research question reads as follows:
Sub RQ 3: To what extent does the chosen time period of the prediction influence the
performance of the model?
When looking at previous research, classification, as well as regression, has been applied to model influent flow rate (Hosseini, 2011). Additionally, different learning algorithms have been applied to model the influent flow rate. Wei, Kusiak and Sadat (2013) applied different models, of which the Multilayer Perceptron outperformed a Random Forest, Boosted Tree and Support Vector Machine. However, Szelag, Bartkiewics, Studzínski and Barbusínki (2017) obtained different results and found that a support vector machine yielded the best results. A more in-depth review of previous used algorithms can be found under subsection 2.3 Models and algorithms. Due to the disunity in the scientific community on how to best model the influent flow rate of a wastewater plant, investigating which algorithm yields the best results for this current work can prove additional relevance and clarification. Hence, the fourth sub research question reads as follows:
Sub RQ 4: How does influent flow rate prediction performance depend on the chosen model?
1.5 Outline
The outline of the remainder of this thesis is structured as follows. Section 2 discusses related work on the subject of influent flow rate. Section 3 describes the method, also known as the experimental procedure. This section also includes a description of the data used. In Section 4, the results of the experiment are presented. Finally, Section 5 provides a general discussion of the results with regard to the research questions. Additionally, recommendations for future research are given.
2. Related work
In this section, relevant literature is discussed. Reviewing previous studies will help to place the research in a broad context, justifying the added value of this current work. Additionally, it will help to identify valuable features in advance and can provide to be helpful with selecting models and algorithms. The section starts with research concerning the possibilities of buffering wastewater in subsection 2.1. In subsection 2.2, the current state of wastewater influent flow rate prediction is discussed. The section concludes with models and algorithms adopted and used in this field of research.
2.1 Buffering of wastewater
“The best kWh is the one that is saved” (De Keulenaer et al., 2004). These researchers investigated that motor driven systems account for approximately 65% of the electricity consumed by EU industry. Implementing new, more efficient systems and improving the current ones would reduce the need for new power plants and would push down the total environmental cost of electricity generation. Pumping, together with other processes, represents more than half of the motor loads. For the pumping of wastewater, energy saving can be determined as improving the efficiency of energy use so that the consumption decreases. The goal is to use as little energy as possible per pumped amount of flow rate (Ruuskanen, 2007). A wastewater pumping system is a controlled system that pumps away the influent. The velocity of the pumps, the rate at which they process the influent, needs to match the flow rate. The inevitable adjustment of the velocity is energy expensive and therefore a variable flow rate may not be the most cost and energy effective approach. Ruuskanen (2007) formulated several different methods to act on variable influent flow rate in a cost and energy effective way, where the main method is the On-Off method. The influent flow rate of wastewater through the treatment process is controlled by switching the pump on or off. For this control method, it is essential to have storage capacity at the wastewater plant or in the sewage. The storage, so-called buffering, of wastewater allows to provide the wastewater plant with a steady flow through the treatment process. In order to effectuate buffering of wastewater, it is important to know the expected influent flow rate to give guidance on whether to buffer at the current time t or not.
2.2 Prediction of wastewater influent flow rate
To maintain a constant gradual flow of incoming wastewater, available buffer capacity in wastewater transport systems and plants can be used to achieve as such. Therefore, it is desirable to know in advance the influent flow rate to the wastewater plant. This allows for optimal scheduling of the pumps, which can in return reduce electrical usage (Wei & Kusiak, 2015). Due to the influent flow rate including both municipal sewage and rainfall runoff, it exhibits non-linear spatial and temporal behaviour which complicates the modelling. Several studies have focussed on modelling the future influent flow rate for wastewater plants. Wei et al. (2013) conducted a research into predicting the influent flow rate, while testing for different lengths of time. The model was constructed using radar reflectivity data, rainfall rate data and the historical influent flow rate data. They build seven prediction models at t + 15, t + 30, t + 60, t + 90, t + 120, t + 150, t + 180, where t is the current time and the added values are in minutes. They found that up until t + 30, the prediction error was stable, but after that increased rapidly. Where Wei et al. did include the precipitation in the areas of the wastewater plants, however, they did not take into account other weather factors that might influence the influent flow rate. Additionally, the volume of municipal sewage water was not taken into account, which could offer interesting insights into the average amount wastewater that households produce per hour on average on any given day. A low mean absolute error (MAE) of 1.09% is achieved, which shows that using data concerning rainfall and local geography are already helpful features for predicting influent flow rate. For both the research of Wei et al. (2013), and Wei and Kusiak (2015) a multi-layer perceptron (MLP) neural network model was used as final model. Several models were tested such as Random Forest, Boosted Tree and Support Vector Machine, however, a MLP yielded the best results.
Lindqvist et al. (2005) also investigated the extent to which influent flow rate could be predicted, but they took a different approach. In this case study, a separation is made between the influent flow rate during dry weather and influent flow rate with additional storm water runoff. Dry weather flow depends mainly on the time of the day as the activities of industries and households vary. Additionally, the flow pattern can differ on weekdays compared to the weekend since more people are at home during the weekend. First, they modelled the dry weather model. After this, they used the dry weather model to remove the periodic base from the incoming flow, giving a clearer view of the correlation between rainfall and runoff. Their results showed that the model is able to make good predictions for periods up to 24 hours in dry weather conditions. But, when adding precipitation predictions, a prediction up to 24 hours became rather inaccurate. For this reason, the prediction time when taking into account precipitation is set to two hours. The authors do not specify what their definition of ‘good predictions’ is, and therefore their results cannot be compared to the results of Wei et al. (2013), and Wei and Kusiak (2015).
Hosseini (2011) added memory parameters to improve the prediction accuracy. The memory parameters used where the influent rates t – 30, t – 60, t – 90 and t – 120, where t stands for the current time and the subtracted values are in minutes. These memory parameters, as anticipated, were found closely associated with the target output. Another research performed by Szelag et al. (2017) was conducted in Poland. Their research mainly focused on the determining the impact of explanatory variables on the accuracy of prediction of daily influent flow rates to wastewater plants in Poland. What others did not take into account, but what these researchers did, is the water levels in the Wislok river (sizable river in Poland). They followed the example of Hosseini and also added memory parameters of the influent flow rate, but also concerning the water levels of the river and the rainfall. They included t – 1, t – 2, where t stands for the current day and the subtracted values are in days, for all three different variables. Their developed correlation coefficient table shows that the highest correlation is between the current influent flow rate, the influent flow rate from the day before and the day before that. Additionally, the water level from the Wislok of yesterday and two days ago also shows a significant correlation, however, the influence of rainfall data of the past two days is insignificant.
Where previous researchers have investigated building a predictive model for influent flow rates, and which features contribute to the predictive model, a limited number of features, as well as a limited number of different models is used. Szelag et al. (2017) researched the most features, but did not take into account for which period of time the model performed best, which is something other researchers as Hosseini (2011), Wei et al. (2013), and Wei and Kusiak (2015) did considered. In the next subsection, the models and algorithms used in previous research on influent flow rate prediction are discussed.
2.3 Models and algorithms
In order to provide an answer to the fourth research question How does influent flow rate prediction performance depend on the chosen learning algorithm? algorithms used in previous research are reviewed. Numerous models and learning algorithms have been adopted for modelling influent flow rate and both classification and regression have found their application. This is in agreement with the so-called No-Free-Lunch (NFL) theorem developed by Wolpert (2001). The NFL theorem states that there is no model that works best for every problem. For this reason, it is common in machine learning to try multiple models and find the one that works best for the current problem. Additionally, one can train a model with multiple algorithms to find the best performing one. For example, a Linear Regression could be trained by Ordinary Least Squares (OLS) or by Gradient Descent. The different models considered are Multilayer Perceptron (MLP), Boosted Tree, Support Vector Machine (SVM), K-Nearest Neighbours (KNN), derived from previous research. Additionally, an ARIMA model will be fitted. This has not been widely applied for predicting influent flow rates, but since the model handles time series data well, it could prove to be useful and is worth exploring. This current work will only consider regression models since a continuous outcome gives a more precise outcome, which is desired by the regional water authority.
The first model that is considered is the MLP that is used in the previous research (Hosseini, 2011; Wei & Kusiak, 2015; Wei et al., 2013) for predicting wastewater influent flow rate. An MLP is a neural network that learns in a supervised manner and is capable of modelling highly non-linear functions. It consists of multiple layers of nodes, where each layer is fully connected to the next. An MLP usually consists out of an input layer, an output layer and a number of hidden layers. Different activation functions, including logistic, hyperbolic tangent, identity and rectified linear unit are used for the hidden and output layers (Gardner & Dorling, 1998). One main advantage of MLP is that it is capable of learning temporal features in data. For this reason, the MLP is used frequently in time series prediction. However, a limitation is that it can only perform prediction of a stationary time series, which is when its statistics do not change over time (e.g. not influenced by seasonal trends), while many real-world problems are not stationary (Koskela, Lehtokangas, Saarinen, & Kaski, 1996). Another drawback is that the model tends to get stuck in a ‘local minimum’ when the data is not high dimensional enough. So far, the MLP has proven its applicability for wastewater influent flow rate prediction, since the model is the best performing one for the research of Hosseini (2011) and Wei et al. (2013).
The second model considered is the Boosted Tree, which is implemented by Wei et al. (2013) and by Kusiak, Zeng and Zhang (2013). Kusiak et al. investigated the performance of a pumping system in the wastewater treatment process, where they modelled the energy consumption and the flow rate. Modelling was done with Boosted Trees, which is a so-called ensemble method. It combines the strengths of two algorithms; regression trees and boosting (a method for combining multiple simple models to achieve improved predictive strength), which can significantly reduce variance and bias (Elith, Leathwick, & Hastie, 2008). For the research of Wei et al., the Boosted Tree was outperformed by the MLP and SVM with an average MAE of 1.77% and MSE of 11.16%. The same applies for Kusiak et al., the Boosted Tree performed well, but is outperformed by other models such as the MLP. However, since a MLP is prone to overfitting and a Boosted Tree is more robust due to the ensemble nature of the model, the Boosted Tree should not be overlooked as a possible strong performing model.
A SVM is the third model considered and is a method used for regression, as well as classification. When using SVM for regression, an alternative loss function must be added to include a distance measure. This involves a kernel, where the kernel determines how similar different features are with respect to each other, and thus imparts weights to their corresponding loss function. Features that are close to each other and have the same output will get grouped together due to more weight. In addition, the name SVM is changed to SVR, referring to Support Vector Regression. One advantage of SVR over MLP is the absence of local minima, which could prove to be useful when dealing with non-high dimensional data (Gunn, 1998). In the research of Wei et al. (2013), SVR proved to be the best performing model after the MLP and the research of Hosseini (2011) yielded the same result. However, for Szelag et al. the SVR model was the best performing one, meaning that the smallest prediction error was obtained when using the SVR model.
The fourth model considered is the KNN algorithm. Kim, Kim, Kim, Piao and Kim (2015) researched to what extent KNN can forecast the influent characteristics of wastewater treatment plants, where one of the characteristics was defined as the influent flow rate. With KNN, the influent flow rate was accurately predicted within one standard deviation of measured values. The KNN method approaches a complex non-linear time series by applying the concept of chaos theory. The chaos theory posits that some part of a time series occurring in the past can occur in the future with highly similar characteristics. Since the chaos theory applies to wastewater influent flow rate, the KNN method is an interesting one to apply to model this problem. The KNN method makes prediction in a relative simple way; it stores all available cases and predicts the numerical target based on a similarity metric, for example distance functions such as Euclidean, Manhattan or Cosine. Szelag et al. (2017) also applied the KNN algorithm for wastewater influent prediction. The researchers made use of memory parameters for the influent flow rate, water level of the nearby river Wislok, and rainfall. When only adding the memory parameters of one day in the past, KNN was the best performing model beating SVM, Random Forest and the Kernel method. However, when adding more memory parameters, SVM outperformed KNN and other models on almost all occasions. This again complies with the NFL theorem that states that different models are suited for different data and that there is no one best performing model.
The last model considered is ARIMA, which stands for Autoregressive Integrated Moving Average (Montgomery, Jennings, & Kulahci, 2015). ARIMA is derived from the ARMA model (autoregressive moving average) but ARIMA suits better when data shows evidence of non-stationarity. One of the disadvantages of MLP, which was often the best performing model for predicting wastewater influent flow rate, is that it cannot deal with non-stationary data. For this reason, ARIMA is an interesting model to investigate. The autoregressive part of the model indicates that the output variable is regressed on its own prior values and the moving average part of the model indicates that the resulting regression error is actually a linear combination of error term of which the values occurred contemporaneously and at various times in the past. The integrated part of the model means that the data values have been replaced with the difference between their values and the previous values.
Despite the sometimes-contradicting findings in the literature, the MLP seems to be the most popular and powerful method for wastewater influent flow prediction. However, other researchers have found different models such as Boosted Tree, Support Vector Machine and K-Nearest Neighbours to perform the best. For this reason, the aforementioned different models will be tested, along with an ARIMA model, to see if again the MLP outperforms others or that different results are achieved.
Most of the times, the models were evaluated using mean absolute error (MAE), mean squared error (MSE), root mean squared error (RMSE) and mean absolute percentage error (MAPE). The same evaluation metrics will be applied in this current work in order to compare the results to previous research outcomes.