evaluation of different objective functions used in the
Post on 22-Oct-2021
4 Views
Preview:
TRANSCRIPT
water
Article
Evaluation of Different Objective Functions Used inthe SUFI-2 Calibration Process of SWAT-CUP onWater Balance Analysis: A Case Study of the PursatRiver Basin, Cambodia
Davy Sao 1 , Tasuku Kato 2,*, Le Hoang Tu 1,3, Panha Thouk 4, Atiqotun Fitriyah 5 andChantha Oeurng 6
1 Department of Agricultural and Environmental Engineering, United Graduate School ofAgricultural Science, Tokyo University of Agriculture and Technology, 3-8-5 Saiwai-cho, Fuchu-shi,Tokyo 183-8538, Japan; davy_sao@hotmail.com (D.S.); tugis07@gmail.com (L.H.T.)
2 Institute of Agriculture, Tokyo University of Agriculture and Technology, 3-8-5 Saiwai-cho, Fuchu-shi,Tokyo 183-8538, Japan
3 Research Center for Climate Change, Nong Lam University, HoChiMinh 700000, Vietnam4 Department of International Environmental and Agricultural Science, Tokyo University of Agriculture
and Technology, 3-8-5 Saiwai-cho, Fuchu-shi, Tokyo 183-8538, Japan; panha.thouk@gmail.com5 Institute of Global Innovation Research, Tokyo University of Agriculture and Technology, 3-8-5 Saiwai-cho,
Fuchu-shi, Tokyo 183-8538, Japan; atiqotun.fitriyah@gmail.com6 Institute of Technology of Cambodia, Faculty of Hydrology and Water Resources Engineering, PO Box 86,
Russian Confederation Boulevard, Phnom Penh 12156, Cambodia; chantha@itc.edu.kh* Correspondence: taskkato@cc.tuat.ac.jp
Received: 8 September 2020; Accepted: 15 October 2020; Published: 17 October 2020�����������������
Abstract: Many calibration techniques have been developed for the Soil and Water Assessment Tool(SWAT). Among them, the SWAT calibration and uncertainty program (SWAT-CUP) with sequentialuncertainty fitting 2 (SUFI-2) algorithm is widely used and several objective functions have beenimplemented in its calibration process. In this study, eight different objective functions were used in acalibration of stream flow of the Pursat River Basin of Cambodia, a tropical monsoon and forestedwatershed, to examine their influences on the calibration results, parameter optimizations, and waterresources estimations. As results, many objective functions performed better than satisfactory incalibrating the SWAT model. However, different objective functions defined different fitted valuesand sensitivity rank of the calibrated parameters, except Nash–Sutcliffe efficiency (NSE) and ratioof standard deviation of observations to root mean square error (RSR) which are equivalent andproduced quite identical simulation results including parameter sensitivity and fitted parameter values,leading to the same water balance components and water yields estimations. As they generatedreasonable fitted parameter values, either NSE or RSR gave better estimation results of annualaverage water yield and other water balance components such as annual average evapotranspiration,groundwater flow, surface runoff, and lateral flow according to the characteristics of the river basinand the results and data of previous studies. Moreover, either of them was also better in calibratingbase flow, falling limb, and overall the entire flow phases of the hydrograph in this area.
Keywords: SWAT model; SUFI-2; objective functions; calibration; water balance components
Water 2020, 12, 2901; doi:10.3390/w12102901 www.mdpi.com/journal/water
Water 2020, 12, 2901 2 of 22
1. Introduction
Several distributed hydrological models have been increasingly and widely employed, which areextremely essential to simulate hydrological phenomena, to manage and plan water resources,to investigate sedimentation and water quality, and to forecast the impact of universal climate andland-use changes. The Soil and Water Assessment Tool (SWAT) is one of the most well-knownand has demonstrated its strengths in the above aspects with a large and growing number ofmodel application in various studies ranging from catchment to continental scales [1–5]. It is opensource software developed by the U.S. Department of Agriculture (USDA) Agricultural ResearchService and Texas A&M. This physically based, watershed-scale, continuous model is commonlyapplied for measuring the impact of land-use and climate changes and for determining the differentwatershed management activities on hydrology, sedimentation, and water quality [6–10]. However,proper calibration/validation and uncertainty analysis are required to improve the model simulationand analysis for the watershed.
Calibration and uncertainty analysis are intimately linked, and no calibration results should bepresented without quantification of the degree of uncertainty of the model prediction [11]. To run aphysically based distributed hydrological model, such as SWAT, various parameters must be calibratedbecause of measurement difficulties. Calibration is performed by carefully selecting values with theirrespective uncertainty ranges and by comparing the simulation output for a given set of measureddata [7]. Several calibration techniques have been developed for SWAT, including manual andautomated calibration procedures using the shuffled complex evolution method and other commonmethods. The SWAT calibration and uncertainty program (SWAT-CUP) links sequential uncertaintyfitting 2 (SUFI-2), generalized likelihood uncertainty estimation (GLUE), parameter solution (ParaSol),Markov chain Monte Carlo (MCMC), and particle swarm optimization (PSO) algorithms to SWAT [12]and has been widely used for calibration and uncertainty analysis [13–15].
Among these algorithms, SUFI-2 has gained the most popularity among users for carrying outparameterization, sensitivity analysis, calibration, validation, and uncertainty analysis of hydrologicalparameters [16]. This is due to the availability of many parameters regarding water balance modeling,the easier calibration process to carry out within realizable time bounds, better accountability foruncertainties, and the smallest number of computational runs to achieve good prediction uncertaintybands and model performance [17–19]. However, many objective functions have been incorporatedinto the SUFI-2 calibration procedure, but most studies have not considered the influence of the choiceof objective functions on the calibration results, the calibrated parameter values/ranges, or the estimatedwater balance components. One study conducted in two Iranian watersheds by Kouchi et al. [20] waslikely the only one thus far that has compared the choice of objective functions during the calibrationprocess in SUFI-2. That study showed that all objective functions used in the calibration had a similarperformance; however, each produced notably different parameter ranges and consequently led tosignificant differences in the water resource estimates.
Thus in this study, the objectives were to examine the influences of eight different objectivefunctions in the calibration process of SWAT-CUP using the SUFI-2 algorithm on the calibration results,parameter optimizations, and water balance component estimations when simulating the stream flowof the tropical monsoon and forested watershed of the Pursat River Basin of Cambodia. Those objectivefunctions include the coefficient of determination (R2), modified coefficient of determination (bR2),Nash–Sutcliffe efficiency (NSE), modified Nash–Sutcliffe efficiency (MNS), ratio of standard deviationof observations to root mean square error (RSR), ranked sum of squares (SSQR), Kling–Gupta efficiency(KGE), and percent bias (PBIAS). We also aimed to identify a proper objective function which producedreasonable calibration results, calibrated parameter sets, and water resources estimations whichreflected the characteristics of this river basin.
Water 2020, 12, 2901 3 of 22
2. Study Area
The Pursat River Basin, located in Pursat Province of Cambodia, is one of the Tonle Sap sub-basinsand drains an area of 5955 km2 [21] (Figure 1a). The river flows for approximately 150 km from theSouthwest at the Cardamom Mountains to the Northeast direction until the Tonle Sap Lake with a basinelevation ranging between 6 and 1717 m above sea level. There is an operating hydrological stationnamed Bak Trakuon with a drainage area of 4245 km2 [22] where this study focused on (Figure 1b).More than 75% of the river basin encompasses a hilly terrain, with an elevation greater than 30 mabove sea level and is covered by forested land of varying density, while the remaining low-lying landis occupied by agriculture [23]. Those over 75% of the forest cover are mainly concentrated in BakTrakuon drainage boundary (Figure 1b) where the major soil types are Dystric Leptosols and DystricLeptosols/Dystric Cambisols (LPd and LPd/CMd) and Gleyic Acrisols/Plinthic Acrisols (ACg/ACp) [22].The climate in Pursat River Basin is influenced by tropic monsoon systems with distinct wet anddry seasons. The wet season, which extends from May to November, receives approximately 90% ofthe total annual rainfall. The dry season, which extends from December to April, is characterizedby the prevalence of hot and dry air [22]. The rainfall in the area increases with elevation, but theannual totals vary considerably from year to year with the average ranging from 1200 to 1700 mm [24].Daily maximum temperatures vary between 36 ◦C during the hottest months (April–May) and 32 ◦Cduring the coldest months (December–January). Daily minimum temperatures vary between 25 and17 ◦C. The annual average temperature is approximately 28 ◦C. The monthly mean relative humidityranges from 66% in the dry season to 71% in the wet season with a mean annual humidity of 70% [22].
Water 2020, 12, x FOR PEER REVIEW 3 of 22
the Southwest at the Cardamom Mountains to the Northeast direction until the Tonle Sap Lake with
a basin elevation ranging between 6 and 1717 m above sea level. There is an operating hydrological
station named Bak Trakuon with a drainage area of 4245 km2 [22] where this study focused on (Figure
1b). More than 75% of the river basin encompasses a hilly terrain, with an elevation greater than 30
m above sea level and is covered by forested land of varying density, while the remaining low-lying
land is occupied by agriculture [23]. Those over 75% of the forest cover are mainly concentrated in
Bak Trakuon drainage boundary (Figure 1b) where the major soil types are Dystric Leptosols and
Dystric Leptosols/Dystric Cambisols (LPd and LPd/CMd) and Gleyic Acrisols/Plinthic Acrisols
(ACg/ACp) [22]. The climate in Pursat River Basin is influenced by tropic monsoon systems with
distinct wet and dry seasons. The wet season, which extends from May to November, receives
approximately 90% of the total annual rainfall. The dry season, which extends from December to
April, is characterized by the prevalence of hot and dry air [22]. The rainfall in the area increases with
elevation, but the annual totals vary considerably from year to year with the average ranging from
1200 to 1700 mm [24]. Daily maximum temperatures vary between 36 °C during the hottest months
(April–May) and 32 °C during the coldest months (December–January). Daily minimum
temperatures vary between 25 and 17 °C. The annual average temperature is approximately 28 °C.
The monthly mean relative humidity ranges from 66% in the dry season to 71% in the wet season
with a mean annual humidity of 70% [22].
Figure 1. (a) Location of the Pursat River Basin and (b) land-use map in 2003 with monitoring stations.
3. Materials and Methods
Figure 2 shows the workflow diagram for this study. To simulate hydrological processes in the
river basin using the SWAT model, input data, such as DEM, soil type, land-use map, and climate
data, were required. The simulation output of the model without any calibration is called the
default/initial result. The initial result was checked by using the SWAT Check program to identify
any potential model problems, and the simulated discharge was compared with the observed data.
If there was a problem or a huge difference between the simulated and observed data, the model
inputs (e.g., spatial data, rainfall) need to be focused on, and if there were no problems, model
calibration/uncertainty analysis was performed. SWAT-CUP with SUFI-2 algorithm was employed
to conduct the model calibration/uncertainty analysis. This process started with parameterization in
which several parameters were selected with indication of their value ranges. This was based on the
difference between the simulated and observed data, watershed characteristics, and literature
reviews. The next step involved applying different objective functions before beginning the
calibration and validation. The results of the calibration and validation were evaluated. If the criteria
was not satisfied, parameterization was rechecked; then, the model was recalibrated and revalidated.
After the calibration and validation process using different objective functions, each objective
function was evaluated based on its calibration result, calibrated parameters, the estimation of
Bak Trakuon
Kravanh
(b) (a)
Pursat
Figure 1. (a) Location of the Pursat River Basin and (b) land-use map in 2003 with monitoring stations.
3. Materials and Methods
Figure 2 shows the workflow diagram for this study. To simulate hydrological processesin the river basin using the SWAT model, input data, such as DEM, soil type, land-use map,and climate data, were required. The simulation output of the model without any calibration iscalled the default/initial result. The initial result was checked by using the SWAT Check programto identify any potential model problems, and the simulated discharge was compared with theobserved data. If there was a problem or a huge difference between the simulated and observeddata, the model inputs (e.g., spatial data, rainfall) need to be focused on, and if there were noproblems, model calibration/uncertainty analysis was performed. SWAT-CUP with SUFI-2 algorithmwas employed to conduct the model calibration/uncertainty analysis. This process started withparameterization in which several parameters were selected with indication of their value ranges.This was based on the difference between the simulated and observed data, watershed characteristics,and literature reviews. The next step involved applying different objective functions before beginningthe calibration and validation. The results of the calibration and validation were evaluated. If the criteria
Water 2020, 12, 2901 4 of 22
was not satisfied, parameterization was rechecked; then, the model was recalibrated and revalidated.After the calibration and validation process using different objective functions, each objective functionwas evaluated based on its calibration result, calibrated parameters, the estimation of dischargeprocesses, and the simulation of the hydrograph components. The details of each step are described inthe following sections.
Water 2020, 12, x FOR PEER REVIEW 4 of 22
discharge processes, and the simulation of the hydrograph components. The details of each step are
described in the following sections.
Figure 2. Workflow diagram.
3.1. SWAT Input Datasets
Input data, such as topography, land-use map, soil type, and weather data, were collected from
different sources and agencies, as listed in Table 1. The digital elevation model (DEM) was from the
Shuttle Radar Topography Mission (SRTM) Global with a 30 m raster resolution and downloaded
from OpenTopography (https://www.opentopography.org/). The land-use and soil maps from 2003
were obtained from Cambodia National Mekong Committee (CNMC) with a 250-m raster resolution.
Weather and hydrological data were collected from the Pursat Provincial Department of Water
Resources and Meteorology. There are several rainfall stations nearby and within the river basin, but
one station, Kravanh, was selected because it is the only station that has lengthy data recorded from
the 1990s and is located in the drained area of the Bak Trakuon outlet (discharge monitoring station
used for model evaluation) (Figure 1b). Because of this limitation, a uniform distribution of rainfall
in the Kravanh station was applied to the model simulation. For minimum and maximum air
temperatures, the dataset in Pursat station located downstream of the river basin was used because
it was the only available temperature station in the study area. Other climate data, such as relative
humidity, wind speed, and solar radiation, were not available and were simulated by a weather
generator. In this study area, the daily river discharge data were the only available hydrological data
and were used for model calibration and evaluation.
Table 1. Description of data used in this study.
Data Description Year/Period Source
Digital
Elevation
Model (DEM)
Shuttle Radar Topography
Mission (SRTM) Global:
Raster resolution of 30 m
- OpenTopography
(https://www.opentopography.org/)
Land-use map Raster resolution of 250 m 2003 Cambodia National Mekong
Committee
Soil data Raster resolution of 250 m - Cambodia National Mekong
Committee
Weather data Daily rainfall at Kravanh
station 1994–2015
Pursat Provincial Department of
Water Resources and Meteorology
Figure 2. Workflow diagram.
3.1. SWAT Input Datasets
Input data, such as topography, land-use map, soil type, and weather data, were collectedfrom different sources and agencies, as listed in Table 1. The digital elevation model (DEM) wasfrom the Shuttle Radar Topography Mission (SRTM) Global with a 30 m raster resolution anddownloaded from OpenTopography (https://www.opentopography.org/). The land-use and soil mapsfrom 2003 were obtained from Cambodia National Mekong Committee (CNMC) with a 250-m rasterresolution. Weather and hydrological data were collected from the Pursat Provincial Department ofWater Resources and Meteorology. There are several rainfall stations nearby and within the river basin,but one station, Kravanh, was selected because it is the only station that has lengthy data recordedfrom the 1990s and is located in the drained area of the Bak Trakuon outlet (discharge monitoringstation used for model evaluation) (Figure 1b). Because of this limitation, a uniform distribution ofrainfall in the Kravanh station was applied to the model simulation. For minimum and maximum airtemperatures, the dataset in Pursat station located downstream of the river basin was used becauseit was the only available temperature station in the study area. Other climate data, such as relativehumidity, wind speed, and solar radiation, were not available and were simulated by a weathergenerator. In this study area, the daily river discharge data were the only available hydrological dataand were used for model calibration and evaluation.
3.2. SWAT Model Setup
In this study, the ArcSWAT 2012 interface was used to setup the model. With a threshold drainagearea of 5000 ha, the basin was discretized into 60 sub-basins, and by classing the slope below and above5%, the sub-basins were further subdivided into 667 HRUs (Hydrologic Response Units). The modelthen required the inputs of weather data before it could be run. For this study, the model was run in amonthly time step for a total simulation period of 26 years from 1990 to 2015 in which the period from1990 to 1994 was treated as the warm-up period to mitigate the initial conditions and was not includedin the analysis.
Water 2020, 12, 2901 5 of 22
Table 1. Description of data used in this study.
Data Description Year/Period Source
Digital ElevationModel (DEM)
Shuttle Radar TopographyMission (SRTM) Global: Raster
resolution of 30 m- OpenTopography (https:
//www.opentopography.org/)
Land-use map Raster resolution of 250 m 2003 Cambodia NationalMekong Committee
Soil data Raster resolution of 250 m - Cambodia NationalMekong Committee
Weather data Daily rainfall at Kravanh station 1994–2015 Pursat Provincial Department ofWater Resources and Meteorology
Daily maximum and minimumtemperature at Pursat station 2001–2015 Pursat Provincial Department of
Water Resources and Meteorology
Hydrological data Daily discharge at BakTrakuon station 1995–2015 Pursat Provincial Department of
Water Resources and Meteorology
3.3. SWAT-CUP with the SUFI-2 Algorithm
Calibration can be difficult and is almost infeasible for many large-scale applications [7]. A numberof auto-calibration and uncertainty analysis tools for SWAT have been developed to solve this problemand are currently available to assist the optimization process. In this study, SWAT-CUP with theSUFI-2 algorithm was used to calibrate the model. SWAT-CUP is an auto-calibration and uncertaintyanalysis module program developed for SWAT [12,25]. SWAT-CUP allows the use of several algorithmsfor the calibration or validation procedure. Among them, the SUFI-2 algorithm [16,26], which is asemi-automated approach used for model calibration, validation, sensitivity, and uncertainty analysis,was used in this study. In SUFI-2, all sources of parameter uncertainties are assigned to parameters.The range of input parameters is assumed to be a uniform distribution, while the model outputuncertainty is quantified by the 95% prediction uncertainty (95PPU) determined at the 2.5% and97.5% levels of the cumulative distribution of output variables obtained via Latin hypercube sampling.The details of the SWAT-CUP program and SUFI-2 algorithm are in the SWAT-CUP user manual [12].
3.3.1. Parameterization
Parameterization is a process of selecting parameters and setting their initial value rangesfor calibration. Based on previous studies on the region [27–29] and literature on SWATcalibration/validation [7,11], nine parameters were selected, and then, their initial value rangeswere defined. Two methods, Relative and Replace, were used for different parameters. When Relativeis reported this means that a percentage relative change is applied to original parameters, and onlywhen Replace the min and max range values allowed for calibration are reported. The descriptions ofeach parameter along with their minimum and maximum values are presented in Table 2.
3.3.2. Objective Functions
Eleven objective functions are currently allowed in the SUFI-2 algorithm. Eight of the elevenwere selected to evaluate their impacts on the calibration results, calibrated parameters, and theoverall discharge processes simulation owing to their popularity and available details on accessibility.For the remaining three objective functions such as the multiplicative form of the square error,the summation form of the square error, and the Chi square, we have no or very little knowledgeabout them and will not be able to make any discussion and evaluation for their performances.The selected eight objective functions include the coefficient of determination (R2), modified coefficientof determination (bR2), Nash–Sutcliffe efficiency (NSE), modified Nash–Sutcliffe efficiency (MNS),ratio of the standard deviation of observations to the root mean square error (RSR), ranked sum ofsquares (SSQR), Kling–Gupta efficiency (KGE), and percent bias (PBIAS). The equations and referencesfor these objective functions are presented in Table 3. For NSE and RSE, it is expected that their
Water 2020, 12, 2901 6 of 22
simulation results are similar as they are equivalent objective functions given that NSE = 1 − RSR2
according to their equations in Table 3.
Table 2. Parameters used for the calibration and their initial ranges.
Parameter Extension Method DescriptionInitial Range
Min Max
CN2 .mgt Relative SCS runoff curve number −0.25 0.25SOL_AWC () .sol Relative Available water capacity −0.25 0.25
ESCO .hru Replace Soil evaporation compensation factor 0.01 1OV_N .hru Replace Manning’s “n” value for overland flow 0.01 30
HRU_SLP .hru Replace Average slope steepness 0 1SLSUBBSN .hru Replace Average slope length 10 150GWQMN .gw Replace Threshold depth of water in the shallow aquifer 0 5000
GW_REVAP .gw Replace Groundwater “revap *” coefficient 0.02 0.2
REVAPMN .gw Replace Threshold depth of water in the shallow aquiferfor “revap *” to occur 0 500
* Revap is an amount of water moving from shallow aquifer to plants/soil profile in watershed during simulationin mm.
Table 3. List of objective functions used in this study.
Objective Functions Equation Reference
Coefficient of determination R2 =[∑
i(Qm,i−Qm)(Qs,i−Qs)]2∑
i(Qm,i−Qm)2 ∑
i(Qs,i−Qs)2 [30]
Modified coefficient of determination bR2 =
{|b|R2 if |b| ≤ 1|b|−1R2 if |b| > 1
[30]
Nash–Sutcliffe efficiency NSE = 1−∑
i(Qm−Qs)2i∑
i(Qm,i−Qm)2 [31]
Modified Nash–Sutcliffe efficiency MNS = 1−∑
i|Qm−Qs|pi∑
i
∣∣∣Qm,i−Qm
∣∣∣pi
[30]
Ratio of the standard deviation ofobservations to root mean square error RSR =
√∑ni=1(Qm−Qs)
2i√∑n
i=1(Qm,i−Qm)2
[32]
Ranked sum of squares SSQR = 1n
n∑j=1
(Qj,m −Qj,s
)2[33]
Kling–Gupta efficiency KGE = 1−√(r− 1)2 + (α− 1)2 + (β− 1)2 [34]
Percent bias PBIAS = 100∑n
i=1(Qm−Qs)i∑ni=1 Qm,i
[35]
Here, Q is a variable (e.g., discharge), m stands for the measured value, s stands for the simulatedvalue, i represents the ith measured or simulated data, Qm is the mean measured data of variableQ, Qs is the mean simulated data of variable Q, b is the coefficient of the regression line betweenthe measured and simulated data, p is the modified Nash–Sutcliffe efficiency factor (p = 1 was usedin this study), n is the total number of measured or simulated data, j represents the rank, and r isthe linear regression coefficient between the simulated and measured variables, and α = σs
σmand
β =µsµm
, where σs and σm are the standard deviations of the simulated and measured data, respectively.Additionally, µs and µm are the means of the simulated and measured data, respectively.
3.3.3. Model Calibration, Validation, and Evaluation
Model calibration involves fitting parameter values by comparing the predicted output andmeasured data until a satisfactory standard for an objective function is achieved [36], whereas validationis an evaluation of whether the calibrated model has the ability to predict for later periods. In thisstudy, the model was run in a monthly time step. The measured discharge data at the BakTrakuon station during the 1995–2008 and 2009–2015 periods were used for model calibration andvalidation, respectively.
Water 2020, 12, 2901 7 of 22
The strengths of the calibration and validation of each objective function were evaluated.The evaluation was based on statistical indices of all objective functions used in this study. The detaileddescriptions and equations are presented in Table 3, and their statistical ranges, optimal values,and satisfactory values are shown in Table 4. The satisfactory threshold values for bR2 and MNS wereconsidered greater than or equal to 0.4, which was adopted from Kouchi et al. [20], who introducedmeasures based on the results of the studies of Muleta [37], Mehdi et al. [38], and Akhavan et al. [39].For the SSQR, a satisfactory value could not be specified because the measured and simulated variableswere independently ranked, and the value depends on the magnitude of the variables being investigated.Satisfactory values of the other indices were based on Moriasi et al. [32] and Thiemig et al. [40].
Table 4. Performance evaluations for the monthly discharge simulation [20,32,40].
Indices R2 bR2 NSE MNS RSR SSQR KGE PBIAS
Range 0 to 1 0 to 1 −∞ to 1 −∞ to 1 0 to∞ 0 to∞. −∞ to 1 −∞ to∞Optimal Value 1 1 1 1 0 0 1 0
Satisfactory Value >0.5 ≥0.4 >0.5 ≥0.4 ≤0.7 - ≥0.5 <±25
3.4. Evaluation of Each Objective Function
To identify a better objective function when calibrating the discharge in this study area,the simulated result of each objective function was graphically compared with the observed data,and the performance evaluation criteria (as shown in Table 4) was also taken into consideration.Additionally, characteristics of this tropical monsoon river basin were analyzed, and based onthese characteristics, estimated results of water balance components, such as evapotranspiration,surface runoff, lateral flow, groundwater flow, estimated results of the water yield, and calibratedvalues of the best parameter sets obtained from all objective functions during the calibration process,were also evaluated.
Additionally, the MOD16A2 Version 6 Evapotranspiration/Latent Heat Flux product with an 8-daycomposite dataset produced at 500 m pixel resolution from MODIS (Moderate Resolution ImagingSpectroradiometer) was used to confirm the actual evapotranspiration amount of the Pursat RiverBasin [41]. MODIS ET data was obtained for 2001 to 2015 period through R package “MODIStsp”and analyzed using R 4.0.2 version software (The R Foundation for Statistical Computing, Vienna,Austria) [42].
For further evaluation of the hydrological characteristics of this watershed, simulated dischargesof the best calibrated parameter sets of the different objective functions were categorized into the baseflow, rising limb, peak flow, and falling limb phases and then compared with the observed data at therespective flow phases from 1995 to 2008 (calibration period) using scatter plots. Then, the coefficientof determination (R2) and root mean squared deviation (RMSD) were used for the evaluation. The R2,which was calculated as shown in Table 3, determines how much variance the two variables share,and its value varies between 0 (no correlation) and 1 (perfect correlation). However, when the modelsystematically over- or under-predicts all the time, the R2 value is still close to 1. To cope with thisproblem, the slope and intercept of the regression on which R2 is based were taken into account. For agood agreement, the slope and intercept should be close to 1 and 0, respectively. The RMSD representsthe mean deviation of the predicted values with respect to the observed values [43,44] and can becalculated as
RMSD =
√√1
n− 1
n∑i=1
(Qm −Qs)2i , (1)
where n is the total number of observations, Qm is the measured discharge, Qs is the simulateddischarge, and i is the ith measured or simulated data. The unit of the RMSD is the same as the unitof Qm and Qs. The RMSD is always positive, and a value of 0 (almost never achieved in practice)indicates a perfect fit of the data. Generally, a lower RMSD is better than a higher one.
Water 2020, 12, 2901 8 of 22
4. Results
4.1. Simulation Results
Figure 3 graphically represents the best simulated discharges on the monthly time step (calibratedfrom 1995 to 2008 and validated from 2009 to 2015) of different objective functions compared with theobserved data. The simulated discharges corresponded well with the rainfall data. With the R2 objectivefunction, the calibrated model usually overestimated peak flows. When bR2 was used, the peak flowestimation became even more overestimated compared to the observed data. This originated fromthe equations of these objective functions. Both are defined by a minimization of total errors from alinear regression model and the length from the average value, which are not direct errors of measuredand estimated data and could have the effect on the calibrated parameters. According to Legatesand McCabe [45], owing to the squared differences in their equations, they are oversensitive to highextreme values and insensitive to additive and proportional differences between model predictionsand measured data.
For the choice of NSE, MNS, and RSR objective functions, their calibrated results were similar,especially the NSE and RSR objective functions. In addition to increasing the fitness betweenthe simulated and observed base flows and recession curves, the simulated peak flows were alsosignificantly reduced with considerable correspondence to the observed data. Similarly, when we usedthe SSQR and KGE objective functions, the simulation results could also capture well the distributionof the observed flow, although the simulated peak flows were slightly high. Those objective functionsfocus on errors between the measured and estimated data. Then, optimization of these objectivefunctions reduces the errors.
The PBIAS objective function, however, gave a different calibration result. Compared to thecalibration results of the other objective functions, the simulated peak flows were occasionallyoverestimated when we used the PBIAS objective function, whereas the simulated base flows wereoccasionally underestimated. This is the result of the characteristics of this objective function. When themodel overpredicts as much as it underpredicts, PBIAS can still provide a deceptive rating of modelperformance [35,46].
Water 2020, 12, x FOR PEER REVIEW 8 of 22
where n is the total number of observations, Qm is the measured discharge, Qs is the simulated
discharge, and i is the ith measured or simulated data. The unit of the RMSD is the same as the unit
of Qm and Qs. The RMSD is always positive, and a value of 0 (almost never achieved in practice)
indicates a perfect fit of the data. Generally, a lower RMSD is better than a higher one.
4. Results
4.1. Simulation Results
Figure 3 graphically represents the best simulated discharges on the monthly time step
(calibrated from 1995 to 2008 and validated from 2009 to 2015) of different objective functions
compared with the observed data. The simulated discharges corresponded well with the rainfall data.
With the R2 objective function, the calibrated model usually overestimated peak flows. When bR2 was
used, the peak flow estimation became even more overestimated compared to the observed data. This
originated from the equations of these objective functions. Both are defined by a minimization of total
errors from a linear regression model and the length from the average value, which are not direct
errors of measured and estimated data and could have the effect on the calibrated parameters.
According to Legates and McCabe [45], owing to the squared differences in their equations, they are
oversensitive to high extreme values and insensitive to additive and proportional differences
between model predictions and measured data.
For the choice of NSE, MNS, and RSR objective functions, their calibrated results were similar,
especially the NSE and RSR objective functions. In addition to increasing the fitness between the
simulated and observed base flows and recession curves, the simulated peak flows were also
significantly reduced with considerable correspondence to the observed data. Similarly, when we
used the SSQR and KGE objective functions, the simulation results could also capture well the
distribution of the observed flow, although the simulated peak flows were slightly high. Those
objective functions focus on errors between the measured and estimated data. Then, optimization of
these objective functions reduces the errors.
The PBIAS objective function, however, gave a different calibration result. Compared to the
calibration results of the other objective functions, the simulated peak flows were occasionally
overestimated when we used the PBIAS objective function, whereas the simulated base flows were
occasionally underestimated. This is the result of the characteristics of this objective function. When
the model overpredicts as much as it underpredicts, PBIAS can still provide a deceptive rating of
model performance [35,46].
Figure 3. Cont.
Water 2020, 12, 2901 10 of 22
Water 2020, 12, x FOR PEER REVIEW 10 of 22
Figure 3. Monthly simulated and validated discharges of different objective functions compared with
observed data.
4.2. Model Performance
The model performances of the calibrated parameter sets obtained from different objective
functions were evaluated and compared by using the statistical indices of all objective functions
(Figure 4). Many objective functions performed better than satisfactory in terms of calibration and
validation for most of the criteria described in Table 4. The simulated results of the fitted parameter
sets from the NSE, MNS, RSR, and KGE objective functions satisfied all statistical indices, whereas
the results of the parameter set by the R2 objective function failed to satisfy the PBIAS statistical index
during the validation period. For the result of the parameter set by SSQR, it failed to satisfy the MNS
statistical index during both calibration and validation periods. Of the worst, the result of the
parameter set by the bR2 objective function failed to satisfy NSE, MNS, and RSR statistical indices
during the calibration period, whereas the SSQR statistical value of this objective function during the
calibration period was quite high compared to that of other objective functions. For the PBIAS
objective function, the result of the parameter set during the calibration period failed to satisfy the
NSE, MNS, and RSR statistical indices.
In this study, as mentioned earlier, the calibration period was defined longer (1995–2008). The
purpose of this was to cover all the magnitudes of discharge pattern which included not only the
normal years but also the wettest year in 1996 and the driest year in 2002 (Figure 3) in the calibration
Figure 3. Monthly simulated and validated discharges of different objective functions compared withobserved data.
4.2. Model Performance
The model performances of the calibrated parameter sets obtained from different objectivefunctions were evaluated and compared by using the statistical indices of all objective functions(Figure 4). Many objective functions performed better than satisfactory in terms of calibration andvalidation for most of the criteria described in Table 4. The simulated results of the fitted parametersets from the NSE, MNS, RSR, and KGE objective functions satisfied all statistical indices, whereas theresults of the parameter set by the R2 objective function failed to satisfy the PBIAS statistical indexduring the validation period. For the result of the parameter set by SSQR, it failed to satisfy theMNS statistical index during both calibration and validation periods. Of the worst, the result of theparameter set by the bR2 objective function failed to satisfy NSE, MNS, and RSR statistical indicesduring the calibration period, whereas the SSQR statistical value of this objective function duringthe calibration period was quite high compared to that of other objective functions. For the PBIASobjective function, the result of the parameter set during the calibration period failed to satisfy theNSE, MNS, and RSR statistical indices.
Water 2020, 12, 2901 11 of 22
Water 2020, 12, x FOR PEER REVIEW 11 of 22
process. On the contrary, the validation period, which was shorter from 2009 to 2015, included mostly
normal years only. As the result, the performance of the validation as measured by the statistical
indices was better than that of the calibration in many cases (Figure 4).
Figure 4. Results of model performance indices of different objective functions.
5. Discussion
5.1. Discharge Process Estimations
The results of annual average water balance components and water yields estimated by different
objective functions using the best parameter sets in the calibration process are shown in Figure 5. The
results showed that the estimated water balance components and water yields differed by the types
of objective functions used. The estimated annual average evapotranspiration (ET) by different
objective functions ranged from 656.60 to 756.30 mm (48.99% to 56.42% of the total rainfall), whereas
the estimated annual average surface runoff (SurQ) ranged from 116.27 to 560.05 mm (8.67% to
41.78% of the total rainfall) (Figure 5a). The ratios of estimated annual average lateral flow (LatQ)
and groundwater flow (GWQ) to the total rainfall generated by those objective functions ranged from
0.51% (6.83 mm) to 26.02% (348.76 mm) and 0.31% (4.17 mm) to 13.18% (176.7 mm), respectively.
Additionally, the ratios of estimated annual average deep aquifer recharge (DAR) and amount of
water moving from the shallow aquifer to plant/soil profile (Revap) to the total rainfall generated by
those objective functions ranged from 0.26% (3.51 mm) to 0.89% (11.92 mm) and 0.86% (11.54 mm) to
11.84% (158.66 mm), respectively. For the estimated annual average water yield (Figure 5b), with the
MNS objective function, the model generated the lowest annual average water yield of 456.49 mm,
whereas the highest annual average water yield of 679.79 mm was given by the model when we used
the bR2 objective function. The result showed that the estimated water balance components and water
yields of the NSE and RSR objective functions were the same.
Figure 4. Results of model performance indices of different objective functions.
In this study, as mentioned earlier, the calibration period was defined longer (1995–2008).The purpose of this was to cover all the magnitudes of discharge pattern which included not only thenormal years but also the wettest year in 1996 and the driest year in 2002 (Figure 3) in the calibrationprocess. On the contrary, the validation period, which was shorter from 2009 to 2015, included mostlynormal years only. As the result, the performance of the validation as measured by the statisticalindices was better than that of the calibration in many cases (Figure 4).
5. Discussion
5.1. Discharge Process Estimations
The results of annual average water balance components and water yields estimated by differentobjective functions using the best parameter sets in the calibration process are shown in Figure 5.The results showed that the estimated water balance components and water yields differed by thetypes of objective functions used. The estimated annual average evapotranspiration (ET) by differentobjective functions ranged from 656.60 to 756.30 mm (48.99% to 56.42% of the total rainfall), whereas theestimated annual average surface runoff (SurQ) ranged from 116.27 to 560.05 mm (8.67% to 41.78%of the total rainfall) (Figure 5a). The ratios of estimated annual average lateral flow (LatQ) andgroundwater flow (GWQ) to the total rainfall generated by those objective functions ranged from0.51% (6.83 mm) to 26.02% (348.76 mm) and 0.31% (4.17 mm) to 13.18% (176.7 mm), respectively.Additionally, the ratios of estimated annual average deep aquifer recharge (DAR) and amount ofwater moving from the shallow aquifer to plant/soil profile (Revap) to the total rainfall generated bythose objective functions ranged from 0.26% (3.51 mm) to 0.89% (11.92 mm) and 0.86% (11.54 mm) to11.84% (158.66 mm), respectively. For the estimated annual average water yield (Figure 5b), with theMNS objective function, the model generated the lowest annual average water yield of 456.49 mm,whereas the highest annual average water yield of 679.79 mm was given by the model when we usedthe bR2 objective function. The result showed that the estimated water balance components and wateryields of the NSE and RSR objective functions were the same.
Water 2020, 12, 2901 12 of 22Water 2020, 12, x FOR PEER REVIEW 12 of 22
(a) (b)
Figure 5. (a) Results of the water balance components and (b) water yields of different objective
functions using the best parameter sets from the calibration process.
In a study conducted by Shimizu et al. [47], almost all mountainous regions of west Cambodia
(where this study area was also located) had annual renewable freshwater resources (water yield) of
500 mm. Thus, NSE and RSR objective functions provided closer annual average water yield
estimations than other objective functions at the same value of 492 mm, followed by the MNS, KGE,
and PBIAS objective functions, which generated water yields of 456.49, 544.45, and 548.98 mm,
respectively. For the results of the water balance components estimation, of the objective functions
used in this study, the MNS, NSE, RSR, PBIAS, and KGE objective functions provided the most
reasonable estimation results of annual average evapotranspiration (ET) of 756.3, 726.2, 726.2, 723.9,
and 721.7 mm, respectively. These results closely corresponded to the ET value obtained from MODIS
ET of 745 mm on average annually (Figure 6) and the findings in the study conducted by JICA [48]
in southern Cambodia with an estimated annual average ET of 743.9 mm. The goodness of the ET
simulated by these objective functions were also confirmed by the linear regression with the ET
obtained from MODIS ET as shown in Figure 7. Again, the PBIAS, NSE, RSR, MNS, and KGE
objective functions provided relatively small estimated values of groundwater flow (GWQ), and
these objective functions, except for the PBIAS objective function, generated a small proportion of
surface runoff (SurQ) compared to a proportion of lateral flow (LatQ), which reflected the flow
characteristics of this study area. A detailed description of the discharge characteristics and
physiographic condition of the river basin is presented in Section 5.4.
Figure 6. Average monthly (left) and annual evapotranspiration (right) in Pursat River basin
(drainage area at Bak Trakuon outlet) from 2001 to 2015 obtained from MODIS ET.
Figure 5. (a) Results of the water balance components and (b) water yields of different objectivefunctions using the best parameter sets from the calibration process.
In a study conducted by Shimizu et al. [47], almost all mountainous regions of west Cambodia(where this study area was also located) had annual renewable freshwater resources (water yield)of 500 mm. Thus, NSE and RSR objective functions provided closer annual average water yieldestimations than other objective functions at the same value of 492 mm, followed by the MNS, KGE,and PBIAS objective functions, which generated water yields of 456.49, 544.45, and 548.98 mm,respectively. For the results of the water balance components estimation, of the objective functions usedin this study, the MNS, NSE, RSR, PBIAS, and KGE objective functions provided the most reasonableestimation results of annual average evapotranspiration (ET) of 756.3, 726.2, 726.2, 723.9, and 721.7 mm,respectively. These results closely corresponded to the ET value obtained from MODIS ET of 745 mm onaverage annually (Figure 6) and the findings in the study conducted by JICA [48] in southern Cambodiawith an estimated annual average ET of 743.9 mm. The goodness of the ET simulated by these objectivefunctions were also confirmed by the linear regression with the ET obtained from MODIS ET as shownin Figure 7. Again, the PBIAS, NSE, RSR, MNS, and KGE objective functions provided relatively smallestimated values of groundwater flow (GWQ), and these objective functions, except for the PBIASobjective function, generated a small proportion of surface runoff (SurQ) compared to a proportion oflateral flow (LatQ), which reflected the flow characteristics of this study area. A detailed description ofthe discharge characteristics and physiographic condition of the river basin is presented in Section 5.4.
Water 2020, 12, x FOR PEER REVIEW 12 of 22
(a) (b)
Figure 5. (a) Results of the water balance components and (b) water yields of different objective
functions using the best parameter sets from the calibration process.
In a study conducted by Shimizu et al. [47], almost all mountainous regions of west Cambodia
(where this study area was also located) had annual renewable freshwater resources (water yield) of
500 mm. Thus, NSE and RSR objective functions provided closer annual average water yield
estimations than other objective functions at the same value of 492 mm, followed by the MNS, KGE,
and PBIAS objective functions, which generated water yields of 456.49, 544.45, and 548.98 mm,
respectively. For the results of the water balance components estimation, of the objective functions
used in this study, the MNS, NSE, RSR, PBIAS, and KGE objective functions provided the most
reasonable estimation results of annual average evapotranspiration (ET) of 756.3, 726.2, 726.2, 723.9,
and 721.7 mm, respectively. These results closely corresponded to the ET value obtained from MODIS
ET of 745 mm on average annually (Figure 6) and the findings in the study conducted by JICA [48]
in southern Cambodia with an estimated annual average ET of 743.9 mm. The goodness of the ET
simulated by these objective functions were also confirmed by the linear regression with the ET
obtained from MODIS ET as shown in Figure 7. Again, the PBIAS, NSE, RSR, MNS, and KGE
objective functions provided relatively small estimated values of groundwater flow (GWQ), and
these objective functions, except for the PBIAS objective function, generated a small proportion of
surface runoff (SurQ) compared to a proportion of lateral flow (LatQ), which reflected the flow
characteristics of this study area. A detailed description of the discharge characteristics and
physiographic condition of the river basin is presented in Section 5.4.
Figure 6. Average monthly (left) and annual evapotranspiration (right) in Pursat River basin
(drainage area at Bak Trakuon outlet) from 2001 to 2015 obtained from MODIS ET.
Figure 6. Average monthly (left) and annual evapotranspiration (right) in Pursat River basin (drainagearea at Bak Trakuon outlet) from 2001 to 2015 obtained from MODIS ET.
Water 2020, 12, 2901 13 of 22Water 2020, 12, x FOR PEER REVIEW 13 of 22
Figure 7. Linear regression between monthly average evapotranspiration obtained from the
simulation results (ET Sim.) of different objective functions and monthly average evapotranspiration
obtained from MODIS ET.
5.2. Best Parameter Sets and Sensitivity Rank
Table 5 shows the fitted parameter values and sensitivity rank of the parameters used in this
study obtained from different objective functions during the calibration process. Besides the NSE and
RSR objective functions, different objective functions generated different values of best parameter
sets and sensitivity rank. The fitted parameter sets of the NSE, RSR, and MNS objective functions
were in a reasonable range in this study. When we used the R2 objective function, the relative value
of the parameter change for CN2 was high (+13%), whereas the existing parameter value of CN2 of
the initial model was (between 55 and 92) approximately 78 on average, which was already high for
the land-use and soil types of this study area. This led to a big runoff (surface and lateral flow)
amount, as shown in Figure 5a, and high simulated peak flows, as shown in Figure 3. This objective
function produced a small threshold value of GWQMN of 1680.42 mm, which led to more
groundwater flow, and a small coefficient of GW_REVAP of 0.03, which generated a lower
evapotranspiration rate and revap because of the limitation of movement of water from the shallow
aquifer to the root zone. Moreover, these three parameters were among the five most sensitive
parameters, which highly controlled the simulation results of the R2 objective function. Even worst,
the bR2 objective function produced a higher relative value of the parameter change for CN2 of +15%,
a smaller threshold value of GWQMN of 291.86, a low coefficient of GW_REVAP of 0.07, and a large
value of ESCO, which then led to a large runoff, greater groundwater flow, smaller
evapotranspiration, and lower revap (Figure 5a). Consequently, this objective function highly
overestimated the peak flows as shown in Figure 3. Furthermore, while the value of the initial model
of SOL_AWC was small between 84 and 371 mm (approximately 186 mm on average), the fitted value
of the relative change was still underestimated at +16%, and also, the fitted value of SLSUBBSN was
overestimated at 68.03 m. As a result, a huge surface runoff and a neglected lateral flow occurred for
this objective function. Additionally, the calibrated value of the average slope steepness (HRU_SLP)
of bR2 was small. However, this parameter was the least sensitive, which may not have as a
considerable effect as the earlier mentioned six parameters (they were the top six sensitive
parameters).
Figure 7. Linear regression between monthly average evapotranspiration obtained from the simulationresults (ET Sim.) of different objective functions and monthly average evapotranspiration obtainedfrom MODIS ET.
5.2. Best Parameter Sets and Sensitivity Rank
Table 5 shows the fitted parameter values and sensitivity rank of the parameters used in thisstudy obtained from different objective functions during the calibration process. Besides the NSE andRSR objective functions, different objective functions generated different values of best parameter setsand sensitivity rank. The fitted parameter sets of the NSE, RSR, and MNS objective functions were ina reasonable range in this study. When we used the R2 objective function, the relative value of theparameter change for CN2 was high (+13%), whereas the existing parameter value of CN2 of the initialmodel was (between 55 and 92) approximately 78 on average, which was already high for the land-useand soil types of this study area. This led to a big runoff (surface and lateral flow) amount, as shown inFigure 5a, and high simulated peak flows, as shown in Figure 3. This objective function produced asmall threshold value of GWQMN of 1680.42 mm, which led to more groundwater flow, and a smallcoefficient of GW_REVAP of 0.03, which generated a lower evapotranspiration rate and revap becauseof the limitation of movement of water from the shallow aquifer to the root zone. Moreover, these threeparameters were among the five most sensitive parameters, which highly controlled the simulationresults of the R2 objective function. Even worst, the bR2 objective function produced a higher relativevalue of the parameter change for CN2 of +15%, a smaller threshold value of GWQMN of 291.86,a low coefficient of GW_REVAP of 0.07, and a large value of ESCO, which then led to a large runoff,greater groundwater flow, smaller evapotranspiration, and lower revap (Figure 5a). Consequently,this objective function highly overestimated the peak flows as shown in Figure 3. Furthermore,while the value of the initial model of SOL_AWC was small between 84 and 371 mm (approximately186 mm on average), the fitted value of the relative change was still underestimated at +16%, and also,the fitted value of SLSUBBSN was overestimated at 68.03 m. As a result, a huge surface runoff and aneglected lateral flow occurred for this objective function. Additionally, the calibrated value of theaverage slope steepness (HRU_SLP) of bR2 was small. However, this parameter was the least sensitive,which may not have as a considerable effect as the earlier mentioned six parameters (they were the topsix sensitive parameters).
Water 2020, 12, 2901 14 of 22
Table 5. Fitted values and sensitivity ranks (values in parentheses) of the calibrated parameters obtainedfrom different objective functions during the calibration process.
ParameterFitted Parameter Values and Parameter Sensitivity Ranks (Values in
Parentheses) by Different Objective Functions
R2 bR2 NSE MNS RSR SSQR KGE PBIAS
r__CN2.mgt * 13% 15% 3% −1% 3% 6% 8% 14%(4) (5) (2) (2) (2) (4) (5) (1)
r__SOL_AWC().sol * 45% 16% 45% 37% 45% 18% 38% −10%(6) (4) (3) (5) (3) (5) (6) (2)
v__ESCO.hru0.76 0.96 0.76 0.57 0.76 0.95 0.72 0.11(3) (1) (4) (4) (4) (3) (4) (3)
v__OV_N.hru14.84 10.82 22.38 12.37 22.38 29.61 22.70 27.39
(9) (7) (7) (9) (7) (8) (8) (6)
v__HRU_SLP.hru0.83 0.22 0.74 0.91 0.74 0.96 0.89 0.81(7) (9) (5) (3) (5) (9) (9) (5)
v__SLSUBBSN.hru13.53 68.03 13.21 11.00 13.21 37.25 11.55 118.23
(1) (6) (1) (1) (1) (7) (2) (4)
v__GWQMN.gw 1680.42 291.86 2696.49 2381.61 2696.49 3261.89 1731.04 3006.24(2) (3) (8) (6) (8) (1) (3) (8)
v__GW_REVAP.gw 0.03 0.07 0.11 0.14 0.11 0.04 0.07 0.18(5) (2) (6) (8) (6) (2) (1) (9)
v__REVAPMN.gw 380.62 301.52 133.67 416.76 133.67 5.75 349.10 462.03(8) (8) (9) (7) (9) (6) (7) (7)
* The calibrated CN2 and SOL_AWC values were in relative change, while their original values on average from theinitial model were 78 and 186 mm, respectively.
Regarding the SSQR objective function, the problems were that the fitted value of the relativechange of SOL_AWC was small (18%), whereas the fitted value of SLSUBBSN was large (37.25 m),which were the fifth and seventh most sensitive parameters, respectively. This resulted in a largesurface runoff and small lateral flow (Figure 5a). Moreover, the large value of ESCO (0.95), which wasthe third most sensitive parameter, led to a low evapotranspiration; the small value of GW_REVAP(0.04), which was the second most sensitive parameter, led to big groundwater flow and small revap;and the small value of REVAPMN (5.75), which was the sixth most sensitive parameter, contributed toa slightly high deep aquifer recharge. Regarding the KGE objective function, the problems includedthe calibrated parameter set being slightly large for the relative change of CN2 (8%), which was thefifth most sensitive parameter, the slightly small value of GWQMN (1731.04), which was the thirdmost sensitive parameter, and the slightly small value of GW_REVAP (0.07), which was the mostsensitive parameter. As the result, runoff (surface and lateral flow) and groundwater flow generatedby this objective function were slightly large, and the revap was relatively small (Figure 5a). For thePBIAS objective function, the calibrated parameter value of the relative change of CN2, which was themost sensitive parameter, was huge (+14%), whereas the value of the relative change of SOL_AWC,which was the second most sensitive parameter, was negative (−10%) and the value of SLSUBBSN,which was the fourth most sensitive parameter, was too long (118.23 m). Consequently, the runoff
obtained from this objective function was large with a huge surface runoff and a very small lateralflow (Figure 5a). This is shown in Figure 3 as an overestimation of the peak flows. This objectivefunction also produced a very small value for ESCO, which was the third most sensitive parameter,which should produce a large evaporative demand from the soil. However, owing to the limitedavailable plant water (small SOL_AWC), evaporative demand from the soil was also restricted,leading to an evapotranspiration restriction.
Water 2020, 12, 2901 15 of 22
5.3. Hydrograph Components Estimation
To evaluate how different objective functions performed the simulation of each hydrologicalprocess, the monthly calibrated results of discharge and observed data were classified into baseflow, rising limb, peak flow, and falling limb phases. Based on the monthly average hydrograph inSection 5.4, base flow, which is the period of low flow, was considered from January to March. Then,the rising limb or concentration curve, the ascending portion of the hydrograph, was considered fromApril to August. Peak flow or crest segment, the inflection point on the rising limb to the falling limb,was considered from September to October; and the falling limb or recession curve, the descendingportion from the point of inflection at the end of the crest segment to the base flow, was consideredfrom November to December for each year of the calibration period from 1995 to 2008. In this section,the results of RSR objective function was not further presented and discussed as it is equivalent to andproduced the same results with NSE objective function.
Figure 8 presents the scatter plots of the simulated versus observed discharge of base flowperiod for different objective functions. All of the objective functions generally overestimated thebase flows. This behavior can be explained by the small fitted parameter values of GWQMN below3000 mm as generated by many of the objective functions and the small fitted parameter values ofGW_REVAP below 0.05 as defined by some of the objective functions (Table 5). A study conducted byRafiei Emam et al. [49] in central Vietnam, which is also predominant by forest, defined the final rangefor GWQMN between 3133 and 3756 mm. However, among them, NSE and MNS objective functionsproduced better simulation results for the base flows with higher R2 values, better slope (optimumvalue of 1) and intercept (optimum value of 0), and smaller RMSD value. The simulated base flows ofthe MNS objective function achieved an R2 of 0.45, slope of 0.48, intercept of 3.59, and RMSD of 7.78,whereas the simulation results of the NSE objective function provided a value of R2 of 0.48, slope of0.41, intercept of 2.84, and RMSD of 10.76.
Water 2020, 12, x FOR PEER REVIEW 15 of 22
5.3. Hydrograph Components Estimation
To evaluate how different objective functions performed the simulation of each hydrological
process, the monthly calibrated results of discharge and observed data were classified into base flow,
rising limb, peak flow, and falling limb phases. Based on the monthly average hydrograph in Section
5.4, base flow, which is the period of low flow, was considered from January to March. Then, the
rising limb or concentration curve, the ascending portion of the hydrograph, was considered from
April to August. Peak flow or crest segment, the inflection point on the rising limb to the falling limb,
was considered from September to October; and the falling limb or recession curve, the descending
portion from the point of inflection at the end of the crest segment to the base flow, was considered
from November to December for each year of the calibration period from 1995 to 2008. In this section,
the results of RSR objective function was not further presented and discussed as it is equivalent to
and produced the same results with NSE objective function.
Figure 8 presents the scatter plots of the simulated versus observed discharge of base flow period
for different objective functions. All of the objective functions generally overestimated the base flows.
This behavior can be explained by the small fitted parameter values of GWQMN below 3000 mm as
generated by many of the objective functions and the small fitted parameter values of GW_REVAP
below 0.05 as defined by some of the objective functions (Table 5). A study conducted by Rafiei Emam
et al. [49] in central Vietnam, which is also predominant by forest, defined the final range for
GWQMN between 3133 and 3756 mm. However, among them, NSE and MNS objective functions
produced better simulation results for the base flows with higher R2 values, better slope (optimum
value of 1) and intercept (optimum value of 0), and smaller RMSD value. The simulated base flows
of the MNS objective function achieved an R2 of 0.45, slope of 0.48, intercept of 3.59, and RMSD of
7.78, whereas the simulation results of the NSE objective function provided a value of R2 of 0.48, slope
of 0.41, intercept of 2.84, and RMSD of 10.76.
Figure 8. Scatter plots of simulated versus observed base flows for different objective functions.
For the rising limb estimation, none of the simulation results of these objective functions showed
a reasonable correlation with the observed data with an R2 of 0.21 at most and large intercepts;
however, the correlation slopes of some of them reached a value that was greater than 0.6. Again,
NSE objective function provided the closest simulation results (Figure 9) with R2, slope, intercept,
and RMSD values of 0.21, 0.67, 35.16, and 63.97, respectively.
Figure 8. Scatter plots of simulated versus observed base flows for different objective functions.
For the rising limb estimation, none of the simulation results of these objective functions showed areasonable correlation with the observed data with an R2 of 0.21 at most and large intercepts; however,the correlation slopes of some of them reached a value that was greater than 0.6. Again, NSE objectivefunction provided the closest simulation results (Figure 9) with R2, slope, intercept, and RMSD valuesof 0.21, 0.67, 35.16, and 63.97, respectively.
Water 2020, 12, 2901 16 of 22
Water 2020, 12, x FOR PEER REVIEW 16 of 22
Figure 9. Scatter plots of simulated versus observed rising limb for different objective functions.
Conversely, the simulated peak flows of these objective functions gave a reverse performance
result (Figure 10). The simulated peak flows of the PBIAS, bR2, R2, and KGE objective functions
attained slightly higher R2 values compared to the results of the other objective functions at slightly
larger than 0.50, whereas those of the remaining objective functions were a little less than 0.50.
However, the regression slopes of the results obtained from the NSE and MNS objective functions
reached a satisfactory value of above 0.70, and those obtained from the other objective functions were
between 0.54 and 0.64. The intercepts obtained from all objective functions were between 76.82 and
102.54, and the RMSD values were between 106.67 and 127.61.
Figure 10. Scatter plots of simulated versus observed peak flows for different objective functions.
For the falling limb estimation, the performances of most objective functions were good (Figure
11). The MNS objective function performed well in simulating the falling limbs with an R2 of 0.79
with a good slope of 1.01, a small intercept of −9.62, and a RMSD of 37.38. The NSE and KGE objective
functions performed similarly but were lower with an R2 of approximately 0.78 and RMSD of
approximately 41. However, the regression slope and intercept obtained from the NSE objective
Figure 9. Scatter plots of simulated versus observed rising limb for different objective functions.
Conversely, the simulated peak flows of these objective functions gave a reverse performanceresult (Figure 10). The simulated peak flows of the PBIAS, bR2, R2, and KGE objective functionsattained slightly higher R2 values compared to the results of the other objective functions at slightlylarger than 0.50, whereas those of the remaining objective functions were a little less than 0.50. However,the regression slopes of the results obtained from the NSE and MNS objective functions reached asatisfactory value of above 0.70, and those obtained from the other objective functions were between0.54 and 0.64. The intercepts obtained from all objective functions were between 76.82 and 102.54,and the RMSD values were between 106.67 and 127.61.
Water 2020, 12, x FOR PEER REVIEW 16 of 22
Figure 9. Scatter plots of simulated versus observed rising limb for different objective functions.
Conversely, the simulated peak flows of these objective functions gave a reverse performance
result (Figure 10). The simulated peak flows of the PBIAS, bR2, R2, and KGE objective functions
attained slightly higher R2 values compared to the results of the other objective functions at slightly
larger than 0.50, whereas those of the remaining objective functions were a little less than 0.50.
However, the regression slopes of the results obtained from the NSE and MNS objective functions
reached a satisfactory value of above 0.70, and those obtained from the other objective functions were
between 0.54 and 0.64. The intercepts obtained from all objective functions were between 76.82 and
102.54, and the RMSD values were between 106.67 and 127.61.
Figure 10. Scatter plots of simulated versus observed peak flows for different objective functions.
For the falling limb estimation, the performances of most objective functions were good (Figure
11). The MNS objective function performed well in simulating the falling limbs with an R2 of 0.79
with a good slope of 1.01, a small intercept of −9.62, and a RMSD of 37.38. The NSE and KGE objective
functions performed similarly but were lower with an R2 of approximately 0.78 and RMSD of
approximately 41. However, the regression slope and intercept obtained from the NSE objective
Figure 10. Scatter plots of simulated versus observed peak flows for different objective functions.
For the falling limb estimation, the performances of most objective functions were good (Figure 11).The MNS objective function performed well in simulating the falling limbs with an R2 of 0.79 with a goodslope of 1.01, a small intercept of −9.62, and a RMSD of 37.38. The NSE and KGE objective functionsperformed similarly but were lower with an R2 of approximately 0.78 and RMSD of approximately 41.
Water 2020, 12, 2901 17 of 22
However, the regression slope and intercept obtained from the NSE objective function were at 0.97 and−14.46, respectively, whereas those obtained from KGE were 0.90 and −5.52, respectively.
Water 2020, 12, x FOR PEER REVIEW 17 of 22
function were at 0.97 and −14.46, respectively, whereas those obtained from KGE were 0.90 and −5.52,
respectively.
Figure 11. Scatter plots of the simulated versus observed falling limbs for different objective functions.
The results showed that the NSE and MNS objective functions provided overall better estimation
results for all the components of the hydrograph for this river basin. However, KGE, R2, bR2, SSQR,
and PBIAS were among the more poor objective functions, especially for the simulation during low
flow periods. For NSE, the differences between the observed and predicted values were calculated as
squared values. As a result, larger values in a time series are strongly overestimated, whereas lower
values are neglected [45]. Additionally, runoff peaks will tend to be underestimated when NSE is
used in the optimization [34]. However, NSE is good for use with continuous long-term simulations
and can be used to determine how well a model simulates trends for the output response of concern
[46,50]. Because the calibration duration of this study was 14 years, it is likely that the NSE objective
function could capture this long-term trend of the discharge. For the MNS objective function, which
was the modified form of NSE with the modified factor of p = 1 used in this study, it can be expected
that the modified forms are more sensitive to significant over- or under-prediction than the squared
forms [30]. However, in this study, the performance of the MNS objective function was only slightly
better than the performance of the NSE objective function when we simulated the falling limbs, but
it always slightly performed worse than the NSE objective function when we simulated the other
components of the hydrograph.
Another objective function used in this study, KGE, is a decomposition of NSE. Similar to NSE,
the runoff peaks will tend to be underestimated, but when the KGE optimization is used, the
underestimation will not be as severe [34]. As a result, the simulated peak flows of KGE exhibited a
slightly better correlation than that of NSE. However, the slope of the regression line for NSE was
better (Figure 10), which was because of the suitability of NSE in regressing the observed against the
simulated values [34]. The R2 objective function is widely used in hydrological modeling studies, but
it is oversensitive to high extreme values and insensitive to additive and proportional differences
between model predictions and measured data [45]. For the bR2 objective function, the under- or
over-predictions are quantified together with the dynamics, which results in a more comprehensive
reflection of model results [30]. The SSQR objective function aims at fitting the distribution of the
flows, ensuring that the full range of the flows is represented but without considering the time of
occurrence of a given value of the flows [33]. Perhaps due this characteristic, the estimated recession
curves (falling limbs) were typically flatter than the observed data, and the estimated base flows were
shortened (Figure 3). As a result, the simulation performances of the falling limbs (Figure 11) and
base flows (Figure 8) of this objective function were relatively low. For the PBIAS objective function,
Figure 11. Scatter plots of the simulated versus observed falling limbs for different objective functions.
The results showed that the NSE and MNS objective functions provided overall better estimationresults for all the components of the hydrograph for this river basin. However, KGE, R2, bR2, SSQR,and PBIAS were among the more poor objective functions, especially for the simulation during lowflow periods. For NSE, the differences between the observed and predicted values were calculated assquared values. As a result, larger values in a time series are strongly overestimated, whereas lowervalues are neglected [45]. Additionally, runoff peaks will tend to be underestimated when NSE is usedin the optimization [34]. However, NSE is good for use with continuous long-term simulations andcan be used to determine how well a model simulates trends for the output response of concern [46,50].Because the calibration duration of this study was 14 years, it is likely that the NSE objective functioncould capture this long-term trend of the discharge. For the MNS objective function, which was themodified form of NSE with the modified factor of p = 1 used in this study, it can be expected that themodified forms are more sensitive to significant over- or under-prediction than the squared forms [30].However, in this study, the performance of the MNS objective function was only slightly better thanthe performance of the NSE objective function when we simulated the falling limbs, but it alwaysslightly performed worse than the NSE objective function when we simulated the other components ofthe hydrograph.
Another objective function used in this study, KGE, is a decomposition of NSE. Similar toNSE, the runoff peaks will tend to be underestimated, but when the KGE optimization is used,the underestimation will not be as severe [34]. As a result, the simulated peak flows of KGE exhibiteda slightly better correlation than that of NSE. However, the slope of the regression line for NSE wasbetter (Figure 10), which was because of the suitability of NSE in regressing the observed against thesimulated values [34]. The R2 objective function is widely used in hydrological modeling studies,but it is oversensitive to high extreme values and insensitive to additive and proportional differencesbetween model predictions and measured data [45]. For the bR2 objective function, the under- orover-predictions are quantified together with the dynamics, which results in a more comprehensivereflection of model results [30]. The SSQR objective function aims at fitting the distribution of the flows,ensuring that the full range of the flows is represented but without considering the time of occurrenceof a given value of the flows [33]. Perhaps due this characteristic, the estimated recession curves (fallinglimbs) were typically flatter than the observed data, and the estimated base flows were shortened
Water 2020, 12, 2901 18 of 22
(Figure 3). As a result, the simulation performances of the falling limbs (Figure 11) and base flows(Figure 8) of this objective function were relatively low. For the PBIAS objective function, it is usefulfor continuous long-term simulations and can be used to determine how well the model simulates theaverage magnitudes for the output response of interest [46]. PBIAS can provide a deceptive rating ofmodel performance when the model over-predicts as much as it under-predicts, in which case PBIASwill be close to zero even though the model simulation is poor [46]. This may be why there wereseveral sudden peak and drop points of the simulated result of this objective function during the baseflow and rising limb periods (Figure 3), leading to poor model performances in simulating the baseflows and rising limbs, as shown in Figures 6 and 7, respectively.
5.4. Objective Functions Corresponding to the Characteristics of the River Basin
Figure 12 shows the monthly average hydrograph of the observed flow at Bak Trakuon Stationand rainfall at Kravanh Station during the calibration period from 1995 to 2008 (data between 1997and 1998 were excluded due to missing data). The validation period was not included because thisstudy focused on the effect of different objective functions on model calibration. Because the dryseason, which extends from December to April, is influenced by the northeast monsoon system [22],the river discharge is relatively low, approaching zero between January and March. This indicatedthat groundwater from the upstream area did not contribute considerably to the river discharge.However the river discharge begins to rise from April when the wet season starts, and the first peakoccurs in May as the monsoon rain travels north [22]. This peak is followed by a period of lowerrainfall between June and August. The greatest peak occurs between the months of September andOctober and is caused by a southerly shift in the monsoon circulation pattern, which is characterizedby heavy rainfall. This indicated that the flow characteristic in this river basin is highly controlled bythe monsoon rainfall pattern, especially during the wettest period from August of November when thecorrelation between the monthly discharge and the monthly rainfall was so high.
Water 2020, 12, x FOR PEER REVIEW 18 of 22
it is useful for continuous long-term simulations and can be used to determine how well the model
simulates the average magnitudes for the output response of interest [46]. PBIAS can provide a
deceptive rating of model performance when the model over-predicts as much as it under-predicts,
in which case PBIAS will be close to zero even though the model simulation is poor [46]. This may be
why there were several sudden peak and drop points of the simulated result of this objective function
during the base flow and rising limb periods (Figure 3), leading to poor model performances in
simulating the base flows and rising limbs, as shown in Figures 6 and 7, respectively.
5.4. Objective Functions Corresponding to the Characteristics of the River Basin
Figure 12 shows the monthly average hydrograph of the observed flow at Bak Trakuon Station
and rainfall at Kravanh Station during the calibration period from 1995 to 2008 (data between 1997
and 1998 were excluded due to missing data). The validation period was not included because this
study focused on the effect of different objective functions on model calibration. Because the dry
season, which extends from December to April, is influenced by the northeast monsoon system [22],
the river discharge is relatively low, approaching zero between January and March. This indicated
that groundwater from the upstream area did not contribute considerably to the river discharge.
However the river discharge begins to rise from April when the wet season starts, and the first peak
occurs in May as the monsoon rain travels north [22]. This peak is followed by a period of lower
rainfall between June and August. The greatest peak occurs between the months of September and
October and is caused by a southerly shift in the monsoon circulation pattern, which is characterized
by heavy rainfall. This indicated that the flow characteristic in this river basin is highly controlled by
the monsoon rainfall pattern, especially during the wettest period from August of November when
the correlation between the monthly discharge and the monthly rainfall was so high.
Figure 12. Monthly average hydrograph at Bak Trakuon Station during the calibration period from
1995 to 2008 (excluding 1997 and 1998 when rainfall data were missing).
Additionally, the physiographic condition likely influences the hydrological process as well,
particularly at beginning of the rainy season from April to July when the correlation between the
monthly discharge and the monthly rainfall was low as the result of a lag time in the hydrograph.
With the size of the drainage area at Bak Trakuon Station, the elongated shape, the forested land
cover with varying densities, and the textures of Dystric Leptosol and Cambisol may also contribute
to this broad rising limb in the hydrograph (concentration curve of the hydrograph from April to
September) owing to retardation of overland flow and the increase of infiltration and storage
capacities of the soils [51]. However, the sharp slope of the falling limb of the hydrograph (recession
curve of the hydrograph from late October to December) is likely due to the hilly terrain topography
of the drainage area that form a large stream and valley slopes, which result in quick depletion of
storage [51].
Figure 12. Monthly average hydrograph at Bak Trakuon Station during the calibration period from1995 to 2008 (excluding 1997 and 1998 when rainfall data were missing).
Additionally, the physiographic condition likely influences the hydrological process as well,particularly at beginning of the rainy season from April to July when the correlation between themonthly discharge and the monthly rainfall was low as the result of a lag time in the hydrograph.With the size of the drainage area at Bak Trakuon Station, the elongated shape, the forested land coverwith varying densities, and the textures of Dystric Leptosol and Cambisol may also contribute to thisbroad rising limb in the hydrograph (concentration curve of the hydrograph from April to September)owing to retardation of overland flow and the increase of infiltration and storage capacities of thesoils [51]. However, the sharp slope of the falling limb of the hydrograph (recession curve of the
Water 2020, 12, 2901 19 of 22
hydrograph from late October to December) is likely due to the hilly terrain topography of the drainagearea that form a large stream and valley slopes, which result in quick depletion of storage [51].
Compared to other land covers, a forested watershed has some unique features. Mature forestshave relatively large aboveground (i.e., over-story and under-story layers) and belowground (i.e., roots)biomass [52,53]. They generally have a higher canopy surface roughness, higher leaf area index,and deeper roots compared to crops and/or grass [54], which result in a relatively high ET [55] andsoil infiltration capacity. Forest soil permeability would be maintained by defoliation and organicmatter supply from the biomass and considerably reduce the potential for surface flow, lower the totalstream flow, and lower the peak flow [56]. Generally, forest stream flow originates from subsurfaceflow (lateral flow) or groundwater discharge at headwater streams.
Based on the hydrograph and physiographic characteristics of this study area, the NSE, RSR,and MNS objective functions in addition to their goodness of fits for simulating the discharge (Figures 3and 4) were found to be among the better objective functions for estimating hydrological components,such as a higher ET, lower surface runoff, larger lateral flow, smaller groundwater flow, greater revap,and lower water yield (Figure 5). Moreover, their performances in simulating hydrograph componentswere overall better than other objective functions, especially the NSE and RSR objective functions,when simulating the base flow, rising limb, and falling limb (Figures 8, 9 and 11).
6. Conclusions
The SWAT model was employed to simulate the stream flow of Pursat River Basin. Eight differentobjective functions were used in the calibration process of SWAT-CUP with the SUFI-2 algorithmto examine their influences on the calibration results, parameter optimizations, and water resourcesestimations. Many objective functions performed better than satisfactorily for calibrating the SWATmodel. However, different objective functions defined different fitted values and sensitivity ranks ofthe calibrated parameters, resulting in different estimations of water balance components and wateryield. Among the objective functions used in this study, the Nash–Sutcliffe efficiency (NSE) andratio of standard deviation of the observations to root mean square error (RSR) are equivalent andproduced identical simulation results, including the parameter sensitivity and fitted values of thecalibrated parameters, leading to the same water balance components and water yields estimationresults. By taking results and data of previous studies and according to the characteristics of theriver basin, either NSE or RSR objective functions gave the best estimation results of annual averagewater yield and other water balance components, such as annual average evapotranspiration (ET),groundwater flow (GWQ), surface runoff (SurQ), and lateral flow (LatQ), because these equivalentobjective functions generated reasonable fitted values of the calibrated parameters. Moreover, either ofthem was also better at calibrating the base flow, falling limb, and overall the entire flow phases of thehydrograph for this area.
Author Contributions: Conceptualization and methodological framework, D.S. and T.K.; investigation and datacollection, D.S. and P.T.; SWAT model setup, calibration and validation, D.S. and L.H.T.; MODIS ET analysis,A.F. and T.K.; result analysis and visualization, D.S.; writing—original draft preparation, D.S.; writing—reviewand editing, T.K., C.O., L.H.T., P.T., A.F. and D.S.; supervision, T.K. and C.O. All authors have read and agreed tothe published version of the manuscript.
Funding: This work was partially supported by JSPS KAKENHI Grant Number JP18H02295.
Acknowledgments: The authors also would like to thank Enago (www.enago.jp) for the English language review.
Conflicts of Interest: The authors declare no conflict of interest.
Water 2020, 12, 2901 20 of 22
References
1. Gosain, A.K.; Rao, S.; Basuray, D. Climate change impact assessment on hydrology of Indian river basins.Curr. Sci. 2006, 90, 9.
2. Rostamian, R.; Jaleh, A.; Afyuni, M.; Mousavi, S.F.; Heidarpour, M.; Jalalian, A.; Abbaspour, K.C. Applicationof a SWAT model for estimating runoff and sediment in two mountainous basins in central Iran. Hydrol. Sci. J.2008, 53, 977–988. [CrossRef]
3. Huang, T.; Lo, K. Effects of Land Use Change on Sediment and Water Yields in Yang Ming Shan NationalPark, Taiwan. Environments 2015, 2, 32–42. [CrossRef]
4. Näschen, K.; Diekkrüger, B.; Evers, M.; Höllermann, B.; Steinbach, S.; Thonfeld, F. The Impact of LandUse/Land Cover Change (LULCC) on Water Resources in a Tropical Catchment in Tanzania under DifferentClimate Change Scenarios. Sustainability 2019, 11, 7083. [CrossRef]
5. Cambien, N.; Gobeyn, S.; Nolivos, I.; Forio, M.A.E.; Arias-Hidalgo, M.; Dominguez-Granda, L.; Witing, F.;Volk, M.; Goethals, P.L.M. Using the Soil and Water Assessment Tool to Simulate the Pesticide Dynamics inthe Data Scarce Guayas River Basin, Ecuador. Water 2020, 12, 696. [CrossRef]
6. Arnold, J.G.; Srinivasan, R.; Muttiah, R.S.; Williams, J.R. Large area hydrologic modeling and assessment,Part I: Model development. J. Am. Water Resour. Assoc. 1998, 34, 73–89. [CrossRef]
7. Arnold, J.G.; Moriasi, D.N.; Gassman, P.W.; Abbaspour, K.C.; White, M.J.; Srinivasan, R.; Santhi, C.;Harmel, R.D.; Van Griensven, A.; Van Liew, M.W.; et al. SWAT: Model Use, Calibration, and Validation.Trans. ASABE 2012, 55, 1491–1508. [CrossRef]
8. Neitsch, S.L.; Arnold, J.G.; Kiniry, J.R.; Williams, J.R. Soil and Water Assessment Tool—Theoretical Documentation(Version 2005); Agricultural Research Service: Temple, TX, USA, 2005.
9. Neitsch, S.L.; Arnold, J.G.; Kiniry, J.R.; Williams, J.R. Soil and Water Assessment Tool Theoretical DocumentationVersion 2009; Texas Water Resources Institute Technical Report No. 406; Texas A&M University System:College Station, TX, USA, 2011.
10. Gassman, P.W.; Reyes, M.R.; Green, C.H.; Arnold, J.G. The Soil and Water Assessment Tool:Historical Development, Applications, and Future Research Directions. Trans. ASABE 2007, 50, 1211–1250.[CrossRef]
11. Abbaspour, K.C.; Rouholahnejad, E.; Vaghefi, S.; Srinivasan, R.; Yang, H.; Kløve, B. A continental-scalehydrology and water quality model for Europe: Calibration and uncertainty of a high-resolution large-scaleSWAT model. J. Hydrol. 2015, 524, 733–752. [CrossRef]
12. Abbaspour, K.C. SWAT-CUP: SWAT Calibration and Uncertainty Programs—A User Manual; Swiss FederalInstitute of Aquatic Science and Technology: Zurich, Switzerland, 2015.
13. Ha, L.T.; Bastiaanssen, W.G.M.; van Griensven, A.; van Dijk, A.I.J.M.; Senay, G.B. SWAT-CUP for Calibrationof Spatially Distributed Hydrological Processes and Ecosystem Services in a Vietnamese River Basin UsingRemote Sensing. Hydrol. Earth Syst. Sci. Discuss. 2017, 1–35. [CrossRef]
14. Roth, V.; Nigussie, T.K.; Lemann, T. Model parameter transfer for streamflow and sediment loss predictionwith SWAT in a tropical watershed. Environ. Earth Sci. 2016, 75, 1321. [CrossRef]
15. Gholami, A.; Habibnejad Roshan, M.; Shahedi, K.; Vafakhah, M.; Solaymani, K. Hydrological stream flowmodeling in the Talar catchment (central section of the Alborz Mountains, north of Iran): Parameterizationand uncertainty analysis using SWAT-CUP. J. Water Land Dev. 2016, 30, 57–69. [CrossRef]
16. Abbaspour, K.C.; Vejdani, M.; Haghighat, S. SWAT-CUP Calibration and Uncertainty Programs for SWAT;Modelling and Simulation Society of Australia and New Zealand: Christchurch, New Zealand, 2007;pp. 1596–1602.
17. Khoi, D.N.; Thom, V.T. Parameter uncertainty analysis for simulating streamflow in a river catchment ofVietnam. Glob. Ecol. Conserv. 2015, 4, 538–548. [CrossRef]
18. Shivhare, N.; Dikshit, P.K.S.; Dwivedi, S.B. A Comparison of SWAT Model Calibration Techniques forHydrological Modeling in the Ganga River Watershed. Engineering 2018, 4, 643–652. [CrossRef]
19. Sloboda, M.; Swayne, D. Autocalibration of Environmental Process Models Using a PAC LearningHypothesis. In Environmental Software Systems. Frameworks of eEnvironment; IFIP Advances in Informationand Communication Technology; Springer: Berlin/Heidelberg, Germany, 2011; Volume 359, pp. 528–534.ISBN 978-3-642-22284-9.
Water 2020, 12, 2901 21 of 22
20. Kouchi, D.H.; Esmaili, K.; Faridhosseini, A.; Sanaeinejad, S.H.; Khalili, D.; Abbaspour, K.C. Sensitivity ofCalibrated Parameters and Water Resource Estimates on Different Objective Functions and OptimizationAlgorithms. Water 2017, 9, 384. [CrossRef]
21. Ashwell, D.; Lic, V.; Loeung, K.; Maltby, M.; McNaughton, A.; Mulligan, B.; Oum, S.; Starr, A.Baseline Assessment and Recommendations for Improved Natural Resources Management and BiodiversityConservation in the Tonle Sap Basin, Cambodia; Fauna & Flora International: Phnom Penh, Cambodia,2011; 92p.
22. Cambodia National Mekong Committee (CNMC). Basin Development Plan Program (Ed.): Profile of the TonleSap Sub-Area (SA-9C); CNMC: Phnom Penh, Cambodia, 2012.
23. Japan International Cooperation Agency (JICA). Report on Examination of Impact of New Dam Plans on the WestTonle Sap Irrigation Rehabilitation Project in Pursat River Basin; Ministry of Water Resources and Meteorology(MOWRAM): Phnom Penh, Cambodia, 2011.
24. Japan International Cooperation Agency (JICA). Brief Progress Report on the Water Balance Examination Studyfor Pursat and Baribor River Basins; Ministry of Water Resources and Meteorology (MOWRAM): Phnom Penh,Cambodia, 2013.
25. Abbaspour, K.C.; van Genuchten, M.T.; Schulin, R.; Schläppi, E. A sequential uncertainty domain inverseprocedure for estimating subsurface flow and transport parameters. Water Resour. Res. 1997, 33, 1879–1892.[CrossRef]
26. Abbaspour, K.C.; Johnson, C.A.; Van Genuchten, M.Th. Estimating Uncertain Flow and Transport ParametersUsing a Sequential Uncertainty Fitting Procedure. Vadose Zone J. 2004, 3, 1340–1352. [CrossRef]
27. Rossi, C.G.; Srinivasan, R.; Jirayoot, K.; Duc, T.L.; Souvannabouth, P.; Binh, N.; Gassman, P.W. HydrologicEvaluation of the Lower Mekong River Basin with the Soil and Water Assessment Tool Model. Int. Agric.Eng. J. 2009, 18, 1–13.
28. Ly, S.; Oeurng, C. Climate change and water governance in Stung Chrey Bak Cathment of Tonle Sap GreatLake Basin in Cambodia. In Proceedings of the 15th Annual Conference of the Science Council of Asia (SCA),Siem Reap, Cambodia, 17–19 May 2015; pp. 134–138.
29. Ang, R.; Oeurng, C. Simulating streamflow in an ungauged catchment of Tonlesap Lake Basin in Cambodiausing Soil and Water Assessment Tool (SWAT) model. Water Sci. 2018, 32, 89–101. [CrossRef]
30. Krause, P.; Boyle, D.P.; Bäse, F. Comparison of different efficiency criteria for hydrological model assessment.Adv. Geosci. 2005, 5, 89–97. [CrossRef]
31. Nash, J.E.; Sutcliffe, J.V. River flow forecasting through conceptual models part I—A discussion of principles.J. Hydrol. 1970, 10, 282–290. [CrossRef]
32. Moriasi, D.N.; Arnold, J.G.; Liew, M.W.V.; Bingner, R.L.; Harmel, R.D.; Veith, T.L. Model EvaluationGuidelines for Systematic Quantification of Accuracy in Watershed Simulations. Trans. ASABE 2007, 50,885–900. [CrossRef]
33. Van Griensven, A.; Bauwens, W. Multiobjective autocalibration for semidistributed water quality models:Autocalibrations for water quality models. Water Resour. Res. 2003, 39, 1348. [CrossRef]
34. Gupta, H.V.; Kling, H.; Yilmaz, K.K.; Martinez, G.F. Decomposition of the mean squared error and NSEperformance criteria: Implications for improving hydrological modelling. J. Hydrol. 2009, 377, 80–91.[CrossRef]
35. Yapo, P.O.; Gupta, H.V.; Sorooshian, S. Automatic calibration of conceptual rainfall-runoff models: Sensitivityto calibration data. J. Hydrol. 1996, 181, 23–48. [CrossRef]
36. James, L.D.; Burges, S.J. Selection, Calibration, and Testing of Hydrologic Models. In Hydrologic Modeling ofSmall Watersheds; American Society of Agricultural Engineers: St. Joseph, MI, USA, 1982; pp. 437–470.
37. Muleta, M.K. Model Performance Sensitivity to Objective Function during Automated Calibrations.J. Hydrol. Eng. 2012, 17, 756–767. [CrossRef]
38. Mehdi, B.; Ludwig, R.; Lehner, B. Evaluating the impacts of climate change and crop land use change onstreamflow, nitrates and phosphorus: A modeling study in Bavaria. J. Hydrol. Reg. Stud. 2015, 4, 60–90.[CrossRef]
39. Akhavan, S.; Abedi-Koupai, J.; Mousavi, S.-F.; Afyuni, M.; Eslamian, S.-S.; Abbaspour, K.C. Application ofSWAT model to investigate nitrate leaching in Hamadan–Bahar Watershed, Iran. Agric. Ecosyst. Environ.2010, 139, 675–688. [CrossRef]
Water 2020, 12, 2901 22 of 22
40. Thiemig, V.; Rojas, R.; Zambrano-Bigiarini, M.; De Roo, A. Hydrological evaluation of satellite-based rainfallestimates over the Volta and Baro-Akobo Basin. J. Hydrol. 2013, 499, 324–338. [CrossRef]
41. Running, S.; Mu, Q.; Zhao, M. MOD16A2 MODIS/Terra Net Evapotranspiration 8-Day L4 Global 500m SIN GridV006 [Data Set]; NASA EOSDIS Land Processes DAAC: Sioux Falls, SD, USA, 2017.
42. Busetto, L.; Ranghetti, L. MODIStsp: An R package for automatic preprocessing of MODIS Land Productstime series. Comput. Geosci. 2016, 97, 40–48. [CrossRef]
43. Kobayashi, K.; Salam, M.U. Comparing Simulated and Measured Values Using Mean Squared Deviation andits Components. Agron. J. 2000, 92, 345–352. [CrossRef]
44. Gauch, H.G.; Hwang, J.T.G.; Fick, G.W. Model Evaluation by Comparison of Model-Based Predictions andMeasured Values. Agron. J. 2003, 95, 1442–1446. [CrossRef]
45. Legates, D.R.; McCabe, G.J. Evaluating the use of “goodness-of-fit” Measures in hydrologic and hydroclimaticmodel validation. Water Resour. Res. 1999, 35, 233–241. [CrossRef]
46. Harmel, R.D.; Smith, P.K.; Migliaccio, K.W. Modifying Goodness-of-Fit Indicators to Incorporate BothMeasurement and Model Uncertainty in Model Calibration and Validation. Trans. ASABE 2010, 53, 55–63.[CrossRef]
47. Shimizu, A.; Suzuki, M.; Sawano, S.; Kabeya, N.; Nobuhiro, T.; Tamai, K.; Tsuboyama, Y.; Chann, S.; Keth, N.Water Resources Observation and Large-scale Model Estimation in Forested Areas in Mekong River Basin.Jpn. Agric. Res. Q. 2010, 44, 179–186. [CrossRef]
48. Japan International Cooperation Agency (JICA). The Study on Groundwater Development in Southern Cambodia;Ministry of Rural Development: Phnom Penh, Cambodia, 2002; 309p.
49. Rafiei Emam, A.; Kappas, M.; Hoang Khanh Nguyen, L.; Renchin, T. Hydrological Modeling in an UngaugedBasin of Central Vietnam Using SWAT Model. Hydrol. Earth Syst. Sci. Discuss. 2016, 1–33. [CrossRef]
50. Harmel, R.D.; Smith, P.K. Consideration of measurement uncertainty in the evaluation of goodness-of-fit inhydrologic and water quality modeling. J. Hydrol. 2007, 337, 326–336. [CrossRef]
51. Subramanya, K. Engineering Hydrology; Tata McGraw-Hill: New York, NY, USA, 2008; ISBN 978-0-07-133748-9.52. Waring, R.H.; Running, S.W. Forest Ecosystems: Analysis at Multiple Scales, 3rd ed.; Elsevier/Academic Press:
Amsterdam, Boston, 2007; ISBN 978-0-12-370605-8.53. Chapin, F.S.; Matson, P.A.; Vitousek, P.M. Principles of Terrestrial Ecosystem Ecology; Springer: New York, NY,
USA, 2011; ISBN 978-1-4419-9503-2.54. Bonan, G.B. Forests and Climate Change: Forcings, Feedbacks, and the Climate Benefits of Forests. Science
2008, 320, 1444–1449. [CrossRef]55. Sun, G.; Alstad, K.; Chen, J.; Chen, S.; Ford, C.R.; Lin, G.; Liu, C.; Lu, N.; McNulty, S.G.; Miao, H.; et al.
A general predictive model for estimating monthly ecosystem evapotranspiration. Ecohydrology 2011, 4,245–255. [CrossRef]
56. Sun, G.; Lockaby, B.G. Water Quantity and Quality at the Urban-Rural Interface. In Urban-Rural Interfaces;American Society of Agronomy, Soil Science Society of America, Crop Science Society of America, Inc.:Madison, WI, USA, 2012; pp. 29–48. ISBN 978-0-89118-616-8.
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutionalaffiliations.
© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open accessarticle distributed under the terms and conditions of the Creative Commons Attribution(CC BY) license (http://creativecommons.org/licenses/by/4.0/).
top related