10/4/2017

NG

WA

WH

ITE

PAPE

R

Modeling groundwater in the subsurface environment gives us a view and insight of what cannot be easily seen nor readily observed. The U.S. Geological Survey anticipates future modeling demands will be greater, especially addressing multi-disciplinary factors and applied to landscape-level science as scientific questions and resource issues become more complex. The National Ground Water Association (NGWA) determined that to further support the many interests affected by groundwater, it should mobilize its members to collaborate to improve groundwater modeling and its application. On April 14, 2016, NGWA convened nearly 40 prominent groundwater modelers to form the Groundwater Modeling Advisory Panel (GMAP).

The goal of GMAP is to advance the state of groundwater modeling through cooperative information exchange and outreach to groundwater professionals. The GMAP objectives are to address groundwater modeling practice and to research questions and identify alternative/best techniques and responses. GMAP members intend to provide the scientists and engineers involved in groundwater modeling applications—professionals and interested members of the public—with our collective understanding of the subject matter and observations for consideration in professional practice. We decided not to develop standards, as our members saw the modeling field on a path of continuing and rapid evolution.

The members of GMAP identified 33 topics of interest and categorized them into five groups for development. Each group had eight to 15 members and two Co-Leads who facilitated discussions and

Uncertainty in Groundwater ModelingPreface for Groundwater Modeling Advisory Panel White Papers

PrecipitationPrecipitation

2

organized preparation of modeling practice white papers. The five initial topic groups were: field complexity, stepwise/analytical element modeling, uncertainty in modeling, model applications and integrated groundwater/surface water modeling. The papers are written as discussion documents to be updated.

The groups met monthly, distributed document development responsibilities, drafted sections of papers, completed their selected modeling practice topic papers, and conducted peer review from approximately June 2016 to June 2017. The NGWA Board of Directors approved the papers in October 2017.

The papers address questions of:• How should decision makers consider groundwater modeling in project development and solution?• How should the complexity of the subsurface be considered in developing groundwater models?• What considerations should be made in moving from simple to more complex model development?• How can uncertainty be included in modeling to inform decisions for groundwater supply and

emediation?• What approaches can be followed to address interaction of groundwater and surface water in

decisions?GMAP members believe sharing their knowledge and experience will assist both long-time practitioners

and newly-graduated modelers in applying their skills and expertise. The Advisory Panel also hopes modelers will bring other questions forward for consideration and development in future groundwater modeling practice white papers. Future discussions and papers are already planned.

Uncertainty in Groundwater ModelingThe individuals who gave of their time and expertise on this paper were:

Authors Reviewers

Steve Luis Eve Kuniansky Peter Schulmeyer Jaco Nel Rod Sheets Miln Harvey Paul Martin Peter Mock Michael LeFrancois Jill Van Dyke

David Bean

IntroductionUncertainty is inherent in the groundwater modeling process. This paper discusses sources of uncertainty

and highlights some of the tools, methods, approaches, and good practices to help evaluate, quantify, and communicate model uncertainty. Model applications vary considerably and each site is unique, so practitioners should exercise their own judgment when deciding how best to address and document groundwater modeling uncertainties.

I. What Is a Model and How Does Uncertainty Arise?All models are simplified representations of a system, including groundwater models (Oreskes et al. 1994;

Poeter 2007). The groundwater model is used to forecast results to answer questions such as “How much drawdown will there be from a well or a wellfield pumped at some specified volume?” “What could be the long-term effects of this pumping?” Or perhaps, “How long and what path may a contaminant take to reach some point?” These types of questions can be answered by using various mathematical (analytical or numerical) models. • A mathematical model describes the physical processes and boundaries of a groundwater system using

one or more governing equations.

3

• An analytical model is an exact solution of the partial differential equation for groundwater flow given specific boundary conditions and is based on simplifying assumptions (simple geometry, uniform aquifer properties, water of constant density and viscosity, laminar flow, and/or steady-state conditions) to calculate a solution by hand, or by using a software. Most aquifer test analysis software has many different analytical solutions as each one is designed for specific boundary conditions and/or aquifer types.

• A numerical model in the paper refers to the solution for groundwater flow or transport equation(s) by finite-difference or finite element numerical methods that involve piecewise approximation of the groundwater flow system with solution of the flow based on cells or elements that represent discrete volumes of the aquifer. Thus, a more complete representation of a hydrogeologic system within space and time can be represented, such as complex geometry of the aquifer and boundary conditions, which may allow for fewer simplifying assumptions and more tractable solutions to governing equation(s).

Features of the governing equations and boundary conditions that can be specified include aquifer geometry, hydrogeological properties, stresses via groundwater extraction or injection, spatial and temporal changes in recharge, and solute sources and sinks. By incorporating these features, hydrogeologists are able to develop a more complex, and potentially more realistic, representation of a groundwater system than with an analytical model.

To better understand the challenges confronted by the groundwater modeler, Doherty et al. (2010) propose the following thought experiment:

“Suppose that the algorithmic basis of a numerical model is such that the model’s ability to simulate environmental processes at a site is perfect. Such a model would, of necessity, be complex. Furthermore, it would need to account for the spatial variability of hydraulic and other properties of the system that it is to simulate. If these properties were all known and the model was parameterized accordingly, the model would predict, with perfect accuracy, the response of the system under study to a set of user- supplied inputs.”

Unfortunately, the hydrogeologist has to deal with many issues in developing a groundwater model such as the lack of or poor quality of data, data integrity, challenging project objectives, small budgets, and time constraints. The hydrogeologist must reconcile these competing, and sometimes contradictory, issues, often through model simplifications and reduction in data representativeness. In effect, the hydrogeologist must answer questions such as:

• How do I find a solution to the question without simplifying the system to such an extent that causes a lack of hydrogeological representation?

• How confident am I in the model results? To help address such questions, uncertainty analysis comes into play. Rubin (2003) states a good model

is one that accounts accurately for the uncertainty associated with its forecasts (predictions) and provides asymptotically better performance as more data becomes available.

[For further perspective on considering and adding complexity to models, please see the NGWA Groundwater Modeling Advisory Panel papers on “A Decision Framework for Minimum Levels of Model Complexity” and “A Stepwise Approach to Groundwater Modeling.”]

II. Why Perform Uncertainty Analysis—Its Value to the Modeling EffortIn any technical analysis, it is common practice for the practitioner to provide an assessment of the

reliability and uncertainty associated with that analysis. In most modeling applications, models are used as bases for making decisions. Moreover, models are often used as decision-making tools by end users with little or no modeling experience and with limited ability to independently assess model efficacy or

4

reliability (e.g., clients and regulators). Therefore, it is appropriate for groundwater modeling practitioners to perform uncertainty analyses and provide an indication of model reliability and limitations (Anderson et al. 2015; Hill and Tiedeman 2007; Hunt et al. 2013; Reilly and Harbaugh 2004). Indeed, some have argued that doing so is the responsibility of the modeling community (Pappenberger and Beven 2006). Benefits of uncertainty analysis for groundwater models include:

• Identifying data needs to focus investigation efforts in order to optimize allocation of field investigation budgets (e.g., data worth analysis)

• Identifying parameters with the greatest uncertainty for continued refinement to optimize allocation of model-development budgets

• Providing basis for more formal decision analyses, including evaluation of the consequences of incorrect decisions

• Managing expectations of end users (those who must rely upon model results for decision-making purposes) and the public (i.e., those who are affected by decisions based on models).

III. What Are the Drivers for Evaluating Uncertainty—The “Bad” NewsThere are several sources of uncertainty, or “error,” in a model of a hydrogeologic system, including

conceptual/model error, parameter error, structural noise, and predictive error. In this section we outline these sources of uncertainties, with examples of each, in order to help understand and attempt to reduce uncertainty.

Conceptual/Model Error occurs when conceptualizing and constructing a model. Since a model is a simplified version of reality, and system completeness is not known, or when conceptualizing/constructing a model, system assumptions have to be made. These assumptions can lead to various types of error and include the following general categories:

• Incorrect hypotheses—Is the system really understood? For example, oversimplification of the aquifer system, assuming confined conditions, and assuming steady-state conditions.

• Missing processes—Are important processes being missed? For example, ignoring surface water/groundwater interactions or specifying incorrect boundary inflow/outflow, or neglecting internal stresses such as wells or not including recharge and/or evapotranspiration.

• Structural complexity—Is the model too complex, or too simple? For example, too many (or too few) layers, lack of vadose zone, lack of parameter heterogeneity, failure to consider aquifer storage, failure to consider aquifer compaction.

• Temporal complexity—Are the influence of temporal stresses well understood via the influence on results and predictions? For example, oversimplification of boundary conditions (i.e., nearby or connected surface water bodies, rivers, recharge), and/or aquifer stresses (i.e., annualizing well pumping) and assuming steady-state flow conditions.

Parameter Error occurs when developing model parameter values. Parameter values (e.g., hydraulic properties) that go into a model are always uncertain. Nonetheless, modelers often assume the hydrogeologic system behaves in such a fashion that parameter values can be determined with a high degree of certainty. However, parameter errors result from the following factors:

• Measurement errors—Are there errors in field measurements, and if so, how will they affect model parameters? For example, drilling log water levels vs. water levels measured in developed wells.

• Interpolation errors—Are field data sufficient to adequately “map” hydraulic properties? For example, limited aquifer tests when aquifer characteristics are known to vary.

• Scaling errors—Is the scale at which hydraulic properties are measured consistent with the scale applied in the model? For example, using hydraulic conductivities derived from small-scale slug tests for a large-scale aquifer test simulation.

5

Structural Noise is the lack of fit between simulated quantities and measured quantities that cannot be attributed to measurement error/noise. Structural noise illustrates the imperfect nature of a model to simulate reality (Doherty 2010). A model is calibrated to measured field data such as hydraulic heads, stream baseflows, and groundwater age. Typically, the output of the model is systematically compared to the measured data to reduce the model-to-measurement misfit to an acceptable level defined by the modeler or end user.

Predictive Error (also known as Scenario Uncertainty) arises because future conditions are not known with certainty (Hill and Tiedeman 2007). When a model is built and calibrated, it is usually built to predict a response in the system such as how the system will respond to groundwater pumping at a certain rate at a given location, or how the system will respond to a change in effective recharge over time. However, the future is inherently uncertain, so the model always will be vulnerable to uncertainty.

IV. Approaches for Managing Uncertainty—The “Good” NewsIf the bad news is model uncertainty is inevitable, the good news is there are many methods and tools

available to understand and quantify uncertainty. In this section, brief summaries are provided of readily available uncertainty analysis tools commonly used in groundwater modeling.

MethodsSensitivity analysis is the evaluation of model input parameters to see how much they affect model

outputs, such as heads and flows (Reilly and Harbaugh 2004). Trial-and-error calibration methods lead to manual and automated sensitivity analyses where values, of all but one parameter, are fixed and the single parameter is varied by a specified percentage to evaluate the impact of that change on the model output. This approach is limited because usually only a small subset of parameters, selected by the modeler, is tested and the interactions between the parameters is not accounted for. Automated parameter estimation methods automatically calculate parameter sensitivity coefficients for all calibration targets, thereby allowing for more thorough and rigorous evaluations (Anderson et al. 2015; Hill and Tiedeman 2007).

Parameter sensitivity analysis uses the output of automated parameter estimation methods to develop statistical insights about the model (Anderson et al. 2015; Hill and Tiedeman 2007). Identification of parameters that are statistically correlated, or to which simulation results are insensitive, allow the modeler to focus on estimation of uncorrelated parameters and/or parameters to which simulation results are sensitive. These methods also allow identification of relationships between observations and parameter values, thereby potentially guiding sample collection (Anderson et al. 2015; Hill and Tiedeman 2007).

Scenario sensitivity analysis considers potential impacts to prediction reliability of changes in future conditions (Hill and Tiedeman 2007). Models are often built to forecast what might happen if conditions change. Common forecasting scenarios include evaluating impacts of increased pumping or recharge rates, drought, or other changes in boundary condition. Scenario sensitivity analysis is usually performed using linear uncertainty analysis, which takes into account propagation of measurement error and parameter simplification error, and allows the modeler to create “error bars” on the scenarios.

ToolsThere are many computing tools available that quantify groundwater model uncertainty. These tools are

rooted in statistics and range from relatively simple for a novice modeler to very complex that require specialized training and computing resources. Some of these tools assume a static model conceptualization (i.e., the model structure is correct and uncertainty forecasts arise from parameter uncertainty). Additionally, other tools assume there are many plausible model conceptualizations.

6

Single conceptualization toolsBasic scenario analysis is simply running a model post changing assumed values, within the model,

within their possible range, and plotting the results giving the user a sense of what can happen within the range of values used.

Monte Carlo method uses a large number of “random” simulations that are run using a statistical sampling of a combination of the parameters within the model. Thousands of runs are made, with resulting statistics plotted to understand uncertainty, usually with respect to forecasts. Variations of this method are the Markov Chain Monte Carlo Method and the Null-Space Monte Carlo method.

Multiple conceptualization toolsGeneralized Likelihood Uncertainty Evaluation (GLUE) uses Monte Carlo sampling across many

conceptualizations and structures. The Latin Hyper Cube Sampling approach is based on logistic regression which estimates the effect of various predictors on some binary outcome of interest. This method assumes the uncertainty in logistic regression predictions is a function of uncertainty in both the explanatory variables and the regression coefficients (Gurdak et al. 2007).

Bayesian uncertainty analysis is a statistical procedure that describes the likelihood of an event or result using the current hydrogeologic parameters collected from site characterization. The result is the probability or likelihood that for a given a set of parameters the outcome or result will match our observation perfectly. The advantage of a probabilistic representation is that it conveys estimates of parameter value and their uncertainties (Anderson et al. 2015). Risk analysis can also be derived from the Bayesian analysis to provide trade-off curves for decision making about groundwater resources.

V. Benefits of Uncertainty Analysis and CostThere are many benefits to quantifying uncertainty in groundwater models, whereas one of the only

drawbacks might be the time and effort (translated into cost) it takes to provide the analysis. The expectation is the cost/benefit ratio of providing this information is lower than most decision makers think and gets lower as the practitioner becomes more familiar with the tools available to them. The following are just a few benefits to the decision makers and stakeholders, and in the long run to the practitioner: • Assist decision makers and stakeholders in understanding overall reliability of model predictions—in the

past, clients were told “the result of the modeling shows that (something is going to happen)” when in fact we had doubt that that “something” was actually going to happen (and so did the client!). Now we can tell them to some degree of certainty that that “something” is going to happen—since we can, why don’t we?

• Assist decision makers and stakeholders in understanding specific strengths and weaknesses of the models—we practitioners tend to always point out the strengths of the model and rarely the weaknesses even though we know there could be many. We have the ability with modern tools to show statistically, for example, where the best places to collect new data are—turning a weakness into a strength.

• Emphasize the connection between data availability and model prediction reliability. Using this information, limited resources can be allocated to specific modeling and/or sampling tasks to maximize value of information obtained.[For further perspective on benefits from incremental model development, please refer to the NGWA Groundwater Modeling Advisory Panel paper “A Stepwise Approach to Groundwater Modeling.”]

7

VI. Communicating with Non-Technical Audiences (Public, Clients, Decision Makers) Regarding Model Uncertainty

Some uncertainty analysis methods are relatively complex and can be difficult for non-practitioners to understand. Nonetheless, communicating model uncertainty adds significant value to modeling studies, and communication of uncertainty should be considered an appropriate part of documenting model development and application. This section provides brief suggestions for practitioners seeking to communicate with non-technical audiences regarding model uncertainty.• Although analyses of uncertainty can be complex, communications regarding model uncertainty should

employ everyday language and be easy for stakeholders to understand. Almost everyone understands weather forecasts (there is a 70% chance of rain, followed by some reason they believe that result, like a front is coming through). Using similar language, there is a 70% chance this range of drawdown will occur under this pumping scenario; it is a 70% chance because we had to use a wider range of recharge rates and hydraulic conductivity as a consequence of the field data being sparse.

• To facilitate incorporation into decision making, communications regarding model uncertainty should be relevant to the decision at hand and discuss the potential consequences of simulation results being incorrect. For example, following from above, the modeler could report there is a 30% chance the actual drawdown is outside the given range. The modeler could go on to discuss the potential implications. For example, if the actual drawdown is greater than the predicted range, the well in question may cause problematic drawdown in a neighboring well or, if drawdown is less than predicted, pumping rates could be increased.

• Model development and simulation results should be reported using language that reflects the practitioner’s assessment of uncertainty and confidence in the reliability of the analysis. Qualifying language (e.g., “suggests” vs. “shows”) can be used to provide a qualitative indication of uncertainty. A good practice is to employ a formal framework for the treatment of model uncertainty such as that used in the International Panel on Climate Change’s Uncertainty Guidance Note (Table 1 from Mastrandrea et al. 2010) where words like “likely” or “unlikely” are used and associated with a range of values of likelihood.

Table 1. Terms used when statistical analysis gives the user a probability associated with an outcome (from Mastrandrea et al. 2010).

8

VII. ConclusionGroundwater flow and transport models and their predictions are often used as the bases for making

decisions in the water resources community. Too often in the reporting of the models and predictions, there isn’t a sense of how uncertain these models and predictions are, even though as modelers we know there is error or uncertainty in the model input, boundary conditions, or conceptualization of the system. Regrettably, sometimes the decision makers do not understand or appreciate the uncertainty because of how the discussion of uncertainty is approached. Generally, there is a lack of uniform vernacular related to uncertainty discussions that are presented with groundwater models. As a result, oftentimes the end user of groundwater models is less than confident in the model and its predictions.

In this paper, we took a brief tour through the world of uncertainty in groundwater modeling, from discussions of what groundwater models are, in relation to how uncertainty arises, to statistical techniques that are commonly (and less commonly) used to mathematically describe modeling uncertainty. Uncertainty analyses have many benefits to the end user and to the modeling community and should be more uniformly adopted in the groundwater modeling field. We also address the important issue of communicating uncertainty in groundwater models, suggesting a uniform language to explain probability of some outcome related to the modeling. Simplicity and uniformity in uncertainty discussions will lead to more generally acceptable techniques adopted by the modeling community and to the water resources managers who rely on the modeling community for reasonable and understandable model results and predictions to be used in decision making.

ReferencesAnderson, M.P., W.W. Woessner, and R.J. Hunt. 2015. Applied Groundwater Modeling. Elsevier Press, 564 pp.

Beven, K., and P. Young. 2013. A guide to good practice in modeling semantics for authors and referees. Water

Resources Research, vol. 49, pp. 5092-5098.

Doherty, J., and D. Welter. 2010. A short exploration of structural noise. Water Resources Research, 46,

10.1029/2009WR008377

Gurdak, J.J., J.E. McCray, G. Thyne, and S.L. Qi. 2007. Latin hypercube approach to estimate uncertainty in ground water

vulnerability. Ground Water, vol. 45, no. 3, pp. 348-361.

Hill, M.C., and C.R. Tiedeman. 2007. Effective Groundwater Model Calibration: With Analysis of Data, Sensitivities,

Predictions, and Uncertainty. John Wiley & Sons Inc., Hoboken, New Jersey, USA. doi: 10.1002/9780470041086.index

Hunt, R.J. 2012. Uncertainty in Australian groundwater modeling guidelines. Canberra, Australia, National Water

Commission, Waterlines Report Series No. 82, pp. 92-105.

Mastrandrea, M.D., C.B. Field, T.F. Stocker, O. Edenhofer, K.L. Ebi, D.J. Frame, H. Held, E. Kriegler, K.J. Mach, P.R. Matschoss,

G.-K. Plattner, G.W. Yohe, and F.W. Zwiers. 2010. Guidance Note for Lead Authors of the IPCC Fifth Assessment Report

on Consistent Treatment of Uncertainties. Intergovernmental Panel on Climate Change (IPCC). Available at <http://

www.ipcc.ch>.

Oreskes, N., K. Schrader-Frechette, and K. Belitz. 1994. Verification, validation, and confirmation of numerical models in

the Earth sciences. Science, vol. 263, February 4, pp. 641-646.

Pappenberger, F., and K.J. Beven. 2006. Ignorance is bliss: Or seven reasons not to use uncertainty analysis. Water

Resources Research, 42, W05302, doi:10.1029/2005WR004820

Poeter, E. 2007. All models are wrong: How do we know which are useful?—Looking back at the 2006 Darcy Lecture

Tour. Ground Water, volume 45, issue 4, pp. 390-391: doi: 10.1111/j.1745-6584.2007.00350.x

Reilly, T.E., and A.W. Harbaugh. 2004. Guidelines for evaluating ground-water flow models. U.S. Geological Survey

Scientific Investigations Report 2004-5038. https://pubs.usgs.gov/sir/2004/5038/PDF/SIR20045038_ver1.01.pdf

Rubin, Y. 2003. Applied Stochastic Hydrogeology. Oxford and New York, Oxford University Press, 391 pp.

9

© 2017 by National Ground Water Association

ISBN 1-56034-042-8

Published by: NGWA Press National Ground Water Association Address 601 Dempsey Road, Westerville, Ohio 43081-8978 U.S.A Phone (800) 551-7379 * (614) 898-7791 Fax (614) 898-7786 Email ngwa@ngwa.org Website NGWA.org and WellOwner.org

The GroundwaterNGWA

Association

SM

Pres

s

Disclaimer: This White Paper is provided for information purposes only so National Ground Water

Association members and others using it are encouraged, as appropriate, to conduct an independent

analysis of the issues. NGWA does purport to have conducted a definitive analysis on the topic

described, and assumes no duty, liability, or responsibility for the contents of this White Paper.

Those relying on this White Paper are encouraged to make their own independent assessment and

evaluation of options as to practices for their business and their geographic region of work.

Trademarks and copyrights mentioned within the White Paper are the ownership of their respective

companies. The names of products and services presented are used only in an education fashion

and to the benefit of the trademark and copyright owner, with no intention of infringing on

trademarks or copyrights. No endorsement of any third-party products or services is expressed or

implied by any information, material, or content referred to in the White Paper.

The National Ground Water Association is a not-for-profit professional society and trade association

for the global groundwater industry. Our members around the world include leading public and

private sector groundwater scientists, engineers, water well system professionals, manufacturers,

and suppliers of groundwater-related products and services. The Association’s vision is to be the

leading groundwater association advocating for responsible development, management, and use

of water.