1
Integrated Environmental Modeling: A Vision and Roadmap for the Future
Author List: Gerard F. Laniak1, Gabriel Olchin
2, Jonathan Goodall
3, Alexey Voinov
4, Mary
Hill5, Pierre Glynn
6, Gene Whelan
1, Gary Geller
7, Nigel Quinn
8, Michiel Blind
9, Scott
Peckham10
, Sim Reaney11
, Noha Gaber
12, Robert Kennedy
13, Andrew Hughes
14
1US Environmental Protection Agency, Office of Research and Development
2US Environmental Protection Agency, Office of the Science Advisor
3 University of South Carolina, Department of Civil and Environmental Engineering, USA
4 University of Twente, Faculty of Geo-Information Science and Earth Observation (ITC),
Netherlands
5 US Geological Survey, National Research Program
6 US Geological Survey, National Research Program
7 Jet Propulsion Laboratory, California Institute of Technology
8 Berkeley National Laboratory, USA
9 Deltares, Netherlands
10 INSTAAR, University of Colorado – Boulder, USA
11 Durham University, Department of Geography, UK
12 US Environmental Protection Agency, Office of the Administrator
13 US Army Corps of Engineers, Engineer Research and Development Center
14 British Geological Survey, Keyworth, UK
2
Abstract: Integrated environmental modeling (IEM) represents a new paradigm, inspired by modern
environmental problems, decisions, and policies and enabled by transdisciplinary science and
computer capabilities that allow the environment to be considered in a fundamentally different
way. The problems are characterized by the extent of the geo-ecological system involved, the
dynamic and interdependent nature of stressors and their impacts, the diversity of stakeholders,
and the integration of social, economic, and political considerations. IEM provides a science-
based structure to develop and organize relevant knowledge and information and apply it to
explain, explore, and forecast the behavior of environmental systems in response to human and
natural sources of stress. During the past several years a number of workshops were held that
brought IEM practitioners together to share experiences and discuss future needs and directions.
In this paper we present the results of these discussions. First, to provide definition and context
we present a comprehensive view of IEM as a landscape involving four principal elements:
applications, science, technology, and community. This is followed by a vision statement and a
roadmap that includes a set of guiding principles, goals, and activities needed to achieve the
vision. The vision is of a global scale IEM community of practice that leverages modern
technologies to streamline the movement of science-based knowledge from its sources in
research, through its organization into databases and models, to its integration and application for
problem solving purposes. Achieving this vision will require that the global community of IEM
stakeholders transcend social, political and organizational boundaries and pursue greater levels
of collaboration, the first focus of which should be the development of standards for publishing
IEM data and models in forms suitable for automated discovery, access and integration. The
education of the next generation of environmental stakeholders, with a focus on transdisciplinary
research, development, and decision making, is also of primary importance.
1. Introduction
Integrated environmental modeling (IEM) is an emerging discipline inspired by the need to solve
increasingly complex real world problems involving the environment and its relationship to
human systems and activities (technological, social, and economic). The complex and
interrelated nature of real-world problems has lead to a need for higher-order systems thinking
3
and holistic solutions (EPA, 2008b; MEA, 2005; Parker et al., 2002). IEM provides a science-
based structure to develop and organize multidisciplinary knowledge. It provides a means to
apply this knowledge to explain, explore, and forecast environmental system response(s) to
natural and human-induced stressors. By its very nature, it breaks down research silos and brings
scientists from multiple disciplines together with decision makers and other stakeholders to solve
problems for which the social, economic, and environmental considerations are highly
interdependent. This movement toward transdisciplinarity (Tress et al., 2005) and participatory
modeling (Voinov and Bousquet, 2010) fosters increased knowledge and understanding of the
system, reduces the perception of ‗black-box‘ modeling, and increases awareness and detection
of unintended consequences of decisions and policies.
While IEM‘s concepts and early models are now more than thirty years old (Meadows et al.,
1972; Bailey et al., 1985; Cohen, 1986; Mackay, 1991), it is only during recent years that it has
became recognized and appreciated by the greater stakeholder community. National and
international organizations have commissioned studies to determine research directions and
priorities for integrated modeling (ICSU, 2010; NSF1; Blind et al, 2005a,b; EC, 2000,
Schellekens et al, 2011). Senior managers in government, academia, and commercial
organizations are restructuring operations to facilitate integrated and transdisciplinary
approaches (EPA, 2008b). Mid-level managers who realize that no single group has the
comprehensive expertise needed for integrated modeling are actively pursuing inter-organization
collaborations (e.g. ISCMEM2; OpenMI, 2009, Delsman et al, 2009). Researchers and
technology developers, the early adopters of the IEM paradigm, continue a range of research to
develop the science and technologies that enable integrated assessment. Environmental assessors
are utilizing IEM science and technologies to build integrated modeling systems that will address
specific problems at varying scales; examples from the US include the Chesapeake Bay (Linker
et al., 1999) and the San Joaquin River (Quinn and Jacobs, 2006). Finally, policy developers and
decision makers are asking for, and processing, information synthesized from holistic systems-
based modeling approaches; such as those suggested by the US Environmental Protection
1 US National Science Foundation (http://www.nsf.gov/funding/pgm_summ.jsp?pims_id=504707)
2 Interagency Steering Committee for Multi-media Environmental Modeling (http://iemhub.org/topics/ISCMEM)
4
Agency (EPA) (see EPA, 2008b) or practiced in the Murray-Darling Basin of Australia (e.g.
ABARE-BRS, 2010).
The purpose of this paper is to present a comprehensive view of the landscape of integrated
environmental modeling, a vision of its future, and a roadmap for its navigation. The roadmap
consists of goals and activities (destinations and directions) articulated within a comprehensive
contextual framework (landscape). In presenting this comprehensive view of the landscape, we
will attempt to establish a common vantage point with our audience, provide adequate
background, and remain open to feedback from the community. The summaries provided herein
are not intended to be comprehensive, but rather provide an accurate snapshot of the current
state-of-the-art. Further, we do not propose an implementation plan (travel itinerary) – that will
be saved for a later effort.
The primary motivation and input for this paper is drawn from a series of workshops held during
the past several years (Table 1), each of which involved national and/or international
communities. These workshops were convened to share knowledge, experience, and future
visions related to IEM. While the full range of stakeholder perspectives and interests were
represented to the extent possible, the workshops were primarily attended by environmental
modelers and technology developers. Thus, our focus here is the future of IEM in the context of
complex environmental decision making.
The next three introductory sections lay the groundwork for describing the IEM landscape. First,
we present several terms closely related to IEM in an attempt to define them for purposes of this
paper. We then describe the IEM process and its role within the larger decision and policy
contexts. Finally, we organize IEM into four elements: applications, science, technology, and
community. These elements represent layers of the IEM landscape and are described in Sections
2 through 5. Section 6 presents the community vision and roadmap for the future of IEM.
5
Table 1: IEM Workshops
Workshop Title Sponsor Date Organizations Represented Outputs
Environmental Software Systems Compatibility and
Linkage Workshop
US NRC
DOE
March
2000 >40 attendees
A, B, C, D, E Report: NRC (2002)
Integrated Modeling for Integrated Environmental
Decision Making US EPA
January
2007 >100 attendees
B, F, G
Report: EPA (2007)
White Paper: EPA (2008b)
Collaborative Approaches to Integrated Modeling:
Better Integration for Better Decision making US EPA
December
2008 >50 attendees
B, H, I, J, K, L, M, N Report: EPA (2008a)
iEMSs 2010 Conference
Science session: Integrated Modeling Technologies
Workshop: The Future of Science and Technology
of Integrated Modeling
iEMSs July 2010
>75 attendees
International Conference (most
organizations listed below and others)
Workshop Materials
The International Summit on Integrated
Environmental Modeling
BGS
USGS
US EPA
December
2010
>50 attendees
International Conference (most
organizations listed below and others)
Report
AUS NRC: US Nuclear Regulatory Commission (http://www.nrc.gov/) BUS EPA: US Environmental Protection Agency (http://www.epa.gov) CDOE: Department of Energy (US) (http://energy.gov/) DUS ACoE: US Army Corps of Engineers (http://www.usace.army.mil) ENGO: Non-Governmental Organizations FEC: Environment Canada (http://www.ec.gc.ca/) GEU: European Union (http://europa.eu/) HISCMEM: Interagency Steering Committee for Multi-media Environmental Modeling (US Federal Agencies) (http://iemhub.org/topics/ISCMEM) ICEH UK: Center for Ecology and Hydrology, UK (http://www.ceh.ac.uk/) JiEMSs: International Environmental Modeling and Software Society (http://www.iemss.org/society/) KOGC: Open Geospatial Consortium (http://www.opengeospatial.org/) LCUAHSI: Consortium of Universities for the Advancement of Hydrologic Science, Inc. (http://www.cuahsi.org/) MOpenMI: Open Modeling Interface (Association) (http://www.openmi.org/) NUSDA: US Department of Agriculture (http://www.usda.gov)
CSDMS: Community Surface Dynamics Modeling System (http://csdms.colorado.edu/wiki/Main_Page)
NRC (Italy): National Research Council (Italy) (http://www.cnr.it/sitocnr/Englishversion/Englishversion.html)
NSF: National Science Foundation (http://www.nsf.gov/)
ONR: Office of Naval Research (US) (http://www.onr.navy.mil/)
NASA: National Aeronautics and Space Administration (US) (http://www.nasa.gov/)
USGS: US Geological Survey (http://www.usgs.gov)
NOAA: National Oceanic and Atmospheric Administration (US) (http://www.noaa.gov/)
BGS: British Geological Survey (http://www.bgs.ac.uk/)
6
1.1 Common Terms
Within the literature, several terms related to ―integrated environmental modeling‖ are used (see
Box 1). Multiple terms create a distracting confusion for practitioners and users alike. From our
view, the terms are more similar than dissimilar; and are collectively defined within the greater
context of environmental decision making and policy development. In any of the definitions, we
perceive the term integrated to convey a message of holistic or systems thinking (sensu Tress et
al., 2005); assessment as a message of policy relevance (Tol and Vellinga, 1998); while
modeling indicates the development and/or application of computer based models.
Box 1: Terms Related to Integrated Environmental Modeling
Conventional Modeling: a process of creating a simplified representation of reality to
understand it and potentially predict and control its future development. Models can come
in a variety of forms and implementations, including mental, verbal, graphical,
mathematical, logical, physical, etc. (Voinov, 2008).
Integrated Modeling: includes a set of interdependent science-based components (models,
data, and assessment methods) that together form the basis for constructing an appropriate
modeling system (EPA, 2008b; 2009).
Integrated Assessment: seeks to provide relevant information within a decision making
context that brings together a broader set of areas, methods, styles of study, or degrees of
certainty, than would typically characterize a study of the same issue within the bounds of a
single research discipline (Parson, 1995; Weyant et al., 1996; Jakeman and Letcher, 2003).
Integrated Assessment Modeling: an analytical approach that brings together knowledge
from a variety of disciplinary sources to describe the cause-effect relationships by studying
the relevant interactions and cross-linkages (Rotmans and van Asselt, 2001; Rosenberg and
Edmonds, 2005).
Integrated Environmental Decision Making: an approach for evaluating complex
environmental problems holistically by integrating resources and analyses to address the
problems as they occur in the real world; including input from appropriate stakeholders
(EPA, 2000).
Integrated Environmental Strategies (IES): enables researchers to quantify local co-
benefits that could be derived from implementing policy, technology, and infrastructure
measures at a national scale (e.g. reducing air pollutants and GHG emissions) (EPA, 2004).
Participatory Modeling: a generic term used for modeling strategies that rely upon
stakeholder involvement and participation in various forms. In various applications also
known as group model building, mediated modeling, companion modeling, shared vision
planning, participatory simulation, etc. (Voinov and Bousquet, 2010).
7
We also wish to be clear about terms that describe integration of knowledge from more than one
science discipline: multidisciplinarity, interdisciplinarity, and transdisciplinarity. Tress et al.,
(2005) reviewed the literature related to these concepts and suggested definitions in the context
of landscape ecology research; we adapt and apply them here to IEM. These terms differ by the
degree of interaction among disciplines and whether or not non-science oriented stakeholders are
involved. Multidisciplinarity involves participants who essentially execute a portion of a project
related to their individual discipline. Knowledge is exchanged, but not for the purpose of creating
new integrative knowledge or approaches. Interdisciplinarity involves the integration of
knowledge and methods from multiple disciplines in order to form holistic solutions to problems.
Integrating social, economic, and environmental sciences to create a new modeling system to
address complex problems is an example of interdisciplinarity. Transdisciplinarity involves the
integration of knowledge and methods from multiple disciplines (i.e., interdisciplinarity) and the
participation of non-science oriented stakeholders. Transdiciplinarity is focused on societal
issues that require a wide range of science knowledge that is organized and applied via a fully
participatory process. Transdisciplinarity is the highest form of knowledge integration and the
ultimate goal of IEM efforts.
Finally, the term ―stakeholder‖ is generally used to refer to participants in the decision making
process. We group these participants according to their primary role in the overall decision
process. Decision stakeholders are primarily concerned with the problem, its impacts,
representation of interests/concerns, management scenario development, and decisions related to
solving the problem. These stakeholders include those responsible for policies and decisions,
those affected by the problem (including benefits and drawbacks of solutions), including the
public and industry, advocacy groups that support the environment and industry, and assessors
who manage the overall process. Science stakeholders are primarily concerned with the
organization and application of science-based knowledge in the form of data, models, and
methods for the purpose of informing decisions. These stakeholders represent multiple
disciplines and include scientists, modelers, technology developers, and assessors. Assessors
play a unique role as their principal concern is split equally between the decision and the science.
In addition to their technical responsibility, assessors often act as a liaison between the decision
and science stakeholders, facilitating conversations and information exchanges.
8
These two categories of stakeholders are often called experts (Krueger et al., 2012). There are
also stakeholders at large who are the citizens, communities and individuals who may be
impacted directly or indirectly by the decisions made, yet may not be part of the decision making
process. In some cases the boarder line between them and experts can be quite vague because
these stakeholders can also be experts in various domains contributing local knowledge and
experience to the process, which becomes an important factor in participatory modeling (Voinov,
Gaddis, 2008).
In addition to stakeholders involved in specific problems and decisions, we recognize
stakeholders in the broader context of the discipline of IEM. These IEM stakeholders include
those listed above along with the addition of research funding organizations, researchers
representing integration science and individual contributing disciplines, and students. The roles
and interests of stakeholders in this context include:
Advancement of the state of knowledge and practice of complex environmental decision
making and integrated environmental science and modeling.
Prioritization and funding of IEM research and development.
Education of the present and future generations of stakeholders.
1.2 The IEM Process and Decision/Policy Context
Transparency is an important requirement for conducting the decision process. The foundation of
transparency is the articulation of the steps and assumptions of the process and the manner in
which the process is executed. Figure 1 illustrates the overall integrated environmental modeling
and decision process. Similar descriptions are presented by Zagonel, (2002); Jakeman et al.,
(2006); and Liu et al., (2008). The process contains two principal loops: the decision/policy loop
and the modeling/monitoring loop. The process begins with the formulation of a problem
statement that defines the causes of concern, policy or decision context, boundaries and
objectives, management scenarios and options, solution criteria (including tolerance for
uncertainty), and resource constraints. System conceptualization follows and is a high level view
of the social-economic-environmental system within which the problem occurs. It represents the
9
common view of the system shared by all stakeholders and ties the decision loop directly to the
more quantitative modeling/monitoring loop. It forms the basis for developing a detailed
modeling and/or monitoring based solution. This paper focuses on integrated modeling, but
monitoring is also important as both a strategy for responding to problems and as a critical
source of data for modeling. Indeed, data can be treated as models (Voinov, Cerco, 2010) that are
identified and integrated with other models to form a modeling system that is then executed to
simulate cause-and-effect relationships among stressors and impacts. The modeling results are
synthesized to forms useful to decision makers, whereupon the elements of the problem
statement and management scenarios may be modified and the process repeated until satisfactory
policies/decisions are rendered.
Feedback and iteration are essential aspects of the modeling process in general (Jakeman, et al.,
2006), and also the stakeholder-driven IEM and decision process. To execute each step, two
questions must be answered: ―What is needed?‖ and ―What is possible?‖ What is needed is
informed by the previous step(s), while what is possible is informed by knowledge related to
subsequent step(s). The need for iteration and feedback enters because, practically, specifying
what is needed is dependent upon what is possible -- and what is possible is uncertain until it is
attempted. The degree of feedback and iteration is generally a function of the complexity and/or
uniqueness of the problem to be solved. With each repetition of a particular application (i.e.,
combination of problem statement and solution strategy) the process becomes increasingly
prescriptive and sequential.
1.3 Organization of IEM Elements for Discussion
This paper discusses four primary elements of IEM: applications, science, technology, and
community. Applications represent the structured efforts required to address a complex
environmental problem within a decision/policy context (i.e., implementation of the process
shown in Figure 1). The science of IEM focuses on the need to integrate knowledge (data,
models, methods) from multiple disciplines and create a holistic representation and simulation
capability of the relevant social-economic-environmental system. Technology represents the
primary means by which the science of IEM is expressed and applied; including both software
and hardware. Community reflects the social context within which the discipline of IEM evolves.
10
The community plays a vital role in establishing the science domain, organizing the domain
knowledge, identifying and promoting best practices, facilitating professional interactions, and
peer review. Together these elements represent the principal views of the IEM landscape we
wish to present.
Figure 1: The Integrated Environmental Modeling and Decision Process
2. IEM APPLICATIONS
During the past two decades there has been a steady evolution toward IEM in the range and
complexity of environmental issues and problems, related decisions and policies, and the
modeling performed to inform the decisions. While decision makers will continue to address
traditional problem sets involving environmental quality standards and compliance, management
challenges are now framed in ecological, social, and economic terms (MEA, 2005). IEM
applications are the stakeholder community‘s methods for selecting, organizing, integrating, and
processing the combination of environmental, social, and economic information needed to
inform decisions and policies related to the environment. Box 2 lists the characteristics that
11
distinguish IEM applications from more traditional environmental modeling applications.
Combinations of these characteristics manifest in IEM applications to varying degrees. The
sections that follow contain examples to illustrate the range of problems and decisions that
require IEM, and describe future needs and directions as discussed during the workshops (cf.
Table 1).
Box 2. Characteristics of IEM Applications
Problems involve the inter-dependent relationships among multiple stressors, a well-
defined environmental system consisting of multiple compartments (e.g., air, water, soil,
biota, people), and multiple, often competing, endpoints of concern
Decision and policy objectives target resolution of complex environmental problems/issues
in the context of the social and economic landscapes in which they occur.
Solutions are designed to communicate understanding and modeling estimates (predictions)
of environmental system responses to stress and the relative merits of alternative
management strategies.
Decisions and policies involve a diverse set of stakeholders with a high degree of
interaction and information sharing among science and non-science oriented stakeholders.
Solutions require holistic approaches that involve integrating multidisciplinary data,
models, and methods.
2.1 Examples of IEM Applications
The characteristics identified in Box 2 are a suitable way to categorize IEM applications in order
to present an organized view of the IEM landscape. The literature contains a growing number of
studies involving the application of integrated environmental modeling concepts and approaches;
examples of these applications within each category are presented in Table 2.
2.1.1 Future Needs for IEM Applications
Dominant themes throughout workshop discussions (see Table 1) concerning future IEM
applications included adaptive management, stakeholder involvement, best practices, education,
and reusability. Adaptive management refers to the realization that with respect to the
complexities of the human-environmental system we are ―learning as we go‖. We cannot predict
the future with high degrees of certainty. Thus, our decision and policy making process must
accommodate and adapt to new knowledge and data. Given this reality we need to reflect this
adaptability in the design and execution of IEM applications.
12
While the importance and value of involving the full stakeholder community in the decision
process is recognized there remains a significant need for methods and protocols that ensure
appropriate incorporation and integration of perspectives, knowledge, and values.
We also need to document and promote best practices. There is a wide range of approaches that
do not follow any formal protocols, standards, or practices. This makes it very difficult to
understand, review, and reuse applications. There is a need to move toward conceptual
standardization of IEM applications. That is, defining and documenting IEM applications
according to a common process and set of practices. There is a need to establish criteria for
independent peer review and QA/QC, as well as methods for calibration and validation most
appropriate for IEM applications. Best practices should also be organized and made accessible
for purposes of training, education, and interdisciplinary communications.
Reusability of IEM applications implies two things: transferability and interoperability.
Transferability means that IEM applications can be implemented at geographic locations and
time periods other than where and when they originate. Interoperability means that new
applications can be constructed from components used in existing applications. Making this a
reality will require a combination of application metadata (i.e., information that defines the
applications) and technology standards (e.g., software standards for storing and accessing the
metadata and applications). This combination of metadata and standards must be developed and
implemented in direct collaboration with IEM technologists.
3. IEM SCIENCE
The science of integrated environmental modeling provides the knowledge base and integrative
strategies that underpin problem-solving applications. The scientific goals of IEM are to provide
data, models, and methods to explore, explain, and forecast system response to perturbations in
natural or managed environmental systems. To conduct IEM, scientists do not directly pursue
new knowledge within individual disciplines, but rather concern themselves with issues that arise
when domain-specific knowledge bases must be integrated. Box 3 lists the unique characteristics
13
of IEM science. In the following sections we describe these characteristics and present related
future needs and directions.
14
Table 2. Categrories of IEM Applications.
Categories Examples and References
Inter-dependent relationships
among multiple stressors, multiple
environmental compartments, and
multiple endpoints
San Joaquin River Deep Water Ship Channel assessment near Stockton, CA, USA (Jassby and Cloern,
2000; Lehman et al., 2001; Quinn and Jacobs, 2006)
Organic and metal loadings to the Venice Lagoon, Italy (Sommerfreund et al., 2010)
Life-cycle analyses of chemical-based stressors (organics and metals) at a global scale (Sleeswijk and
Heijungs, 2010)
Multi-model comparison of water quality modeling, Pinios River, Greece (Makropoulos et al., 2010)
Groundwater-surface water flooding; linked modelling approach UK basins (Hughes et al, 2010)
See also: Mackay (2001) and Fenner et al., (2005)
Ecological applications focused on
decision/policy objectives with
alternative management strategies
Farber et al. (2006) explore the role of integrated ecological–economic models in quantifying the trade-
offs among ecosystem services in complex, dynamic systems.
Bolte et al. (2006) and Guzy et al. (2008) address important issues for understanding policy effects in
social-ecological systems by agent-based modeling of land use and land cover.
Hulse et al. (2008) alternative future actions in western Oregon‘s Willamette River Basin.
Boumans et al. (2002) describe a unified metamodel of the biosphere developed to simulate the earth
system as comprised of the atmosphere, lithosphere, hydrosphere, biosphere, and anthroposphere.
See also: Maxwell and Costanza (1995); Daniels (1999); Noth et al., (2000); Costanza et al. (2002);
Sengupta and Bennet (2003)
Applications involving a diverse
set of stakeholders Stahl et al. (2011) had science stakeholders work with decision stakeholders to apply a multi-criteria
integrated resource assessment (MIRA) method (Stahl et al., 2002)
Solutions requiring holistic
systems-based approaches that
involve integration of
multidisciplinary data, models, and
methods
Akbar et al. (this issue) establish a monitoring/modeling system capable of informing real-time decisions
related to preparing for and responding to hurricanes impacting the Gulf of Mexico region along the
southeastern United States.
Babendreier and Castleton (2005) describe a national scale assessment of the impacts of chemical
releases from land based waste management units.
15
3.1 System Conceptualization
A system conceptualization captures the essence of the real-world problem within the context of
the processes, cycles, and flows that characterize the relevant social-economic-environmental
system. It establishes a view of the system shared by all stakeholders, thus forming a framework
for understanding and communication. It provides the basis for developing the more detailed
modeling (and perhaps monitoring) methodology and the strategy for synthesizing modeling
results.
The level of complexity of the conceptualization is an important consideration (Robinson, 2006).
It must include a combination of system processes, cycles, and flows that represent decision
control points (i.e., those system elements that can be managed by decision makers) and the state
of the system (e.g., environmental quality, economic impact). The formulation of a conceptual
Box 3: Characteristics of IEM Science
System conceptualization ensures a holistic and transdisciplinary representation of
relationships among the elements of the system; processes, stressors (forcing functions),
impacts, and management options (controls) that are important for mitigation, remediation,
restoration, sustainability, system resilience, etc.
IEM methodology represents the science of integration as applied to environmental modeling:
integration of science knowledge in the form of multiple disciplinary science components
(knowledge, data, models, methods) to form a system simulation capability that is both
coherent and transparent, and transcends disciplinary priorities and perceptions.
Data requirements are diverse and represent multiple science domains. They are produced by
similarly diverse sources whose delivery formats are not typically designed for cross-
disciplinary integration. Data may represent widely varying dimensions related to modeling
(e.g., spatial and temporal scales, population demographics, economic markets).
Model evaluation methods address performance of both individual components and the
integrated system. Performance is defined as a combination of transparency, refutability, and
uncertainty quantification.
Synthesis of modeling results is designed to serve the needs of each stakeholder for
information that contributes to their knowledge of how the system works and defines the
uncertainty related to modeling results.
16
model is a social as well as a scientific process. A primary challenge is translating and merging
the mental models of stakeholders with scientific models for explaining system relationships and
dynamics and making the merged modeling layer transparent, adaptable, and responsive to
queries (Lancaster, 2007; Otto-Banaszak et al., 2011; Voinov and Bousquet, 2010; Voinov et al.,
2008; Zagonel, 2002).
There are several methods and formalisms available for developing and documenting a
conceptual model. Luna-Reyes, 2003 presents examples of mapping tools used to graphically
represent dynamic systems. Fischenich, 2008 describes the purpose, constructs, and relative
advantages and disadvantages of different conceptual models. Other formalisms, based upon
specific quantitative modeling approaches, include systems dynamics (Boumans et al., 2002;
Fenner et al., 2005; Muetzelfeldt and Masheder, 2003), fuzzy cognitive mapping (Özesmi and
Özesmi, 2004), and Bayesian inference (Reckhow, 2003).
3.1.1 Future needs for system conceptualization
Despite the availability of methods and evidence that engaging stakeholders in the system
conceptualization process is a growing priority (Voinov and Bousquet, 2010), the methods
currently employed in practice rarely conform to any specific social or science-based guidelines
or protocols. Looking to the future, there is a need to develop a more structured process
associated with the development and documentation of the conceptual model. In particular, the
following needs should be addressed:
Continued research and documentation of best practices with respect to the social
process of eliciting and merging the array of stakeholder mental models.
Explicit mapping of the elements of a conceptual model to elements of the problem
statement and the quantitative modeling construct.
Documentation of the conceptual model in machine readable metadata to facilitate
stakeholder understanding and communication both within and across applications.
17
3.2 IEM Methodologies
An IEM methodology is a combination of a modeling system and an implementation strategy.
The modeling system represents an integration of data and knowledge from across relevant
science domains and is the quantitative and computational form of the conceptual model. The
mathematical form of the modeling system may be empirical, statistical, process based, or a
combination of the above (Linker et al., 1999; Hart et al., 2009; Schwarz et al., 2006). The
implementation strategy specifies how the modeling system will be deployed in the context of an
IEM application. Deployment may include such strategies as applying the modeling system to
representative locations across a regional or national landscape (Babendreier and Castleton,
2005), executing the modeling system within a monte carlo simulation protocol to address
uncertainty (Johnston et al., 2011), or applying the modeling system repeatedly within an
adaptive management strategy (Akbar et al., this issue). Multiple implementation strategies may
be applied with the same modeling system.
Workshop discussions of IEM methodologies focused on issues related to the construct of IEM
modeling systems, namely; integral vs integrated modeling systems, complexity, and coherence.
Figure 2 illustrates the conceptual difference between integral and integrated constructs. An
integrated modeling system (Figure 2A) uses computer software to link components (e.g., legacy
models) that represent processes or subsystems within the larger system of interest. On the other
hand, integral models represent a singular formalism built from the merger of knowledge of
subsystems or system views held by various disciplinary communities (e.g., legal, economic,
political, social, environmental, Figure 2B). Each of these approaches result in a holistic
modeling based representation of the full system.
The two approaches can also be described as model integration vs. knowledge integration and
the choice between the two has both theoretical and operational implications for IEM (Voinov
and Shugart, 2011). Although examples of each exist in the literature, integrated modeling
systems dominate, because of the IEM community‘s desire to leverage the inventory of legacy
models. It is relatively easier to link existing models than to conduct an intensive participatory
process to merge knowledge bases and build an integral model. However, there are several
18
science integration issues that must be addressed; in the long run, it is not clear how the
resolution to these issues will influence the choice of IEM model system construction.
Figure 2. Two ways of integration. In an integrated model (A) models of components and subsystems are
linked to build a representation of the whole system. In integral models (B) whole system models, based on
disciplinary, or stakeholder, priorities and paradigms, are merged through a participatory process to form a
new and singular formalism representing the system (Kryazhimskiy, personal communication).
Complexity reflects the level of detail at which the problem relevant system is modeled and
includes breadth (how many processes and interactions to include) and depth (dimensional scale
and resolution, e.g., spatial/temporal). We know that complexity should be a direct function of
the problem statement, decision objectives, and the conceptual model; we also know that it is
impossible (and unnecessary) to include all known science related to the social, economic, and
environmental disciplines (Liu et al., 2008). The challenge is to determine which of the detailed
processes are important in simulating the system behavior at an appropriate scale of application
(Sidle, 2006). Oreskes (2003) describes the complexity paradox. As understanding increases,
then the natural reaction of any scientist is to add complexity to their models. In other words, as
data is collected, and understanding correspondingly improves, then more and different
processes could be added to any model. However, as more processes (and subsequent
parameters) are added, the overall uncertainty in the model increases (Hanna, 1988). Further, it is
difficult to know if the tests of increasingly complex models are meaningful (Oreskes, 2003).
19
So a more complex model better captures the nuances of the natural system, but it is more
difficult to determine whether the model successfully reproduces the natural system. More
complex models that emerge from integration are also likely to be more difficult to calibrate and
test (Voinov, Cerco, 2010). This has important implications for complex systems of linked
models.
Another central theme related to IEM methodologies is coherence. Coherence exists when
components are scientifically consistent across the system with respect to complexity, data
requirements, and uncertainty (EPA, 2008b). When designing a modeling system, the following
elements (among others) of coherence must be addressed:
Conservation of energy, momentum, and mass across the modeling system.
Geo-spatial and temporal boundaries of models
Representation and processing of time and space
Differing scales and resolution of dimensional variables between modeling components
(e.g., land use categories, ecological species, human demographics, etc.)
Coherence for integral modeling constructs is addressed at the design stage of the system and
becomes an intrinsic part of the modeling system. When constructing an integrated modeling
system, the issues of coherence must be addressed across the individual components with
appropriate adjustments made at the interfaces.
3.2.1 Future needs for IEM Methodologies
In moving forward, there are several important questions that should be asked concerning our
application of integral and integrated modeling systems. How are the two approaches
complementary? What can we learn from our experience associated with combining various
forms of knowledge (personal experience, group preferences, expert opinion, science-based
knowledge, etc.) in a participatory modeling framework to improve integration between
components? Reciprocally, how can component integration help us better integrate knowledge of
stakeholders? What guidance can we formulate to help make the decision at the problem level
more effective and transparent?
20
With respect to complexity, there is a need to develop better understanding of the relationship
between the elements of a problem statement, conceptual model, and quantitative model. The
degree of complexity should match the need. This is easy to state but difficult to implement,
especially when models are assembled from components previously developed to serve very
different needs. The community also needs guidelines and protocols for constructing
appropriately complex modeling systems for IEM. Similarly, guidelines and tools for achieving
coherence are needed, particularly in cases where information is shared between components of
different science disciplines (e.g., economic and environmental) and among legacy models that
are integrated to form larger modeling systems.
3.3 Data for IEM
Three important uses of data within the context of modeling are: data used in model development
(parameter estimation); data as input to models; and data for evaluating models. Table 3 lists the
range of environmental data relevant to IEM and Figure 4 illustrates the geo-spatial nature of
much of the data. The data draw from several science disciplines (multidisciplinarity) and span
most spatial and temporal scales. Data preparation for IEM is a resource intensive task; the steps
of the process are described in Box 4.
Table 3. Categories of IEM Data
Environmental Features, Characteristics, and
Observations
Atmosphere
Meteorology
Watersheds
Surface Soils
Land use / Land cover
Digital Elevations
Vadose zone
Groundwater systems
Surface water systems
Ecosystems
Sources
Waste management units, waste properties
Agricultural settings and practices
Industrial settings and practices
Natural events
Geographic/Topographic Boundaries
Administrative
Political
Ecoregions
Watersheds/Waterbodies
Aquifers
General Stressors
Chemical properties
Land use change scenarios
Global warming scenarios (future precipitation
and temperature)
Natural events
Exposure Pathways and Health Effects Factors Population Distributions, Demographics, and Behaviors
21
Human
Ecological
Human
Ecological (aquatic, terrestrial)
Figure 4. Example geo-spatial data layers for IEM. Image courtesy of RTI, International.
Box 4: Steps for Data Preparation
1) Discovering data sources: that may be located on the internet, in a journal article or
government report, or in a private data file,
2) Devising methods for accessing the data which requires understanding the semantics
and storage formats of each source and devising extraction methods,
3) Processing the data which includes performing mathematical/statistical transformations
and QA/QC of the data to ready it for the application,
4) Integrating the data which establishes system coherence and connectivity, and
5) Building input files which requires placing the data into individual model input files.
Executing these steps requires reconciliation of varying semantics across the numerous data
sources (Box 4, steps 2 and 3) and establishing an operational ontology that specifies, for
models, the relationship among physical entities of the system (Box 4, steps 4 and 5). The
ontological relationships include the geo-spatial distribution of environmental features (Figure 4)
22
and the connectivity of these features that define movements and flows through the system (e.g.,
individual catchments connectivity to individual stream segments to specify water runoff routes).
Currently, the execution of this process for individual applications can be characterized as a
semi-manual task of cutting and pasting file fragments to form a single coherent dataset. Needed
data are dispersed among a myriad of sources, each representing data with its own unique
combination of vocabulary, storage formats, and accompanying descriptive information
(metadata). Significant effort and science-based judgment are required to locate, access, process,
and prepare data for IEM applications. Complicating the issue is the fact that new sources of data
are coming online seemingly every day, e.g., real-time remote and in situ sensing, social media.
3.3.1 Future needs for IEM data
The resource intensive task of gathering and organizing data for IEM applications represents a
barrier to the wider acceptance and use of IEM to support decisions and policies. The challenge
for IEM is discovering and accessing the numerous sources of data and building an integrated
dataset that provides a single coherent representation of the system to be shared by all modeling
components. Because major portions of the IEM community (e.g., nationally) access the same
primary sources of data and process it to the same end (i.e., model inputs) we can gain
tremendous efficiencies if we view this challenge as a community infrastructure need. In short,
there is a need to develop community accessible tools and methods for: (a) automated discovery,
access, and retrieval of data from individual sources; and (b) ensuring quality and facilitating
integration of data from multiple sources and populating model input files. Proper documentation
(metadata) and standards for storage, retrieval and update are of paramount importance for this
end.
The solution to this science-based need is dependent on technology and thus, Section 4 describes
ongoing and emerging efforts to improve access to environmental data and suggest future needs
and directions as discussed at the IEM workshops.
23
3.4 Model Evaluation
Model evaluation combines quantitative and qualitative information about a modeling system‘s
appropriateness for the problem and ability to characterize the uncertainty of model predictions.
This characterization must be organized for use by all stakeholders. Key attributes of model
evaluation are transparency, refutability, and uncertainty quantification. Together, they establish
the scientific veracity and stakeholder confidence/acceptance of an application and the
information it produces. Transparency requires that all aspects of the application design and
execution be accessible to facilitate understanding and reproducibility. Refutability requires a
hypothesis-testing framework in which data are used in specific ways to test the model‘s ability
to simulate the system of interest. This may include meaningful model validation efforts, in
which the model is tested with data not included during its development. This may require the
involvement of decision stakeholders because the ultimate test of a model is always its utility and
usability by end users. Refutability is difficult for any model of an environmental system and
evaluating an IEM system is even more challenging.
Challenges related to prediction and uncertainty in integrated modeling were experienced in
recent climate change modeling (e.g. IPCC, 2007). Several textbooks on these subjects have
been produced in recent years, and an active scientific community continues to explore new
methods. A few references include: Menke (1989) and Aster et al. (2005) from geophysics;
hydrology (Beven, 2009; Clark et al., 2011; Hill and Teideman, 2007); and Saltelli et al. (2008)
from econometrics. However, application of these ideas to more complicated IEMs is only
beginning to emerge, (Beven, 2007; Refsgaard et al, 2007; Ascough et al, 2008; McIntosh et al.,
2011; Benett et al., 2012) and there is still much to learn in developing methods and case studies.
We expect that the growing use of IEMs will test the limits of existing methods and lead to
additional innovations. In Table 4 we describe basic model evaluation methods that are useful in
a wide range of circumstances and that have characteristics advantageous to IEMs.
24
Table 4. Summary of methods used for model evaluation.
Technique Description
Error evaluation and
propagation
This starts with data being used to construct the component and IEM models, proceeding to the evaluation of predictions. This is critical to IEMs
because data availability and errors in one part of the system invariably affect uncertainties in other parts.
Such holistic considerations of error and uncertainty is critical to integrated model development.
Error-based weighting This is critical to integrated use of data in IEMs and is complicated by the need to synchronize the weights among various disciplines since
variations in the importance of the processes and data may occur. With error-based weighting, weighted discrepancies between observed and
simulated values become comparable, partly because they become dimensionless and partly because error-based weighting puts the discrepancies in
a consistent statistical framework
Sensitivity analysis This can be conducted by combining computationally efficient local linear methods and computationally expensive global methods. Linear methods
are attractive for IEMs because they require on the order of 10s of model runs to obtain useful results
Alternative models These are important in IEM model development for assessing the effects of conceptual model uncertainty. Alternative conceptualizations
considered often affect more than one aspect of the system, and integrated models allow the consequences to be represented realistically throughout
all relevant aspects of the system simulation (e.g., the hydrologic cycle, Clark et al., 2011).
Optimized calibration
methods
These are used to improve objectivity and reproducibility. The optimization process can identify the information content of a given set of
observations and, along with sensitivity analysis methods, can identify important new data. For IEMs, this can be particularly challenging because
of the number of linked/coupled component models involved.
Uncertainty
Quantification
Should be done with the best possible methods, considering the computational demands of the model and requirements of end users of model
results. IEM models require both knowledge and uncertainty from different system components to be conveyed to uncertainty quantification.
Model Tests Should be undertaken against alternative data sets. Often called model validation ―test‖ is used here because the model may be deficient, in which
case the ―test‖ may not validate the model. In IEM models it is important to test component models and integration requires additional tests
throughout the IEM.
Post-Audits Comparing model predictions to what actually happens are called post-audits, and may involve response to changes that take moments, days, years,
decades, centuries, or longer. Post-audits require monitoring. Either errors in economic and/or social evaluations or differences in expected and
actual changes, and in model simulated processes can be highlighted. IEMs provide important opportunities for post-audits because simulated
results can affect resources important to large ecological systems and to many people.
Calibration and testing in
data scarce conditions
Because of complexity, the likelihood of having complete datasets for extensive model analysis is low. Therefore, confidence building and
uncertainty analysis has to be performed in data-scarce conditions. The model and theories used in modeling can suggest specific monitoring foci to
collect the most important data sets to help decrease uncertainty and facilitate adaptive management strategies. However, this means longer delivery
times. Involving stakeholders and other users in model analysis is also essential to judge modeling success.
25
3.4.1 Future needs for model evaluation
The expanding applications of IEM require critical improvements to our model evaluation
methods. The transdisciplinary nature of this modeling requires methods and tools to harmonize
and integrate model evaluation information from different disciplines into a single coherent
representation. Further, there is a need to establish and consistently apply best practices,
including identification of sources of uncertainties and tracking their propagation through a
modeling system.
3.5 Synthesis of modeling results
Like system conceptualization, the synthesis of modeling results represents a key interface
between science and decision stakeholders (see Figure 1). The objective is to interpret,
consolidate, and present the results of complex integrated modeling to stakeholders and decision
makers. Synthesis must produce information that is not only of high scientific quality but, also
useful to the decision makers. McNie (2006) discusses the challenges of ―reconciling the supply
and demand‖ of scientific information between scientists and decision makers, defining ―useful‖
as a combination of salient, credible, and legitimate information. Salience implies contextual
relevance, credibility refers to scientific veracity, and legitimacy refers to a lack of bias.
Decision support systems assimilate the synthesized modeling results and prepare them, along
with other relevant information, for presentation to the decision stakeholders (McIntosh et al.,
2011). While scientists routinely synthesize results for other scientists via scientific journal
articles, the synthesis of scientific information for decision making is not well understood and
executed (NRC, 2005). With IEM, the challenge again is to serve diverse users and stakeholders
from various disciplinary backgrounds and with different vocabularies, priorities and vested
interests. This may require multiple synthesization streams tailored to particular audiences and
designed hierarchically to move seamlessly from very general displays of overall results to
highly detailed, component-based visualizations (Ellarby and Kite, 2006; Liu et al., 2008).
3.5.1 Future needs for synthesization of modeling results
Looking to the future it will be necessary for stakeholders (representing both science and
decision makers) to work more closely together to develop means for synthesizing modeling
26
results for their effective use in decision making. Studies, such as reported in this issue
(Arciniegas et al., 2012), must be encouraged and expanded. The participatory modeling
approach offers another promising interface with stakeholders, however specific applications that
can make IEMs available for stakeholder use and acceptance are still largely missing (Voinov et
al., 2008).
4. IEM TECHNOLOGY
IEM‘s technology component covers the computational and informatics tools, approaches, and
concepts required for achieving overall IEM objectives. The objectives of IEM are ambitious and
require sophisticated, state-of-the-art technologies to understand integrated environmental
systems (i.e., databases; models; assessment methods; and visualization tools) that span multiple
scientific disciplines, but need to be summarized concisely to support policy and decision
making goals. Box 5 lists the characteristics that distinguish IEM technologies from those
employed in conventional modeling.
Box 5: Characteristics of IEM Technology
Facilitates linkage of and information transfer among components of integrated modeling
systems.
Facilitates data collection from disparate sources, data processing and integration, and
preparation for model consumption.
Manages the execution of integrated modeling systems across software and hardware
platforms.
Provides user interfaces to facilitate information exchange (e.g., scenario development, data
input, data analysis/visualization, and stakeholder involvement)
The technological characteristics associated with IEM are broad and require a diverse set of
informatics and computing approaches. For practical reasons, we limit the scope of this
discussion to two key topics: 1) modeling infrastructures (aka frameworks) and interoperability,
and 2) data access and processing. We discuss each of these in the following sections.
4.1 Modeling Infrastructures and Interoperability for IEM
27
Modeling infrastructure is software designed to facilitate the information and data flow among
components of a modeling system. Components of conventional environmental modeling
systems include user interfaces, data analysis and visualization tools (including GIS), individual
science models, calibration and optimization tools, and perhaps distributed computing tools. In
addition to these, and unique to IEM, is infrastructure needed to construct modeling systems
comprised of multiple science-based models not originally designed to "talk" to each other, i.e.,
they are not designed for interoperability. Interoperability is the ability of different information
technology components, systems, and software applications to communicate and exchange data
accurately, effectively, and consistently, and to use the information that has been exchanged
(IEEE, 1989; Heubusch, 2006). Linking models not designed for interoperation is often
surprisingly difficult, and a brute-force approach to the problem requires a significant investment
of time and effort. The following paragraphs discuss some of the technological difficulties that
arise when tackling model interoperability.
Programming Language Incompatibilities: Models are written in different programming
languages and conversion to a single language is impractical, time-consuming and error-prone.
Languages do not offer the same data types and features, so special tools are required to make
function calls across the "language barrier". If models are not converted to a common language,
then a language interoperability tool must be provided to allow components to share data.
Inadequate Model Documentation and Metadata If models are to be coupled so they can
exchange information during a simulation, it is beneficial for model documentation to be both
human-readable and machine-readable. A detailed, standardized description of each component
should be provided by the scientists and implemented, perhaps as a standard XML file that is
machine readable, but also easily rendered in various human-readable formats (HTML, PDF,
etc.). Documentation and metadata move models from black boxes to being reusable components
within a larger software system. This documentation will ensure that a component is applied
properly.
Spatial and Temporal Incompatibilities Models may have different dimensionality (1D, 2D, or
3D) and generally consist of one or more arrays (1D, 2D, or 3D) that are being advanced in time
28
according to differential equations or other rules. The models generally have a geographic
context and are often used in conjunction with GIS tools. They may use different types of grids
(e.g., rectangles, triangles, and Voronoi cells). Each model has its own time loop or clock, but
when linked they must use a single clock. The numerical scheme may be either explicit or
implicit, time steps fixed or adaptive. Creating a model infrastructure that can operate over a
variety of spatial and temporal scales is challenging, and adding logic that informs modelers
when model components are or are not compatible due to spatial and temporal mismatches is
even more challenging.
Non-standardized Input and Output File Formats One of the most basic forms of incompatibility
between models is inconsistent input and output files. In the atmospheric modeling community,
NetCDF is the de facto file type used, and there are standardized conventions (e.g., CF
conventions) for semantics used within NetCDF files. No such standards exist in environmental
modeling. While there have been attempts to standardize file formats (e.g., the ArcHydro data
model), it has proven difficult to create a single description of the natural environment that can
form the foundation for creating standardized input and output files.
Reliability and Sustainability of Models A key question in reliability and sustainability of models
is component testing--specifically, what type of unit testing should be performed and how often?
As a link in a computational chain, each component must be reliable. Special measures may be
needed to ensure numerical stability, accuracy, and consistency across the full range of intended
use. All modern software builds on existing software, but such dependence must be carefully
scrutinized for future availability. Besides these challenges, there are others related to managing
the software contributions to the system. When software is contributed by a group of developers,
there must be quality assurance checks in place to verify reliability of the software. There must
also be higher level structure to the software contributions so that they do not, for example,
reproduce functionality already within the system.
Classifying Models Wrapping model components with a single, common interface like OpenMI
(Moore and Tindall, 2005) allows them to be linked, but does not address which components are
interchangeable. This requires defining component categories or types and requiring that all
29
components of the same type return some common set of output variables. As long as they
provide this common set, they are permitted to use different input variables and additional output
variables. One advantage of the ―common set of outputs‖ approach is that users no longer need
to connect input and output variables one by one, manually. Instead, they simply connect
components and that connection accesses all of a component‘s output variables.
User-Friendliness Lastly, and perhaps most importantly, the graphical user interface design for
the system must be intuitive for end users, ranging from experienced modelers to the wider
community of stakeholders. Components are more likely to be reused if they provide a user-
friendly GUI for preparing the various types of input data that they require.
Several modeling frameworks that address these issues in a holistic and integrated manner have
been developed during the past decade. Castronova and Goodall (2010) and Lloyd et al. (2011)
summarize a variety of these IEM frameworks that use both procedural and object-oriented
programming languages. Examples include ESMF (Hill et al., 2004) and CSDMS (Peckham,
2010); OpenMI (Moore and Tindall, 2005); SEAMLESS (Ittersum et al., 2008), and OMS
(Ahuja et al., 2005; David et al., 2002). Each of these frameworks employ a set of standards to
address the interoperability issues described above. Unfortunately, the standards are ―local‖, that
is, they apply within the particular infrastructure or framework only. The resulting reality is that
the issues of interoperability described above must be resolved for every combination of science
component and modeling infrastructure. And, more importantly, the sharing of science
components among frameworks, a priority for IEM applications and science, remains difficult
and resource limited. For these reasons, it is common to see IEM modeling systems and
frameworks that contain only a limited number of components; typically that set familiar to the
developers and needed to solve a particular problem.
This issue has been recognized and progress toward increased interoperability is occurring.
Groups have ―discovered by doing‖ that the issues described above are common across
integrated modeling systems and that the required functionality can be abstracted and
standardized. In particular, groups have found that it is necessary to provide separate component
interface functions for initializing a model, stepping through time and updating state variables,
30
finalizing a model, retrieving data from another component, and assigning data to another
component. The simple but critical reason for this is that every model was originally designed to
have its own clock or time loop, but in a coupled configuration only one component can be in
charge of the clock. Components that provide the interface functions listed above are able to
provide their caller with fine-grained control of their functionality, which is key to usability in a
plug-and-play framework. This interface in no way prevents them from being used as stand-
alone models. As an example, the Common Component Architecture (CCA) is a set of
component and framework standards developed within high-performance, scientific computing.
CCA-compliant components can be reused in other CCA-compliant frameworks (e.g. Ccaffeine,
XCAT and SciRUN).
Other examples of groups creating solutions with increased emphasis on interoperability issues
include CUASHI and OpenMI. CUAHSI (Maidment et al, 2009) has designed and implemented
standards for exchanging hydrologic data over the web including the WaterOneFlow web
services and the Water Markup Language (WaterML). These were used to construct a
Hydrologic Information System with a service-oriented architecture and able to integrate
hydrologic observational data from international, federal, state, local, and academic data
providers. The design principle is to keep databases separate and autonomous while providing
standards that allow software programs to query and extract data from them using standardized
approaches. And, the OpenMI Association has designed standards for linking water resource
models at run time that require input or output from other models. When using the OpenMI
paradigm, models are treated as components of a larger simulation system so they must adopt
standard software interfaces in order to be integrated. The OpenMI standard has received
international attention and movement of existing frameworks to accommodate the standard is
occurring.
4.1.1 Future needs for model infrastructure and interoperability
From workshop discussions the highest priority for IEM infrastructure technology is to further
enable interoperability. Communication technologies are central for achieving this goal. They
emphasize protocols for coupling system components using well-documented data models that
can be expressed as object-oriented data structures, database schemas, and mark-up languages,
31
while providing a consistent view of data across systems. Service-orientation is critical for the
system to allow modularity of models that remain interoperable within a larger system. This
means standards. To guide this movement toward higher degrees of interoperability we look to
Wang et al. (2009) who describe six levels of interoperability (Figure 3); in order of increasing
capacity for interoperation: technical, syntactic, semantic, pragmatic, dynamic, and conceptual.
These levels are further characterized as advancing from integratability, to interoperability, to
composability. Differences between these levels reflect the type of information to be exchanged,
not the technology that implements the exchange.
Figure 3. Levels of Conceptual Interoperability Model (Image adapted from Tolk et al., 2006).
A reasonable goal for IEM in the near term is pragmatic interoperability, which according to
Wang et al. (2009), populates a modeling system design with components made available by
different developers. To achieve this with technology, the components must be published with
syntactic, semantic, and contextual information that enables integration choices and decisions.
This information should be in machine readable format. Before technology can automate the
32
interoperability process, scientists and modelers representing the disciplines of IEM must
convene and develop the required linguistics.
In the longer term, the IEM community should focus attention on achieving automated
composability, which makes it possible to construct IEM systems in a truly plug-n-play
environment. Achieving this level of interoperability will require the codification of metadata
representing a combination of problem statement, decision objectives, and conceptual model
developed by the IEM stakeholders. This information forms criteria input to a software system
capable of discovering and scanning web-based repositories and retrieving modeling components
that satisfy the criteria. These components can then be automatically integrated to form an
operational modeling system. In addition, data needed by the newly configured modeling system
will be discovered, retrieved, and processed into input files. This level of interoperation requires
codification of the intelligence and reasoning that formulates the problem statement and creates
individual components.
Perhaps the most interesting aspect of this seemingly daunting challenge is the fact that there is
no new information required to achieve these goals. Each and every IEM application currently
processes all of this information. The problem is that we have no commonly accepted and
applied means of expressing this information, much less rendering it into a machine-consumable
format.
4.2 Data access and processing for IEM
There is growing recognition by governments and academic institutions that access to data that
describes environmental and ecosystem states is critical to solving complex environmental
problems. Table 5 summarizes several ongoing efforts to provide such data at national and
international scales, including government, education, and technical organizations.
As seamless access to data becomes available (Box 4, steps 1 and 2) the next challenge is to
process and transform the data for use in IEM systems (Box 4, steps 3 and 4). This requires
extensive geo-processing, statistical treatment of data, and methods for integrating
multidisciplinary data. Tools and software systems to automate these functions in an integrated
33
fashion are beginning to emerge. The Geoscience network (GEON, Ludascher et al, 2003,
http://www.geongrid.org/) is an open collaborative project funded by the US National Science
Foundation to develop cyberinfrastructure with the ultimate goal of linking heterogeneous
scientific data and information for the purposes of knowledge discovery, sharing, and integration
into scientific workflows. The GEON Open Earth Framework (OEF) is a visual analytics suite
of software libraries and applications that assist in these data processing functions. The suite's
collection of interactive visual techniques are designed to help scientists derive insight from
complex, ambiguous, and often conflicting Earth science data. Also, Data for Environmental
Modeling (D4EM - Johnston et al., 2011) is an open source software system developed expressly
to access, retrieve, and process data for IEM. D4EM is currently linked to several US
Government databases and performs all data processing required to serve data directly to an
integrated modeling system designed to simulate interactions among watersheds and aquatic
ecosystems
4.2.1 Future needs for IEM data and data processing
There are several areas where access and processing of data can be improved. It will be
necessary for the IEM community to increase involvement in national and international
initiatives and to encourage the collection of additional types of data to enable the wide range of
IEM applications and research. Developing more standardized tools to enable discovery of and
access to data for IEM will enhance the community‘s ability to conduct IEM. Developing
methods and accessible tools for processing data into forms needed by IEM will reduce the
necessary effort and increase overall consistency among applications. It will be advantageous to
represent model results as data for direct comparison with observations. This will bring the
information flow full circle and provide researchers, assessors, and decision makers with a
coherent and continuous flow of relevant information.
34
Table 5. Examples of data access initiatives of relevance to IEM
Initiative Geoscope Description
Global Earth Observation System
of Systems (GEOSS)1
Global An international effort to coordinate a comprehensive monitoring of the state of the earth, study large
scale processes, and predict behavior of the earth system
Infrastructure for Spatial
Information in the European
Community (INSPIRE)2
EU The stated purpose is to assist in environmental policy making across national boundaries. The spatial
information considered includes seventeen topical and technical themes originating from numerous
sources throughout the EU
European Shared Environmental
Information System (SEIS)3
EU A web based system where public information providers share environmental data and information. In
implementing SEIS, the EEA is building on existing reporting systems and tools: the INSPIRE directive,
Global Monitoring for Environment and Security (GMES), and (GEOSS.)
Water Information Service for
Europe (WISE)4
EU Gateway to information on European water issues. It comprises a wide range of of data and information
collected by EU institutions to serve several stakeholders.
Environmental Resources
Information Network (ERIN)5
Australia The objective is to organize environmental information from many sources and provide standards based
tools for discovery, access, and use. The information includes maps, species distributions, documents and
satellite imagery, and covers environmental themes ranging from endangered species to drought and
pollution
Environmental Dataset Gateway
(EDG)6
US A gateway developed by the US EPA to web-based information and services. It enables data consumers to
discover, view and access data sets, as well as geospatial tools. Users also have the ability to catalog and
maintain their geospatial metadata contributions via the EPA Metadata Editor Tool
National Ecological Observation
Network (NEON)7
US Collection of data across the United States on the impacts of climate change, land use change and invasive
species on natural resources and biodiversity. Designed to detect and enable forecasting of ecological
change at continental scales over multiple decades.
US based Consortium of
Universities for the Advancement
of Hydrologic Science, Inc.
(CUASHI)8
US Developing the Hydrologic Information System (HIS), an internet-based system that provides for sharing
hydrologic time series data contributed by a wide range of providers, including the National Water
Information System of the US Geological Survey.
35
Open Geo-spatial Consortium
(OGC)9
Global Efforts to work toward global standards for access, retrieval, and use of geo-spatical data.
1 http://www.earthobservations.org/index.shtml
2 http://inspire.jrc.ec.europa.eu/index.cfm/pageid/48
3 http://ec.europa.eu/environment/seis/
4 http://water.europa.eu/
5 http://www.environment.gov.au/erin/about.html
6 https://iaspub.epa.gov/sor_internet/registry/edgreg/home/overview.do & https://edg.epa.gov/EME/
7 http://neoninc.org
8 http://his.cuahsi.org
9http://www.opengeospatial.org/
36
5. IEM COMMUNITY
In these formative years of IEM, the need for structured and unstructured communication, has
been critical. Resolving differences among disciplines and stakeholders is an intensely social
process. Facilitating and documenting social interactions is the foundation of an integrative
process. The IEM community, as represented at the workshops (Table 1), has called for even
greater efforts in this area. Integrated environmental modeling is not just an activity that
discovers, organizes, and delivers knowledge and technology for others to assimilate and apply.
The IEM enterprise (science, technology, community, and applications) is driven by the needs of
others: it exists because problems and decisions require it. Thus, the formation, structure, and
social processes of the broad IEM community are an important area of focus and study. In many
fields and sectors, openness, collaboration, sharing, and social learning have been shown to drive
innovation and growth (Tapscott and Williams, 2006). Structured community processes can
reduce duplication of efforts and increase leveraging of resources and overall efficiency. These
behavioral attitudes and characteristics are often expressed through formal communities of
practice. Lave and Wagner (1991) introduced the term ―community of practice‖ and defined it as
―a set of relations among persons, activity, and world, over time and in relation with other
tangential and overlapping communities of practice.‖ Box 5 lists the key characteristics of an
IEM community of practice.
37
Box 5: Characteristics of an IEM Community of Practice
The community is transdisciplinary, comprised of a diverse set of stakeholders representing
environmental policy and decision making, funding organizations (research and application),
social, economic, and environmental sciences, industry, environmental advocacy groups, and the
public.
Membership is voluntary and participation is open and equally available to all regardless of
organizational affiliation and stakeholder identity.
Interactions involve interdisciplinary communication, sharing, collaboration, and negotiation
oriented toward achieving shared goals of integration.
The community establishes relationships with other communities that may provide specific
discipline or stakeholder components.
Transparency, efficiency, and openness are promoted. Collaboration is encouraged across
organizational boundaries.
5.1 Current state of the IEM community of practice
The early phases of IEM development were conducted by disparate small groups in government,
academia and the private sector working on science and applications, mostly independent of one
another. Results of these efforts were disseminated through traditional outlets such as conference
presentations, technical reports, journal articles, and websites. These groups advanced the
science and application of IEM, but their efforts and products typically were not coordinated and
compatible. These early efforts produced a large number of IEM approaches and functionally
similar and operationally unique software components and systems. For example, there are now
dozens of modeling frameworks, each claiming unique merits (van Evert et al., 2005). Recently,
several larger groups relevant to IEM (not necessarily formal communities of practice)
specifically related to modeling domains or select entities (Table 5) have formed. The emergence
of these communities demonstrates a widely shared belief that topics related to IEM benefit
greatly from the establishment of formal relationships among stakeholders.
38
Table 5. IEM Relevant Communities or Communities of Practice.
Name Scope
ISCMEM1 Formal effort of modeling groups at nine U.S. federal agencies to share ideas, models, and projects.
CSDMS2 Convenes experts to facilitate development and dissemination of integrated software modules that
simulate dynamics of the earth's surface, focusing on the interface between lithosphere, hydrosphere,
cryosphere, and atmosphere
ESMF3
Produces shareable software for climate, weather, and related applications by building high-
performance, flexible infrastructure that increases ease of use, performance portability,
interoperability, and reuse in climate, numerical weather prediction, data assimilation, and other earth
science applications.
OpenMI4
Community of organizations that has proposed a standard for exchanging data between environmental
models.
CUASHI5 Community that facilitates discovery and access to hydrologic data (Hydrologic Information System,
HIS), sharing of hydrologic models and codes (Community Hydrologic Modeling Platform, CHyMP),
and a web portal for interactive access to widely used simulation codes and high performance
computing (HydroHub).
CIEM6 Worldwide open community whose goals are to:
Enhance IEM learning and education (establish best practices; produce and share educational
tools);
Leverage IEM solutions (make them accessible and reusable);
Promote transparent, efficient, and open science and technology;
Facilitate scientific collaboration; and
Allow efficient use of resources (limit duplication in technology development).
1ISCMEM: Interagency Steering Committee for Multi-media Environmental Modeling (http://iemhub.org/topics/ISCMEM) 2CSDMS: Community Surface Dynamics Modeling System (http://csdms.colorado.edu/wiki/Main_Page) 3ESMF: Earth System Modeling Framework (http://www.earthsystemmodeling.org/index.shtml) 4CUAHSI: Consortium of Universities for the Advancement of Hydrologic Science, Inc. (http://www.cuahsi.org/) 5OpenMI: Open Modeling Interface (Association) (http://www.openmi.org/) 6CIEM: Community of Practice for Integrated Environmental Modeling (http://www.iemhub.org)
5.1.1 Future needs of IEM Community of Practice
With the formation of several IEM-related communities and the launch of iemHUB with its web-
based collaboration tools, the community approach is becoming a reality. The challenge is to
foster and promote participation from the full community; which is often easier said than done
(Voinov et al. 2010). This section includes several goals related to establishing a highly
interactive community for IEM: extending the community, removing organizational and
institutional barriers, and formalizing the community of practice.
39
At the present, the IEM community is represented by the individuals who have participated in the
workshops. In order to achieve the goals and conduct the activities proposed in this paper, the
membership of a formal community of practice must be extended to include additional science
stakeholders (i.e., modelers from the disciplines of economics and social systems) and decision
stakeholders (i.e., lay public, citizen scientists, and decision/policy experts). IEM will not move
forward without the ongoing cooperation and collaboration from a comprehensive community.
This is exemplified best by considering the potential contribution of citizen science networks.
Citizen science networks can contribute in many different ways. First, they can directly monitor
natural resources and environmental conditions by gathering information and data that might be
used in IEM applications. Citizen scientists can help engage the broader community in the design
and review of IEM applications by asking the questions that need to be answered, and by
providing greater transparency on the methods by which those questions will be answered. If
interested and enlightened citizen scientists do not understand the essentials of an IEM
application, there is little chance that the broader community and policy/decision makers will.
Citizen scientists can help to educate the general public about IEM science and its applications.
Because of their interest and knowledge, citizen scientists are an excellent group to test new IEM
and monitoring technologies and processes (e.g. Smartphone apps). They can be used to vet or
advise on structured decision making and other community engagement and decision making
processes that are critical to the success of IEM applications. Finally, citizen science groups can
provide historical knowledge and local stakeholder continuity to ensure persistence and
improvement of IEM application efforts.
Removing organizational and institutional barriers to the community conduct of IEM is a
challenge. The scale at which a community of practice is achieved reflects the scale at which
IEM learning occurs and problem solutions are accessible and transferable. Currently, the scale
of community where access and transfer of solutions occurs is, at best, organizational or defined
by a particular integrated modeling domain (e.g. Table 5). Sharing high-level ideas is easier than
sharing the process that produces tangible products. Each organization or group has a mission
which defines its work, needs, and priorities. There are many fundamental differences in the
types of inter- or intra-organizational responsibilities including: regulatory/enforcement, resource
management, scientific research/monitoring, education and outreach, issue advocacy, community
40
engagement, commercial or private sector development, etc. Even when there is a joint interest
among several organizations or groups (e.g., working together to develop or apply an IEM),
differences in priorities or alignment of perspectives can create barriers to effective
collaborations. Organizations can be the greatest facilitators, or barriers, to a community of
practice approach. For the IEM community to be successful organizations must come to terms
with the need to share and collaborate in their day to day operations. This means recognizing
mutual interests and facilitating cross-organizational working relationships to pursue them.
As collaboration tools are put in place and institutional barriers are removed, the full community
moves toward participation. It is the perfect time to formalize the discipline of IEM and establish
its community identity and practices. To that end, a draft charter, describing the structure and
operation of a formal Community of Practice for IEM has been developed3. The next step is to
share this document with the larger IEM community to gain feedback and with organizational
managers to gain sponsorship.
6. THE IEM VISION AND ROADMAP TO THE FUTURE
The previous sections provided a holistic view of the IEM landscape and described, from the
current community‘s perspective, what is needed to move IEM forward. With this background
we now turn to organizing the future of integrated environmental modeling activities. Our goal is
to present a vision of IEM for the next decade and provide a roadmap that can help achieve the
vision. Section 6.1 presents the vision. The roadmap is described as a combination of cultural
changes and specific goals and activities to be pursued. The cultural changes, described in
Section 6.2, reflect a shift in the way environmental problems are defined and how solutions are
developed. The goals and activities are presented in Section 6.3; they represent the desired future
state of IEM and the steps required to achieve this state. These goals and activities represent an
integration of the information presented throughout the paper and thus include items that cross
the four principal elements of the IEM landscape.
3 http://iemhub.org/resources/382/supportingdocs
41
Three notes concerning the roadmap, first we assume that disciplinary research and development
will continue within the discipline-based communities. We do not articulate specific science
directions per discipline however IEM scientists and modelers will need to work with these
communities in an effort to efficiently access and properly integrate their science. Secondly, it is
clear that monitoring data and observations will play an important role in IEM and its
applications and supported decisions. However, the present roadmap does not attempt to define
specific monitoring based activities, but instead encourages increased collaboration between the
IEM modeling community and the monitoring community. Finally, this roadmap is analogous to
a national roadway system, as opposed to a detailed street map of cities and neighborhoods. It
does not attempt to design an implementation plan, although we believe the seeds to such efforts
are given.
6.1 Vision of the Future of Integrated Environmental Modeling
The IEM vision (Box 6) reflects the complexities of modern environmental problems and the
challenges of providing the information needed to inform decisions and policies.
Box 6. The Vision of the Future of Integrated Environmental Modeling The vision is of a future where complex environmental decisions and policies are based on systems science,
cost effective, socially responsible, adaptive, and sustainable.
Applications reflect problem formulations and solution approaches that are holistic, establishing a view of the
human-environmental system that recognizes the interdependent relationships among responsible organizations,
management scenarios (decision options), co-occurring stressors, the environment, and socio-economic
structures. This integrative systems approach engages all stakeholders throughout the decision process.
IEM Science is transdisciplinary, involving the integration of social, economic, and environmental science into
modeling systems that describe and forecast the behavior of the human-environmental system in response to
natural and human induced stress. The science provides methods for characterizing and communicating model
sensitivities and uncertainty to both decision and science stakeholders. IEM is a specific curriculum and is
taught in schools and Universities.
Information Technology provides for discovery, access, and integration of science components (data, models,
and assessment strategies) developed by stakeholders world-wide. Integrated modeling systems are constructed
and executed on a variety of platforms serving research, applications, and education.
A Community of IEM stakeholders and associated organizations engages in, invests in, and contributes to, the
shared and open development of integrated environmental science and related computer-based technologies.
Additionally, the community grows to engage interested or concerned citizens, decision makers, and their
associations and gains their support for the development and application of IEM.
42
6.2 Guiding Principles for Advancing IEM
Modern environmental problems have required fundamental adjustments in the execution of the
IEM supported decision and policy development process. In particular, the manner in which
environmental problems are perceived, articulated, assessed, and resolved is evolving rapidly. In
this section we present several key concepts, attitudes, and priorities that are driving this
evolution. We suggest that these guiding principles should form the foundation for planning and
executing IEM activities. They are closely related to one another but each possesses a defining
uniqueness that warrants attention. The guiding principles include: stakeholder involvement,
community development, systems thinking, openness, investment, and reuse.
6.2.1 Stakeholder Involvement
There is a need to expand current efforts to mobilize and involve the array of stakeholders within
the IEM process. The numerous stakeholders involved in an IEM problem and solution (Section
1.1) possess specific interests, concerns, and expertise and are principally responsible for one or
more steps in the decision process. The relationship among stakeholders is conceptually similar
to the relationship among components of the social-economic-environmental system. That is,
there is an information flow among stakeholders; each stakeholder provides information for
others and consumes information contributed by others. There is a system goal to which all
stakeholders are oriented.
Ensuring appropriate stakeholder participation, assimilating the range of stakeholder
perspectives, contributions and needs, and developing a consensus understanding of the problem,
decision goals, conceptualized system, and solutions is a significant challenge. As a principle,
full and open stakeholder involvement must be viewed as an essential ingredient for successful
decision making.
6.2.2 Community Development
There are two aspects of community development that are important to the future of IEM and
supported decisions. First, there is a need to link together the stakeholder communities
responsible for individual problems and decisions. Secondly, there is a need to link disciplinary
43
scientists and modelers with IEM scientists and modelers to form a larger community where
transdisciplinary research and development is pursued.
These communities are important because of the degree of social and technical complexity
associated with modern problems. Defining the right problem and configuring an appropriate
solution requires experience, which in turn requires time and effort. Being connected enables
experience gained addressing one problem to be available in a timely and transparent manner to
others facing similar problems.
Each of these community development efforts is an effort to form a community of communities.
At the present time we bump into each other in various venues (public meetings, scientific
conferences) but by and large we are not organized to maximize the exchange of knowledge and
experience related to IEM. Given the magnitude of the problems being faced and the
consequences of poor decisions and policies it is in our collective best interest to address the
problem of community development.
6.2.3 Systems Thinking
A systems framework for articulating, discussing, and resolving complex environmental
problems is necessary, because modern environmental problems are characterized by: (1) the
extent of the natural and built environment involved; (2) the dynamic and inter-dependent nature
of stressors and their impacts; (3) the diversity of the stakeholder community; and (4) the explicit
integration of social, economic, and political considerations. Intrinsic to complex systems are the
virtually limitless combinations of significant component interactions that may occur and the
idea of emergent properties; i.e. the properties of the system that result from interactions among
the parts. Intrinsic to solutions of complex problems is the idea that an appropriate decision (i.e.,
one that is science-based, cost effective, socially responsible, adaptive, and sustainable) is driven
by social interactions among stakeholders holding a myriad of partially compatible interests.
This is only possible in a systems framework, without which the effectiveness of decisions is left
solely to chance.
44
Systems thinking is not only a responsibility for science and technology oriented stakeholders. It
is also a responsibility for other stakeholders, including decision makers, managers, assessors,
the public, industry representatives, etc. Understanding and appreciating how the social-
economic-environmental system works and how individual interests, concerns, or processes fit
into the larger system are key to achieving the IEM vision. This has several consequences. First,
movement toward systems thinking means that every IEM related activity, no matter how small,
is conceptualized, designed, and executed with full consideration of the relevant system of
interacting parts. Second, all interests, preferences, and concerns are viewed together. Third,
trade-offs and compromises are explicitly recognized and tracked as an intrinsic responsibility of
IEM-based decision making.
There are no unique barriers to this undertaking. Progressive, and integrated, systems thinking
are natural to human development. Our lack of knowledge and experience with this topic,
combined with the scale and scope of consequences of poor decisions and policies, simply makes
the process more challenging and urgent.
6.2.4 Openness
The IEM community must embrace and encourage openness, cooperation, and collaboration as it
relates to the planning and execution of IEM activities. The greater IEM community presently
shows a high degree of openness and willingness to share ideas. The process by which product
development is initiated and executed, however, remains relatively closed. Products (data,
models, methods) are typically made available by their developers only upon completion and/or
publication. Ironically, once completed, it is common for developers to invite collaboration on
further development. Groups and organizations are less inclined to accept these delayed
invitations and more inclined to conduct their own in-house development (often aimed at nearly
the same product functionality).
Why does this happen and can it be changed? An unflattering explanation is the ‗not invented
here‘ syndrome; everyone wants to be in the driver‘s seat. Given the choice to lead (and control)
or collaborate (and contribute), we often choose to lead. Another explanation is that sharing of
information, especially with a large community where personal relationships are not well
45
established, often does not fully recognize (and reward) the efforts and the leadership of those
who have contributed. Such sharing may also result in a loss of intellectual property rights or in a
loss of economic development potential by a given party. Yet another possible explanation is that
conducting development that is dependent on timely and quality execution by external parties
often incurs risks and uncertainty that would not exist if the development is conducted ―in-
house‖. Uncertainties and risk exist because existing products are not sufficiently accessible and
documented to allow timely adoption for a new, albeit similar, purpose. The reality is most likely
that each of these considerations contribute to an explanation why openness is lacking.
Despite these issues, progress can be made. There are an increasing number of open source
software development projects that attract voluntary contributions from a world-wide
community. From these experiences we know that under some conditions, open, cooperative,
collaborative efforts do work. The future of IEM depends on identifying processes and
mechanisms that overcome the barriers and encourage openness and collaboration.
6.2.5 Reusable products
Across the world there is a large community of scientists, modelers, and technologists
developing databases, modeling components, and infrastructure for IEM. Our collective ability to
efficiently share and reuse the products of these efforts remains limited. Reuse can be achieved at
different levels. On the high end, reusable means an IEM component can be accessed and reused
in a context other than its original development without additional effort or reusable ―as is‖.
From a more practical perspective, reusability means that only a fraction of the original effort is
required in order to assimilate and utilize the component or application, and that the re-user is
certain of knowing the science basis of the application or component, its complete functionality,
and setup (i.e., assumptions and parameterizations).
There are real benefits to rethinking the way we build, execute, and publish the applications,
science, and technology of IEM to optimize reuse and adaptation:
More cost effective assessments.
More decisions informed with science-based integrated modeling.
46
Increased consistency in the application of science. This, in turn, increases the
effectiveness of communication among stakeholders.
A natural evolution, identification, promotion, and sharing of best practices related to
applications, science, and technology.
Increased ease of review and implementation of quality controls and audits for IEM and
its applications; and consequently increased credibility for IEM-supported decision
making.
Ability to extend the life of a solution and employ adaptive management strategies (e.g.,
by collecting new data and updating the solution and management strategy.
Ability to assimilate solution components from multiple developing organizations to form
solutions to new problems
Increased research opportunities. Using software tools researchers could access archived
applications/solutions and assess and improve the modeling process itself.
6.2.6 Investment
Infrastructure to facilitate IEM is currently limited to scales that serve groups or organizations
that have resources to develop IEM systems. These systems are generally focused on specific
applications of importance to the organization. The science and technology contained in these
systems, however, is applicable to many other problems in other organizational contexts. It
remains relatively inaccessible due to a combination of incompatible technologies and a lack of
documentation. For IEM to effectively serve decisions and policies in the 21st century, strategic
investments are required in infrastructure to facilitate the wider community‘s ability to discover,
retrieve, integrate, and apply the products of IEM science. Similarly, strategic investments
should focus on the development of human-based and technology facilitated infrastructure for
engaging IEM stakeholders in the greater IEM process, including problem definition, scenario
development, data gathering, modeling, reviews, and decision making frameworks. In essence,
IEM and related decision processes need to become a recognized and supported science
discipline. Until these investments become a specific community priority, IEM will remain an ad
hoc solution to an expanding inventory of complex problems.
6.3 The IEM Roadmap
47
This section reviews the needs expressed throughout the paper and organizes them into a
coordinated, coherent set of goals and related activities. The goals define desired endpoints and
the activities identify potential steps to take in order to achieve them. Some activities standalone,
that is, they are not dependent on information or products from other tasks. Most activities,
however, have dependencies and require a certain sequencing.
In figures 5 through 9, activity sequencing progresses from the bottom to the top of the figure.
Each activity is labeled with icons representing which landscape elements are involved (i.e.,
application, science, technology, community). The first icon listed represents the primary
element involved and the others contribute important information, expertise, and perspective.
Thus, for example, to achieve the goals relevant to IEM applications may require the integration
of efforts across science, technology, and community elements. The roadmap goals and activities
are summarized below.
Goal 1: Establish and promote both the science discipline of IEM and a formal community
of practice. The activities required to achieve this goal focus on: defining the science and
community domains; establishing appropriate roles and relationships among the many
stakeholders; establishing a web based technology to facilitate interactions involving a globally
dispersed community; and conducting several activities oriented toward outreach and
establishing community culture and practices. The sequencing of activities is presented in Figure
5.
Goal 2: Provide community access to tools for designing, executing, documenting, and
archiving IEM applications and modeling science. The desire is to move beyond the relatively
limited accessibility and usability of IEM science and technology to dramatically increasing
collaboration among developers and ease of use for others. There are two related activity tracks
(Figure 6): one focused on the science and technology of decision support systems and another
focused on quantitative modeling. The activities required to achieve these goals are similar and
include developing a series of standards designed to facilitate the development and publication of
reusable IEM technologies.
48
Goal 3: Advance the state-of-the-art of IEM applications. Activities supporting this goal
(Figure 7) include developing better methods for involving all stakeholders in the application
process, advancing the decision making process by further developing adaptive management
strategies and methods, developing methods for peer reviewing complex applications, and using
IEM applications and vehicles for educating stakeholders.
Goal 4: Advance the science of IEM. A principle focus revolves around the need to apply
systems thinking to the process of building modeling solutions for complex problems. Important
considerations are being explicit and rigorous in developing problem statements and
management scenarios and building appropriately complex conceptualizations of the system.
Other important science activities (Figure 8) include: advancing practitioners access to the wide
array of data required for IEM; developing the science of information exchange among modeling
components with varying dimensional attributes; encouraging the development of thorough peer
review protocols; and developing standards methods for addressing model evaluation and
uncertainty.
Goal 5: Establish software development protocols and explore innovative technologies to
enhance the development and application of IEM. The supporting activities (Figure 9) are
specific to advancing the technological means for developing, documenting, testing, reviewing,
and archiving products ranging from small local databases to science algorithms of an
environmental process to full modeling systems to full applications. It will be necessary to
develop standardized means for versioning software, discovering and integrating components,
publishing software for reuse, and exploring innovative means of leveraging the internet and
other emerging social media for the purpose of extending the overall effectiveness of IEM in the
decision making context.
7. CONCLUDING STATEMENT
This paper has described the current state and understanding of IEM. Based upon a knowledge of
what has already been accomplished and on the results of a number of workshops that were
critical to the IEM community, it has proposed a roadmap for the future. The main elements of
49
the roadmap can be split into applications, science, technology and community. The hope is that
within the next decade there is a vibrant, connected international community that will have the
tools and techniques by which to routinely undertake IEM and effectively interact with
stakeholders. When the goals for our roadmap are achieved, IEM will be accessible to a wider
community of users and a new generation of research and development will be enabled. We also
envison that environmental management and regulation decisions at governmental levels can
become increasingly informed by IEM applications. With greater sharing, hard work, and
appropriately directed resources, this vision is achievable.
50
Figure 5: IEM Goal 1
51
Figure 6: IEM Goal 2
52
Figure 7: IEM Goal 3
53
Figure 8: IEM Goal 4
54
Figure 9: IEM Goal 5
55
Disclaimer
The views expressed in this paper are those of the authors and do not necessarily reflect the
views or policies of the affiliated organizations.
References
ABARE-BRS (Australian Bureau of Agricultural and Resource Economics – Bureau of Rural
Sciences), 2010. Assessing the regional impact of the Murray–Darling Basin Plan and the
Australian Government‘s Water for the Future Program in the Murray–Darling Basin.
ABARE–BRS Project 43051. Commonwealth of Australia. Canberra, ACT, Australia.
Akbar, M., Aliabadi, S., Patel, R., Marvin, W., 2011. Fully Automated and Integrated Multi-
Scale Forecasting Scheme for Emergency Preparedness. Environ. Model. Software (this
issue).
Ahuja, L.R., Ascough II, J.C., David, O., 2005. Developing natural resource models using the
object modeling system: feasibility and challenges. Adv. Geosci. 4, 29–36.
Arciniegas, G., Janssen, R., Rietveld, P., 2011. Effectiveness of collarborative map-based
decision support tools: results of an experiment. Environ. Model. Software (this issue).
Ascough II, J.C., Maier, H.R., Ravalico, J.K., Strudley, M.W., 2008. Future research challenges
for incorporation of uncertinaty in environmental and ecological decision-making.
Ecological Modelling 219, 383-399.
Aster, R.C., Borchers, B., Thurber, C. H., 2005. Parameter Estimation and Inverse Problems.
Amsterdam, Elsevier Academic Press.
Babendreier, J. E., Castleton, K. J., 2005. Investigating uncertainty and sensitivity in integrated,
multimedia environmental models: tools for FRAMES-3MRA. Environ. Model. Software
20 (8), 1043-1055.
Bailey, G.W., Mulkey, L.A., Swank, R.R., 1985. Environmental Implications of Conservation
Tillage: A System Approach, in: D‘Itri, F.M. (Ed.), A System Approach to Conservation
Tillage. Lewis Publishers, Inc., Chelsea, MI pp. 239-265.
Beven, K., 2007. Towards integrated environmental models of everywhere: uncertainty, data and
modelling as a learning process. Hydrol. Earth Syst. Sci. 11 (1), 460-467.
Beven, K., 2009. Environmental Modelling: An Uncertain Future? Taylor & Francis, UK.
56
Blind, M.W., Moore, R.V., Scholten, H.M., Refsgaard, J.C., Borowski, I., Giupponi, C., Estrela,
M. and Vanrolleghem, P.A., 2005a. Current results of the EC-sponsored catchment
modelling (CatchMod) cluster: Part 1: ―Cross-cutting issues‖, IWA 2005 Watershed &
River Basin Management Specialist Group Conference, Calgary, Alberta, Canada.
Blind, M.W., Borgvang, S.A., George, D.G., Froebrich, J., Zsuffa, I., Vanrolleghem, P.,
Jørgensen, L.F. & de Lange, W.J., 2005b. Current results of the EC-sponsored catchment
modelling (CatchMod) cluster: Part 2: ―Water topics and synthesis‖ IWA 2005
Watershed & River Basin Management Specialist Group Conference, Calgary, Alberta,
Canada.
Bolte, J.P., Hulse, D.W., Gregory, S.V., Smith, C., 2006. Modeling biocomplexity - actors,
landscapes and alternative futures. Environ. Model. Software 22, 570-579.
Boumans, R, Costanza, R., Farley, J., Wilson, M.A., Portela, R., Rotmans, J., Villa, F., Grasso,
M., 2002. Modeling the dynamics of the integrated earth system and the value of global
ecosystem services using the GUMBO model. Special Issue: The Dynamics and Value of
Ecosystem Services: Integrating Economic and Ecological Perspectives. Ecol. Econ. 41,
529–560.
Castronova, T.A., Goodall, J.L., 2010. A generic approach for developing process-level
hydrologic modeling components. Environ. Model. Software 25, 819–825.
Clark, M.P.,Kavetski, D., Fenicia, F., 2011. Pursuing the method of multiple working hypotheses
for hydrological modeling. Water Resour. Res. 47, W09301.
Cohen, Y., 1986. Pollutants in a Multimedia Environmental. Plenum Press, New York.
Costanza, R., A. Voinov, R. Boumans, T. Maxwell, F. Villa, H. Voinov and Wainger, L. 2002.
Integrated ecological economic modeling of the Patuxent river watershed, Maryland.
Ecological Monographs, 72 (2), pp.203-231.
Daniels, M., 1999. Integrating Simulation Technologies with Swarm, Agent Simulation:
Applications, Models and Tools Conference. University of Chicago.
David, O., Markstrom, S.L., Rojas, K.W., Ahuja, L.R., Schneider, W., 2002. The object
modeling system, in: Ahuja, L.R., Ma, L., Howell, T.A. (Eds.), Agricultural System
Models in Field Research and Technology Transfer. Lewis Publishers, Boca Raton, FL,
USA, pp. 317-344.
Delsman, J., Kukuric, N., Veldhuizen, A., Tiktak, A., Kroon, T., 2009. Netherlands Hydrological
Modeling Instrument - Coupling groundwater and surface water for national policy
analysis. In: IAHS-IAH Conference Hyderabad, 7 September 2009
Ellarby, M.J., Kite, S.S., 2006. Are Rich Visual Environments a Gimmick or a Real Aid in the
Understanding and Acceptance of Simulation Models and Results, in: Robinson, S.,
57
Taylor, S., Brailsford, S., Garnett, J. (Eds.), Proceedings of the 2006 Operations Research
Society Simulation Workshop. Leamington Spa, UK, March 28-29, 2006, pp. 295-299.
EPA (US Environmental Protection Agency). 2000. Toward Integrated Environmental Decision-
making. EPA-SAB-EC-00-011. Washington, DC. Science Advisory Board.
http://www.epa.gov/sab/pdf/ecirp011.pdf
EPA (US Environmental Protection Agency). 2004. The Integrated Environmental Strategies
Handbook. EPA430-B-04-006. Washington, DC. Office of Air and Radiation.
http://www.epa.gov/ies/documents/ies_handbook.htm
EPA (US Environmental Protection Agency), 2007. Workshop Report: Integrated Modeling for
Integrated Environmental Decision Making Workshop. January 30–February 1, 2007.
Research Triangle Park, NC http://www.epa.gov/crem/crem_integmodelwkshp.html
EPA (US Environmental Protection Agency), 2008a. Workshop Report: Collaborative
Approaches to Integrated Modeling: Better Integration for Better Decision-Making.
December 10–12, 2008. Phoenix, AZ http://www.epa.gov/crem/integrated-modeling-
workshop2008.html
EPA (US Environmental Protection Agency), 2008b. Integrated Modeling for Integrated
Environmental Decision Making. EPA-100-R-08-010. Washington, DC. Office of the
Science Advisor.
http://www.epa.gov/CREM/library/IM4IEDM_White_Paper_Final_(EPA100R08010).pd
f
EPA (US Environmental Protection Agency). 2009. Guidance on the Development, Evaluation,
and Application of Environmental Models. EPA/100/K-09/003. Washington, DC. Office
of the Science Advisor. http://www.epa.gov/CREM/library/cred_guidance_0309.pdf
European Commission, 2000, Directive of the European Parliament and of the Council
2000/60/EC Establishing a Framework for Community Action in the Field of Water
Policy, Official Journal 2000 L 327/1, European Commission, 73pp., Brussels, Belgium.
Farber, S., Costanza, R., Childers, D.L., Erickson, J., Gross, K., Grove, M., Hopkinson, C.S.,
Kahn, J., Pincetl, S., Troy, A., Warren, P., Wilson, M., 2006. Linking Ecology and
Economics for Ecosystem Management. BioScience 56 (2), 117-129.
Fenner, K., Scheringer, M., MacLeod, M., Matthies, M., McKoone, T.E., Stroebe, M., Beyer, A.,
Bonnell, M., Le Gall, A.-C., Klasmeier, J., Mackay, D., van de Meent, D.W., Pennington,
D., Scharenberg, B., Suzuki, N., Wania, F., 2005. Comparing Estimates of Persistence
and Long-Range Transport Potential among Multimedia Models. Environ. Sci. Technol.
39 (7), 1932-1942.
58
Fischenich, J.C. 2008. The Application of Conceptual Models to Ecosystem Restoration. U.S.
Army Corps of Engineers ERDC/EBA TN-08-1. ERDC Environmental Laboratory,
Vicksburg, MS.
Gijsbers, P., Gregersen, J., Westen, S., Dirksen, F., Gavardinas, C., Blind, M., 2005. OpenMI
Document Series: Part B Guidelines for the OpenMI (version 1.0). Centre for Ecology
and Hydrology, Wallingford, UK. (http://www.openmi.org/reloaded/about/documents-
publications/B_Guidelines.pdf, last accessed September 26, 2011).
Guzy, M. R., Smith, C.L., Bolte, J.P., Hulse, D.W., Gregory, S.V., 2008. Policy research using
agent-based modeling to assess future impacts of urban expansion into farmlands and
forests. Ecol. Soc. 13 (1), 37.
Hanna, S.R., 1988. Air quality model evaluation and uncertainty. J. Air Pollut. Control Assoc. 38
(4), 406-412.
Hart, B.T., Shenton, W., Chan, T., 2009. Beyesian Network Models for Einvornmental Flow
Decision-Making. Interim Final Report. Land & Water Australia Project UMO49.
Available online at http://www.sci.monash.edu.au/wsc/publications/docs/2009-interim-
final-report.pdf. Accessed December 5, 2011)
Heubusch, K. 2006. Interoperability: What it Means, Why it Matters. J. AHIMA 77 (1), 26-30.
Hill, C., DeLuca, C., Balaji, Suarez, M., da Silva, A., 2004. The architecture of the earth system
modeling framework. Comput. Sci. Eng. 6 (1), 18-28.
Hill, M.C., Tiedeman, C.R., 2007. Effective calibration of groundwater models, with analysis of
data, sensitivities, predictions, and uncertainty. John Wiley and Sons, New York.
Hughes, Andrew; Jackson, Chris; Mansour, Majdi; Bricker, Stephanie; Barkwith, Andrew;
Williams, Ann; Abesser, Corinna; Wang, Lei. 2011 Integrated modelling within the
Thames Basin : examples of BGS work. [Poster] In: Cities, catchments and coasts:
applied geoscience for decision-making in London and the Thames Basin, London, UK,
13 May 2011. (http://nora.nerc.ac.uk/14267/)
Hulse, D., Branscomb, A., Enright, C., Bolte, J., 2008. Anticipating floodplain trajectories: a
comparison of two alternative futures approaches. Landsc. Ecol. 24 (8), 1067-1090.
ICSU (International Council for Science), 2010. Regional Environmental Change: Human
Action and Adaptation. International Council for Science, Paris.
IEEE (Institute of Electrical and Electronics Engineers), 1989. IEEE Standard 610.3-1989 –
IEEE Standard Glossary of Modeling and Simulation Terminology, Institute of Electrical
and Electronics Engineers Standards Association, New York, New York.
IPCC (Intergovernmental Panel on Climate Change), 2007. Climate Change 2007: The Physical
Science Basis. Contribution of Working Group I to the Fourth Assessment Report of the
59
Intergovernmental Panel on Climate Change, in: Solomon, S., Qin, D., Manning, M.,
Chen, Z., Marquis, M., Averyt, K.B., Tignor, M., Miller, H.L. (Eds.), Cambridge
University Press, Cambridge, United Kingdom and New York, NY.
Jakeman, A. J., Letcher, R. A., 2003. Integrated assessment and modelling: features, principles
and examples for catchment management. Environ. Model. Software 18 (6), 491-501.
Jakeman, A. J., Letcher, R. A., Norton, J. P., 2006. Ten iterative steps in development and
evaluation of environmental models. Environ. Model. Software 21 (5), 602-614.
Jassby, A. D., Cloern, J. E., 2000. Organic Matter Sources and Rehabilitation of the Sacramento-
San Joaquin Delta (California, USA). Aquat. Conserv: Mar. Freshw. Ecosyst. 10, 323-
352.
Johnston, J. M., McGarvey, D. J., Barber, M. C., Laniak, G., Babendreier, J. E., Parmar, R.,
Wolfe, K., Kraemer, S. R., Cyterski, M., Knightes, C., Rashleigh, B., Suarez, L.,
Ambrose, R., 2011. An integrated modeling framework for performing environmental
assessments: Application to ecosystem services in the Albemarle-Pamlico basins (NC and
VA, USA). Ecol. Model. 222 (14), 2471-2484.
Lancaster, J.J., 2007. Integrated Monitoring, Modeling, and Management (IM3) Methodology
Applied to Governance of Global Energy Resources and Global Energy Use. Proceedings
of the Amsterdam Conference on the Human Dimensions of Global Environmental
Change ‗Earth System Governance: Theories and Strategies for Sustainability.‘
Amsterdam, The Netherlands. May 2007.
Lave, J., Wenger, E., 1991. Situated Learning: Legitimate Peripheral Participation. Cambridge
University Press, Cambridge.
Lehman, P. W., Giulianotti, J., Sevier, J., 2001. The Contribution of Algal Biomass to Oxygen
Demand in the San Joaquin River Deep Water Channel, Fall 2000 (Draft Report),‖
Department of Water Resources, Environmental Services Offices, Sacramento, CA.
Linker, L.C., Shenk, G.W., Dennis, R.L., Sweeney, J.L., 1999. Cross-Media Models for the
Chesapeake Bay Watershed and Airshed. Chesapeake Bay Program Office, Annapolis,
MD.
Litton, G. M., 2001. Sediment Oxygen Demand, Sediment Deposition Rates and Biochemical
Oxygen Demand Kinetics in the San Joaquin River near Stockton, California, Fall 1999
(Final). Report prepared for City of Stockton and San Joaquin River Dissolved Oxygen
TMDL Technical Committee, June 25 (2001).
Liu, Y., Gupta, H., Springer, E., Wagener, T., 2008. Linking science with environmental
decision making: Experiences from an integrated modeling approach to supporting
sustainable water resources management. Environ. Model. Software 23 (7), 846-858.
60
Lloyd, W., David, O., Ascough II, J.C., Rojas, K.W., Carlson, J.R., Leavesley, G.H., Krause, P.,
Green, T.R., Ahuja, L.R., 2011. Environmental modeling framework invasiveness:
Analysis and implications. Environ. Model. Software. 26, 1240-1250.
Ludascher, B., Lin, K., Brodaric, B., Baru, C., 2003. GEON: Toward a Cyberinfrastructure for
the Geosciences—A Prototype for Geologic Map Integration via Domain Ontologies.
U.S. Geological Survey Open-File Report 03-071.
Luna-Reyes, L.F., 2003. Model Conceptualization: A Critical Review, in: Eberlein, R., Diker,
V., Langer, R., Rowe, J. (Eds.), Proceedings of the 21st International Conference of the
System Dynamics Society. July 20 - 24, 2003 New York City, NY
http://www.systemdynamics.org/conferences/2003/proceed/PAPERS/236.pdf (last
accessed December 6, 2011)
Maidment, D.R., Hooper, R.P., Tarboton, D.G., Zaslaksky, I., 2009. Accessing and sharing data
using CUAHSI Water Data Services. Hydroinformatics in Hydrology, Hydrogeology and
Water Resources (Proc. of Symposium JS.4 at the Joint IAHS & IAH Convention,
Hyderabad, India, September 2009). IAHS Publ. 331, 213-223.
Maxwell, T., Costanza, R., 1995. Distributed modular spatial ecosystem modelling. Int. J. of
Comp. Simul. 5 (3), 247-262.
Mackay, D. 1991. Multimedia Environmental Models: The Fugacity Approach. Lewis
Publishers, Michigan.
Mackay, D., 2001. Multimedia environmental models: The Fugacity Approach. 2nd ed. CRC
Press, Boca Raton, Florida, USA.
Makropoulos, C., E. Safiolea, S. Baki, E. Douka, A. Stamou, and M. Mimikou, 2010. An
integrated, multi-modelling approach for the assessment of water quality: lessons from
the Pinios River case in Greece. International Environmental Modelling and Software
Society (iEMSs). 2010 International Congress on Environmental Modelling and Software
Modelling for Environment's Sake, Fifth Biennial Meeting, Ottawa, Canada.
McIntosh, B.S., Ascough, J.C., Twery, M., Chew, J., Elmahdi, A., Haase, D., Harou, J.J.,
Hepting, D., Cuddy, S., Jakeman, V., Chen, S., Kassahun, A., Lautenbach, S., Matthews,
K., Merritt, W., Quinn, N.W.T., Rodriquez-Roda, I., Sieber, S., Stavenga, M., Sulis, A.,
Ticehurst, J., Volk, M., Wrobel, M., van Delden, H., El-Sawah, S., Rizzoli, A.E. and
Voinov, A.A. (2011) Environmental Decision Support Systems EDSS development:
challenges and best practices. In: Environmental Modelling and Software, 26, 1389-1402.
McNie, E. C., 2006. Reconciling the supply of scientific information with user demands: an
analysis of the problem and review of the literature. Environ. Sci. Pol. 10, 17 - 38.
MEA (Millenium Ecosystem Assessment), 2005. Ecosystems and Human Well-being: Synthesis.
Island Press, Washington, D.C.
61
Meadows, D.H., Meadows, D. L., Randers, J., Behrens, W., 1972. The Limits to Growth.
Universe Books, New York.
Menke, W. 1989. Geophysical data analysis, discrete inverse theory, Volume 45. International
Geophyscias Series. Academic Press, Inc., San Diego, CA.
Moore, R.V., Tindall, C., 2005. An overview of the open modelling interface and environment
(the OpenMI). Environ. Sci. Pol. 8 (3), 279-286.
Moore R., Gijsbers, P., Fortune, D., Gergersen, J., Blind, M., 2005. OpenMI Document Series:
Part A Scope for the OpenMI (Version 1.0), HarmonIT.
Muetzelfeldt, R., Masheder, J., 2003. The Simile visual modeling environment. Eur. J. Agron.
18, 345 – 358.
Noth, M., Borning, A., Waddell, P., 2000. An Extensible, Modular Architecture for Simulating
Urban Development, Transportation and Environmental Impacts. Technical Report 2000-
12-01. University of Washington, Department of Computer Science.
NRC (National Research Council), 2005. Decision Making for the Environment: Social and
Behavioral Science Research Priorities. National Academies Press, Washington, DC.
NRC (US Nuclear Regulatory Commission), 2002. Proceedings of the Environmental Software
Systems Compatibility and Linkage Workshop. NUREG/CP-0177, PNNL-13654. Hosted
by The U.S Nuclear Regulatory Commission, Professional Development Center.
Rockville, MD March 7–9, 2000. http://www.nrc.gov/reading-rm/doc-
collections/nuregs/conference/cp0177/ (last accessed: December 6, 2011).
OpenMI, 2009. The Open-MI life project website http://www.openmi-life.org/ (last accessed:
December 6, 2011).
Oreskes, N., 2003. The role of quantitative models in science, in: Canham, C.D., Cole, J.J.,
Lauenroth, W.K., (Eds.) Models in Ecosystem Science. Princeton University Press,
Princeton. pp. 13-31.
Otto-Banaszak, I., Matczak, P., Wesseler, J., Wechsung, F., 2011. Different perceptions of
adaptation to climate change: a mental model approach applied to the evidence from
expert interviews. Reg. Environ. Change 11, 217–228.
Özesmi, U., Özesmi, S.L. 2004. Ecological models based on people‘s knowledge: A Multi-step
Fuzzy Cognitive Mapping Approach, Ecol. Model. 176, 43–64.
Parker, P., Letcher, R., Jakeman, A., Beck, M.B., Harris, G., Argent, R.M., Hare, M., Pahl-
Wostl, C., Voinov, A., Janssen, M., Sullivan, P., Scoccimarro, M., Friend, A.,
Sonnenshein, M., Barker, D., Matejicek, L., Odulaja, D., Deadman, P., Lim, K.,
Larocque, G., Tarikhi, P., Fletcher, C., Put, A., Maxwell, T., Charles, A., Breeze, H.,
Nakatani, N., Mudgal, S., Naito, W., Osidele, O., Eriksson, I., Kautsky, U., Kautsky, E.,
62
Naeslund, B., Kumblad, L., Park, R., Maltagliati, S., Girardin, P., Rizzoli, A., Mauriello,
D., Hoch, R., Pelletier, D., Reilly, J., Olafsdottir, R., Bin, S., 2002. Progress in integrated
assessment and modelling. Environ. Model. Software 17 (3), 209-217.
Parson, E. A., 1995. Integrated assessment and environmental policy making: In pursuit of
usefulness. Energy Policy 23 (4-5), 463-475.
Peckham, S.D., 2010. Valuation of Model Coupling Frameworks for Use by the Community
Surface Dynamics Modeling System (CSDMS).
http://csdms.colorado.edu/mediawiki/images/Peckham_2008_IGWMC.pdf (last accessed
December 6, 2011).
Quinn, N.W.T., Jacobs, K.C., 2006. An Emergency Environmental Response System to Protect
Migrating Salmon in the Lower San Joaquin River. Environ. Model. Software 22, 416-
422.
Reckhow, K.H., 2003. Bayesian approaches to ecological analysis and modeling, in: Canham,
C.D., Cole, J.J., Lauenroth, W.K., (Eds.), Models in Ecosystem Science. Princeton Univ
Press, 168-183
Refsgaard, J.C., van der Sluijs, J.P., Hojberg, A.L., Vanrolleghem, P.A., 2007. Uncertainty in the
environemtnal modeling process – A framework and guidance. Environmental
Modelling & Software 22, 1543-1556.
Robinson, S. 2006. Conceptual Modeling for Simulation: Issues and Research requirements, in:
Perrone, L.F., Wieland, F.P., Liu, J., Lawson, B.G., Nicol, D.M., Fujimoto, R.M.(Eds.),
Proceedings of the 2006 Winter Simulation Conference, 792-800.
Rotmans, J., van Asselt, M.B.A., 2001. Uncertainty Management in Integrated Assessment
Modeling: Towards a Pluralistic Approach. Environ. Monit. Assess. 69 (2), 101-130.
Rosenberg, N. J., Edmonds, J.A., 2005. Climate Change Impacts for the Conterminous USA: An
Integrated Assessment: From Mink to the ‗Lower 48‘. Clim. Chang. 69 (1), 1-6.
Saltelli, A., Ratto. M., Andres, T., Campolongo, F., Cariboni, J., Gatelli, D., Saisana, M.,
Tarantola, S., 2008. Global sensitivity analysis, the primer: John Wiley and Sons,
Hoboken, New Jersey.
Schellekens, J., Veldhuizen, A.A., Manders, A.M.M., Winsemius, H.C. M., van Verseveld, W.J.,
te Linde, A.H., van Ulft, L.H., van Meijgaard, E., Schaap, M., Barendregt, A., Schot,
P.P., Wassen, M.J., Ward, P. J., Veldkamp, T. I. E., Mulder, H.M., Verkaik, J.,2011.
KKK-Model platform coupling. Summary report KKF01b. Deltares, TNO, Alterra, VU
Amsterdam, Universiteit Utrecht, KNMI, The Netherlands, Final Report
KFC/038A/2011, ISBN 978-94-90070-00-7.
63
Schwarz, G.E., Hoos, A.B., Alexander, R.B., and Smith, R.A., 2006. The SPARROW surface
water-quality model: Theory, application and user documentation: U.S. Geological
Survey Techniques and Methods. Book 6, Section B, Chapter 3
http://pubs.usgs.gov/tm/2006/tm6b3/. (last accessed December 6, 2011).
Sengupta, R., Bennett, D.A., 2003. Agent-based modeling environment for spatial decision
support. Int. J. of Geogr. Inf. Sci. 17 (2), 157-180.
Sidle, R.C., 2006. Field observations and process understanding in hydrology: essential components in
scaling. Hydrological Processes 20: 1439-1445.
Sleeswijk, A.W., Heijungs, R., 2010. GLOBOX: A spatially differentiated global fate, intake,
and effect model for toxicity assessment in LCA. Sci. Total Environ. 408, 2817–2832.
Sommerfreund, J.K., Gandhi, N., Diamond, M.L., Mugnai, C., Frignani, M., Capodaglio, G.,
Gerino, M., Bellucci, L.G., Giuliani, S., 2010. Contaminant fate and transport in the
Venice Lagoon: Results from a multi-segment multimedia model. Ecotoxicol. Environ.
Saf. 73, 222–230.
Stahl, C.H., Cimorelli, A., Chow, A., 2002. A new Approach to Environmental Decision
Analysis: Multi-criteria Integrated Resource Assessment (MIRA). Bull. Sci. Tech. Soc.
22 (6), 443-459
Stahl, C., Cimorelli, A., Mazzarella, Jenkins, B., 2011. Toward sustainability: A case study
demonstrating trans-disciplinary learning through the selection and use of indicators in a
decision making process, Integr. Environ. Assess. Manag. 7, 483–498.
Tapscott, D., Williams, A.D., 2006. Wikinomics: How Mass Collaboration Changes Everything,
Portfolio, New York, NY.
Tol, R.S.J., Vellinga, P., 1998. The European Forum on Integrated Environmental Assessment.
Environ. Model. Assess. 3 (3), 181-191.
Tolk, A., Diallo, S.Y., Turnitsa, C.D., Winters, L.S., 2006. Composable M&S Web Services for
Net-centric Applications. J. Def. Model. Simul. 3 (1), 27-44
Tress, G., Tress, B., Fry, G., 2005. Clarifying Integrative Research Concepts in Landscape
Ecology. Landsc. Ecol. 20 (4), 479-493.
van Evert, F.K.; Holzworth, D.; Muetzelfeldt, R.M.; Rizzoli, A.E.; Villa, F. Convergence in
integrated modeling frameworks, in: Zerger, A., Argent, R.M. (Eds). Proceedings of the
MODSIM 2005 International Congress on Modelling and Simulation. Melbourne,
Australia, December, 2005.
van Ittersum, M. K., Ewert, F., Heckelei, T., Wery, J., Olsson, J.A., Andersen, E., Bezlepkina, I.,
Brouwer, F., Donatelli, M., Flichman, G., Olsson, L., Rizzoli, A.E., van der Wal, T.,
Wien, J.E., Wolf, J., 2008. Integrated assessment of agricultural systems – A component-
64
based framework for the European Union (SEAMLESS). Agricultural Systems 96, 150-
165.
Voinov, A., Gaddis, E., 2008. Lessons for Successful Participatory Watershed Modeling: A
Perspective from Modeling Practitioners. Ecological Modelling: 216, p. 197–207.
Voinov A. 2008. Systems Science and Modeling for Ecological Economics. Academic Press,
430 p.
Voinov, A., Bousquet, F., 2010. Modelling with stakeholders. Environ. Model. Software 25 (11),
1268-1281.
Voinov, A., and C. Cerco. 2010. Model integration and the role of data. Environmental
Modelling & Software 25, no. 8: 965-969.
Voinov, A., Shugart, H., 2011. ―Integronsters‖, integral and integrated modeling. Environmental
Modelling and Software (this issue).
Voinov, A., Hood, R.R., Daues, J.D., Assaf, H., Stewart, R., 2008. Building a community
modelling and information sharing culture, in: Jakeman, A.J., Voinov, A., Rizzoli, A.E.,
Chen, S.H. (Eds.) State-of-the-Art and Futures in Environmental Modelling and
Software. Elsevier, New York. pp. 345–365.
Voinov, A., D. Arctur, I. Zaslavskiy, and S. Ali, 2008. Community-based software tools to
support participatory modelling: a vision. In: M. Sànchez-Marrè, J. Béjar, J. Comas, A.
E. Rizzoli, G. Guariso (Eds.) Proceedings of the iEMSs Fourth Biennial Meeting:
International Congress on Environmental Modelling and Software (iEMSs 2008).
Voinov, A., DeLuca, C., Hood, R.R., Peckham, S., Sherwood, C.R., Syvitski, J.P.M., 2010. A
Community Approach to Earth Systems Modeling. EOS, Trans. Am. Geophys. Union 91
(13), 117-124.
Wang, W.G., Tolk, A., Wang, W.P. 2009. The Levels of Conceptual Interoperability Model:
Applying Systems Engineering Principles to Modeling and Simulation, in: Proceedings
of the Spring Simulation Multiconference. Society for Modeling & Simulation
International (SCS), San Diego, CA.
Weyant, J. P., Davidson, O., Dowlatabadi, H., Edmonds, J.A., Grubb, M., Parson, E.A., Richels,
R., Rotmans, J., Shukla, P.R., Tol, R.S.J., Cline, W.R., Fankhauser, S., 1996. Integrated
assessment of climate change: An overview and comparison of approaches and results,
in: Bruce, J.P., Lee, H., Haites, E.F., (Eds.), Climate Change 1995: Economic and Social
Dimensions of Climate Change Contribution of Working Group III to the Second
Assessment Report of the Intergovernmental Panel on Climate Change. Cambridge
University Press, New York City, NY, pp. 368-396.
65
Zagonel, A.A., 2002. Model Conceptualization in Group Model Building: A Review of the
Literature Exploring the Tension Between Representing Reality and Negotiating a Social
Order. Proceedings of the 2002 International Conference of the System Dynamics
Society. Palermo, Italy (July 28 – August 1).