overview journal article

Available online at www.sciencedirect.com

www.elsevier.com/locate/agsy

Agricultural Systems 96 (2008) 150–165

Integrated assessment of agricultural systems – A component-basedframework for the European Union (SEAMLESS)

Martin K. van Ittersum a,*, Frank Ewert a, Thomas Heckelei b, Jacques Wery c,Johanna Alkan Olsson d, Erling Andersen e, Irina Bezlepkina f, Floor Brouwer g,

Marcello Donatelli h, Guillermo Flichman i, Lennart Olsson d, Andrea E. Rizzoli j,Tamme van der Wal k, Jan Erik Wien k, Joost Wolf a

a Plant Production Systems, Wageningen University, P.O. Box 430, 6700 AK Wageningen, The Netherlandsb Food and Resource Economics, University of Bonn, Nussallee 21, D-53115 Bonn, Germany

c UMR System #1123, SupAgro Cirad Inra, 2 Place Viala 34060 Montpellier, Franced Lund University Centre for Sustainability Studies, Lund University Box 117, 221 00 Lund, Sweden

e FLD, Royal Veterinary and Agricultural University (KVL), Hørsholm Kongevej 11, Hørsholm DK-2970, Denmarkf Business Economics, Wageningen University, Hollandseweg 1, 6706 KL Wageningen, The Netherlands

g LEI, Wageningen UR, P.O. Box 29703, 2502 LS The Hague, The Netherlandsh CRA-ISCI, Via di Corticella 133, 40128 Bologna, Italy

i IAMM-CIHEAM, 3191 Route de Mende, 34093 Cedex 5, Montpellier, Francej IDSIA-SUPSI, Via Cantonale, Galleria 2, 6928 Manno, Lugano, Switzerland

k Alterra, Wageningen UR, P.O. Box 47, 6700 AA Wageningen, The Netherlands

Received 10 January 2007; received in revised form 16 July 2007; accepted 17 July 2007Available online 1 October 2007

Abstract

Agricultural systems continuously evolve and are forced to change as a result of a range of global and local driving forces. Agricul-tural technologies and agricultural, environmental and rural development policies are increasingly designed to contribute to the sustain-ability of agricultural systems and to enhance contributions of agricultural systems to sustainable development at large. The effectivenessand efficiency of such policies and technological developments in realizing desired contributions could be greatly enhanced if the qualityof their ex-ante assessments were improved. Four key challenges and requirements to make research tools more useful for integratedassessment in the European Union were defined in interactions between scientists and the European Commission (EC), i.e., overcomingthe gap between micro–macro level analysis, the bias in integrated assessments towards either economic or environmental issues, the poorre-use of models and hindrances in technical linkage of models. Tools for integrated assessment must have multi-scale capabilities andpreferably be generic and flexible such that they can deal with a broad variety of policy questions. At the same time, to be useful forscientists, the framework must facilitate state-of-the-art science both on aspects of the agricultural systems and on integration. This paperpresents the rationale, design and illustration of a component-based framework for agricultural systems (SEAMLESS Integrated Frame-work) to assess, ex-ante, agricultural and agri-environmental policies and technologies across a range of scales, from field–farm to regionand European Union, as well as some global interactions. We have opted for a framework to link individual model and data componentsand a software infrastructure that allows a flexible (re-)use and linkage of components. The paper outlines the software infrastructure,indicators and model and data components. The illustrative example assesses effects of a trade liberalisation proposal on EU’s agricultureand indicates how SEAMLESS addresses the four identified challenges for integrated assessment tools, i.e., linking micro and macroanalysis, assessing economic, environmental, social and institutional indicators, (re-)using standalone model components for field, farmand market analysis and their conceptual and technical linkage.� 2007 Elsevier Ltd. All rights reserved.

0308-521X/$ - see front matter � 2007 Elsevier Ltd. All rights reserved.

doi:10.1016/j.agsy.2007.07.009

* Corresponding author. Tel.: +31 317 482382; fax: +31 317 484892.E-mail address: [email protected] (M.K. van Ittersum).

mailto:[email protected]

M.K. van Ittersum et al. / Agricultural Systems 96 (2008) 150–165 151

Keywords: Bio-economic farm model; Cropping system model; Impact assessment; Indicators; Market model; Sustainable development

1. Introduction

Agricultural systems around the globe continuouslychange as a result of enlarging trade blocks, globalisationand liberalisation, introduction of novel agro-technologies,changing societal demands and climate change. Parallel toliberalisation of markets, the European Union (EU) hasengaged in a political ambition to devise policies that aimto improve sustainability of agricultural systems, i.e., theireconomic viability, environmental soundness and socialacceptability, and to enhance the contribution of agriculturalsystems to sustainable development of society and ecosys-tems at large (EC, 2001, 2005b). Agricultural, environmentaland rural development policies must contribute to theseaims, but in a cost-effective and efficient manner to makethem politically acceptable. Strong interactions between pol-icies and adoption of agro-technologies exist. Assessing thestrengths and weaknesses of new policies and interactionswith agro-technologies, prior to their introduction, i.e.,ex-ante integrated assessment, is vital to devise policies thatpromote sustainable development. The European Commis-sion (EC), for instance, has introduced Impact Assessmentof its policies as an essential step in the development andintroduction of new policies since 2003 (EC, 2005a).

Integrated Assessment has been defined as ‘‘an interdis-ciplinary and participatory process combining, interpretingand communicating knowledge from diverse scientific dis-ciplines to allow a better understanding of complex phe-nomena’’ (Rotmans and van Asselt, 1996). Integratedassessment and modelling (IAM) has been proposed byresearch as a means of enhancing the management of com-plex systems and to improve integrated assessment (Par-son, 1995; Harris, 2002; Parker et al., 2002). It is basedon systems analysis as a way to consider, in a balancedintegration, the biophysical, economic, social and institu-tional aspects of a system under study. The assumptionunderlying IAM is that computerized tools from sciencecontribute to better informed ex-ante integrated assess-ments of new policies and technologies. They certainly donot replace a participatory process in which many otherfactors and knowledge sources play a determining role,but allow safe and relatively cheap experimentation, andquantification of effectiveness and efficiency of differentpolicy alternatives. Agricultural science has a history inusing systems analysis and what may be characterized asintegrative modelling approaches for analyzing bio-eco-nomic problems (Heckelei et al., 2001; Kropff et al.,2001; Van Ittersum and Donatelli, 2003; Arfini, 2005; Ver-burg et al., 2006). However, based on our interactions withpotential users of research tools for integrated assessmentand studying the literature we argue there are several majorchallenges to overcome to make research tools more usefulfor integrated assessment.

First, existing tools, methods and data each cover onlysome of the hierarchical levels needed within an integratedassessment and in particular do not link the micro (field–farm-small region) and macro (market or sector) levels.This is partly a matter of not bridging scales and partly amatter of lack of interdisciplinarity. Policy questions tobe addressed cannot be solved at micro or macro level only,but need cross-scale consideration. Dalgaard et al. (2003)recognized scaling from one hierarchical level to anotheras a key issue in agro-ecology. Hansen and Jones (2000)describe different methods for upscaling, and Ewert et al.(2006a) address the issue of bridging different hierarchiesof scales in natural, economic and social disciplines. Initialattempts to bridge micro and macro analyses focused ondeveloping countries for relatively local markets (Sissoko,1998; Kruseman, 2000). A crucial challenge is to developmulti-scale methods in general and, more specifically,methodologies that allow bridging analyses at micro andmacro scales.

Second, the existing IAM approaches are heavily biasedtowards either the biophysical, economic or social disci-plines, and imbalanced in their degree of quantification.Social aspects pertain to employment, income distribution,quality of life of farmers, gender in farming, etc., and aregenerally not well represented in modelling tools. Further-more, institutional constraints are often entirely lacking inpresent integrated research tools. Institutions are defined asthe formal and informal rules of a society or of organisa-tions (Spangenberg et al., 2002), that can either facilitateor hamper the decision making, subsequent implementa-tion of policies or use of new technologies, and thus, influ-ence the resulting behaviour of targeted actors (e.g.,compliance with regulations, intended behaviouralchanges). In short, current integrated assessment tools arestill restricted in the range of issues they tackle.

Third, many of the existing models and databases arecurrently case specific, restricting their re-use in new prob-lems and their timely availability when new issues arise.Also, their limited re-use is not cost-effective. Althoughthere is an inherent tension between being generic in modelformulation and sufficiently meaningful in applications, wethink we can largely benefit from the concept of compo-nent-based modelling which breaks up larger models in dis-crete and re-usable components (Szyperski et al., 2002;Argent, 2004). This author advocates the use of compo-nent-based modelling (modelling frameworks) in overcom-ing the tension between the need for good science and theneed to be relevant in terms of applications.

The fourth obstacle and challenge is related to all threeprevious challenges and strongly to the third one. Integra-tion of research and the cross-fertilisation of ideas from dif-ferent disciplines are hindered by the variety of formalisms,which is also reflected in the software tools implementing

Pre

Modelling

Post-modelling

Problem definition

Scenariodescription

Indicatordevelopment

Definition ofsimulation experiment

Modelselection andcomposition

Parameterizationand

simulation

Postanalysis

Visualization of results

Documentation/communication

Pre-modelling

Modelling

Post-modelling

Problem definition

Scenariodescription

Indicatorselection

Definition ofsimulation experiment

Modelselection andcomposition

Parameterizationand

simulation

Post-modelanalysis

Visualization of results

Documentation/communication

Dat

a an

d kn

owle

dge

base

Use

rs/s

take

hold

ers

Fig. 1. Integrated assessment procedure using SEAMLESS-IF, with pre-modelling, modelling and post-modelling phase.

152 M.K. van Ittersum et al. / Agricultural Systems 96 (2008) 150–165

the research results. Models are rarely re-usable outside theenvironments for which they were developed. Over the pasttwo decades various research groups have been trying toexploit integrated modelling in supporting inter-disciplin-ary research (Rizzoli et al., 1998). This has resulted in thesuccessful examples of model linkage and modelling frame-works, in water- and environmental-resource managementmodelling (Leavesley et al., 1996), cropping system model-ling (ModCom; Hillyer et al., 2003), hydrological model-ling (TIME; Rahman et al., 2004 and OpenMI; Gijsbersand Gregersen, 2005).

This paper introduces the design and functionality of theSEAMLESS Integrated Framework (SEAMLESS-IF) foran ex-ante, integrated assessment of agro-environmentalpolicies and agro-technological innovations in the Euro-pean Union (EU). SEAMLESS stands for System for Envi-ronmental and Agricultural Modelling; Linking EuropeanScience and Society, and brings together over 100 scientistsfrom a broad range of disciplines and 15 countries. Thedefinition of SEAMLESS-IF was driven by the four chal-lenges described above. SEAMLESS aims at deliveringan integrated framework to underpin integrated assessmentof agricultural systems at multiple scales (from field, farm,region to EU and global), to provide analytical capabilitiesfor environmental, economic, social and institutionalaspects of agricultural systems and to develop a compo-nent-based system that allows re-use for new problems,while using a software infrastructure that facilitates there-use and linkage of the components. A vision of the finalversion of SEAMLESS-IF, due early in 2009, is presentedin this paper and illustrated with an example from its firstand second working prototypes which are available onwww.seamless-ip.org (Ewert et al., 2006b). We realize thatthe contents and design of research tools are just one deter-mining factor in the usefulness and uptake of such tools inintegrated assessment; the process of user engagement andparticipatory development is probably equally important(McIntosh et al., in preparation). Although the four issuesmentioned above are a result of an interactive process ofuser interaction in relation to impact assessment in theEC, the process of user engagement and participatorydevelopment is not a specific subject of this paper.

2. SEAMLESS integrated framework: domain, approach

and users

2.1. System definition

SEAMLESS focuses on the land-bound agriculturalactivities (arable cropping, grasslands, livestock, perenni-als, including orchards, agro-forestry and vineyards) andtheir interactions with the environment, economy and ruraldevelopment. SEAMLESS-IF aims at providing analyticalcapacity to assess sustainability of agricultural systems inthe European Union and contributions of the EU’s agricul-tural systems to sustainable development at large, includingsome effects on the entire production chain (transport, pro-

cessing and packing) and other land uses (e.g., nature). InSEAMLESS, we have conceptualized the system by distin-guishing actors (e.g., farmers, policy makers) taking actionswhich have an effect on the environment (in the broadestsense, i.e., biophysical, economic or social), which resultsin certain conditions that in turn may affect the actors.Impacts on the institutional environment are not consid-ered but the constraints that institutions may pose onactors and decisions are included in the assessment (seeSection 4.2). As such SEAMLESS embodies interdisciplin-ary, integrated assessment and modelling.

SEAMLESS-IF facilitates assessing proposed policyoptions by comparing a baseline scenario capturing theautonomous trends and already accepted policies with pol-icy scenarios which differ from the baseline in the proposedpolicies. The scenarios are assessed through a set of indica-tors that capture the key economic, environmental, socialand institutional issues of the questions at stake (Fig. 1).The indicators in turn are assessed using outputs fromquantitative model components, typologies and databases.In the final steps (Fig. 1) the indicator values are visualized,aggregated or weighted in an interactive process with usersand stakeholders.

The framework is designed to compare policy alterna-tives, in interaction with agro-technological options, for adefined time horizon; these time horizons are defined bythe policy questions at stake. The models used withinSEAMLESS-IF (Section 3) have some flexibility in termsof time horizons, although the economic market model iscurrently designed to handle questions with a time horizonof up to 10–15 years from now.

Spatially, the framework allows analysis at EU-25 level(and wherever data are available also for the two newmember states Bulgaria and Romania, and for Norway,Switzerland, Croatia, Bosnia-Herzegovina, FYR Macedo-nia, Turkey and Albania), at member state level andadministrative units (so-called NUTS-2 regions which,approximately, match provinces within the member states;NUTS stands for Nomenclature of Territorial Units forStatistics in the European Union). Within the NUTS-2

http://www.seamless-ip.org


regions agri-environmental zones and farm types are distin-guished. At the other end of the spectrum SEAMLESS-IFaims at the assessment of some interactions between theEU and the rest of the world in terms of (agricultural)economy and trade flows.

Applicability of SEAMLESS-IF is evaluated andimproved in two real-world Policy Cases. Policy Case 1 isdriven by global economic policy changes, analysing theimpact of further trade liberalisation as currently negoti-ated in the Doha round of the World Trade Organization.Policy Case 2 analyses what would happen if the EU coun-tries, regions and farmers implement instruments to com-ply with the EU directives on water and nitrate. Theimpacts will be assessed with the economic, social and envi-ronmental indicators at the various levels represented inSEAMLESS-IF. Specific attention will be paid to the inter-action between the various agro-technologies and land uses(such as integrated crop management, conservation agri-culture and agro-forestry) and the existence and degree ofspecific policy incentives to use these technologies.

2.2. Component-based modelling in SEAMLESS-IF

Inspired by the approach of modular modelling by Zeig-ler (1987) and by recent works in component-oriented soft-ware engineering (Szyperski et al., 2002) we have opted foran Integrated framework supporting integrated assessmentin which Individual (stand-alone) knowledge componentscan be linked through a software Infrastructure, allowingthe use of selected and linked components to underpin inte-grated assessments. We have named this the Triple I concept.

The individual components can be either models repre-senting different processes at specific hierarchical levels, dat-abases, or indicators derived from model outputs and/ordata. The software infrastructure of SEAMLESS-IF allowsthe seamless linkage of selected components into modelchains that assess certain indicators. These componentshave value in their own right and can be either, existingor newly developed models and databases, focusing, forinstance, on crop growth and externalities, farm responsesor market simulation. The models have been designed tosimulate aspects and processes of agricultural systems atspecific levels of organisation, i.e., point or field scale, farm,region, EU and world. Linking models designed for differ-ent scales and pertaining to different domains is the essentialtrait of integrated modelling. Such a task requires cross-dis-ciplinary expertise, which is rarely embodied in a singleresearcher, but it is in the union of the experiences andknowledge of teams of researchers. The software architec-ture of SEAMLESS-IF allows the re-use of model compo-nents for scale and disciplinary integration and theirinclusion in model chains for the computation of indicators.

2.3. Users of SEAMLESS-IF

SEAMLESS-IF is being designed and developed as aresponse to a research call from the Directorate General

(DG) Research of the European Commission. Frequentinteractions with DG Research shaped the requirementsand design of the framework (see Section 1 of this paper).Since the start of the project, interactions with various DGsof the EC have been initiated to engage with foreseen usersof the framework, in particular DG Agriculture, DG Envi-ronment and DG Economics and Finances. User Forummeetings are organized twice a year and smaller meetingsin between upon request.

We originally distinguished six classes of users in theanalysis of requirements for SEAMLESS-IF, i.e., coders,linkers, runners, providers, viewers and players, with dis-tinct requirements. Each of these classes still determinedevelopment of the framework but for the development ofgraphical user interfaces they have been merged into threegroups of users, i.e., the policy expert, the integrative mod-eller and the domain specific modeller. The policy expertsare engaged in the pre-modelling and post-modelling phase(Fig. 1) to define scenarios and indicators and are mainlyinterested in the results of the impact assessment. The inte-grative modellers are those foreseen to set-up integratedassessments with SEAMLESS-IF, implementing and run-ning model chains which involve model components per-taining to different domains and spanning different scales.They work in tight interaction with the policy expert, to pre-pare and assess the scenarios which will be studied duringan impact assessment procedure. The domain specific mod-ellers are the experts of the individual components and theircode or data; they are not being served through a specificgraphical user interface.

3. Detailed description of SEAMLESS-IF

This section presents the major components of SEAM-LESS-IF, the quantitative models, the database and theindicators and the software infrastructure that allows the(re-)use and linkage of these components in applicationspertaining to different impact assessments.

3.1. SEAMLESS-IF software infrastructure

The SEAMLESS-IF software infrastructure is a resultof analyses of technical and user requirements within theproject and review of software frameworks which areaimed at supporting a component-oriented approach tomodelling (Szyperski et al., 2002; Argent, 2004). A numberof environmental modelling frameworks allowingimproved model development and deployment and modelcomponent linking have inspired the software architectureand design of SEAMLESS-IF (Argent, 2004; Argent andRizzoli, 2004; Argent et al., 2006). The main philosophyis to allow for (re-)using and linking a variety of availableknowledge components, such as models, databases, expertrules and analysis tools while facilitating linking, transpar-ency and documentation through using semantically richmeta-information (Van der Wal et al., 2005). These


semantically annotated components might further evolve,while they can be used in the integrated framework at thesame time. The components follow a strict separation ofdata, model and graphical user interfaces.

The main component in the SEAMLESS-IF softwareinfrastructure is SeamFrame, the core component that runson a server and provides the services that can be used bythe several SEAMLESS client components/applications.The main components of SeamFrame are: the modellingenvironment, project manager, processing environmentand the domain manager (Fig. 2). The SeamFrame serverinteracts with the SEAMLESS database and knowledgebase.

The SEAMLESS ontology plays a central role inSEAMLESS-IF to harmonize and relate the different con-cepts from models, indicators, source data, etc. Seam-Frame uses ontology to structure domain knowledge andsemantic meta-information about components in order tofacilitate organisation, retrieval and linkage of knowledgein components. To guarantee consistency between thedatabase and the ontology, the domain manager generatesthe relational database schemas from the ontology. Theontologies and their content are stored in a so-calledknowledge base (Rizzoli et al., 2007; Villa and Athanasia-dis, 2006; Villa et al., 2006). The use of ontologies tosemantically annotate the component models allows,among other things, for checking the match betweensources in terms of linking the proper output variables ofa component to the input variables of a second component.

Modelling environments assist users (domain specificmodellers) to develop and edit their executable modelsand datasets with as current choices MODCOM (Hillyeret al., 2003) for the biophysical models and GAMS(Brooke et al., 2006) for farm economic and market models(Section 3.2.2). Modellers can use a model development

For linking the models,the model componentsimplement an OpenMI

based interface

<<device>>

SEAMLESS client<<component>>

SEAMLESS-IFGUI

<<device>>

SEAMLESS client

<<component>>

Seam:PRES

<<device>>

SEAMLESS client

<<component>>

Future application

<<devi

SeamFram

<<component>>

Domain manager

<<component>>

Processingenvironment

<<component>>

Project manager

in

<<device>>

SEAMLESS client<<co

<<device>>

SEAMLESS client

mp t>>

<<device>>

SEAMLESS client

<<component>>

Future application

<<de

SEAMLESS client

<<comp en

Future application

<<devi

<<component>><<component>>



Fig. 2. SEAMLESS-IF software infrastructure, with its S

tool to deliver models which will be wrapped up as compo-nents thanks to specific wrappers in SeamFrame. All modelcomponents implement an interface based on the standardOpen Modelling Interface (OpenMI; www.openmi.org;Gijsbers and Gregersen, 2005). OpenMI provides a stan-dardized interface to define, describe and transfer databetween software components that run simultaneously orsubsequently. The standard is extended to meet the require-ments of SEAMLESS (Gijsbers et al., 2006). The modelcomponents can then be arranged in model chains bydomain specific modellers and integrative modellers.

The project manager assist the user in the configurationof the integrated assessment problems: the user is guided inthe definition of the problem description, the selection ofthe indicators and the model chains used to compute them.The project manager also allows to set the model parame-ters, define which data sets are to be used as inputs and tochoose among alternative policy options to be tested andevaluated.

The processing environment orchestrates the executionof the experiments associated with an integrated assess-ment problem: it launches simulations and optimisationswithin the model components. It is envisaged that in futurereleases the processing environment will allow to performsensitivity analyses covering the whole chain of models.

On the client side we find a set of rich internet applica-tions (Tidwell, 2006) that have been designed to facilitatethe interaction with the user. At present, there are thegraphical user interface of the Project Manager (SEAM-LESS-IF GUI) that assists the user in the formulation ofa project to perform the integrated assessment of alterna-tive policy options, and the graphical user interface of theresult browser (Seam:PRES) which enables the user to eas-ily access, display, and compare the results of the policyassessments.

<<device>>

SEAMLESSDatabase

Model components can be based on the SEAMLESS modelling environment(e.g. APES is built inthe MODCOM system;SEAMCAP is built inGAMS)

<<device>>

SEAMLESSOntology

Relational schema

ce>>

e server<<component>>

APES

<<component>>

FSSIM-AM

<<component>>

FSSIM-MP

<<component>>

EXPAMOD

<<component>>

e

ng

in

<<device>>

Relational schema

ce>>

<<component>>

<<component>>

<<component>>

FSSIM-MP

<<component>>

<<component>>

EXPAMOD

<<component>>

<<component>>

SEAMCAP<<component>>

eamFrame server and end-user (client) applications.

http://www.openmi.org

Market model


Finally, building on the infrastructure of this frameworka number of SEAMLESS end-user applications (e.g.,APES, FSSIM-AM, FSSIM-MP, SEAMCAP – see below)are delivered.

3.2. Individual knowledge components

3.2.1. Indicators and indicator frameworks

Baseline and policy scenarios are assessed and comparedin SEAMLESS-IF through a set of indicators. So-calledindicator frameworks are developed to structure a broadrange of indicators and to facilitate their interactive selec-tion by the policy experts or other stakeholders. The indi-cators form a basis for the translation of the users’ policyquestions into something that SEAMLESS-IF can assess.Indicator frameworks will also facilitate a balanced selec-tion of indicators across the dimensions of sustainabledevelopment (economic, social and environmental), as wellas their meaningful combination. And finally they create asystematic and flexible approach for weighing and aggre-gating indicators if desired by users. Discriminating classesof the initial indicator framework used in SEAMLESShave been listed in Table 1. Indicators are assessed eitherthrough output variables from quantitative model compo-nents (mainly environmental and economic indicators), ordirectly through data or qualitative interpretations ofproxy metrics and (trends in) primary data (social andinstitutional indicators).

Table 1Discriminating classes and variants of these classes of the goal-orientedindicator framework (GOF) in SEAMLESS-IF

Class Variants

Dimensions ofsustainability

Environmental, economic, social

Levels oforganisation

Field, farm, region, EU-25, globe

Domains Sustainability of agricultural systems, contributionof agricultural to sustainable development at large

Themes Goals (e.g., achieving a certain quality of water orlife), Process of achievement (e.g., nitrate leachingor gross margin of farms)Means (e.g., land cover or productive capital)

Table 2Examples of indicators in Prototypes 1 and 2, the dimension of sustainabilitymodel is listed that is used to assess the indicator of SEAMLESS-IF

Indicator (Unit) Dimension

Gross agricultural income (€) EconomicBudgetary costs of CAP, first pillar (€) EconomicProducer and consumer prices of commodities (€) EconomicProduction of main agricultural commodities (ha) or (€) EconomicFarm income (€ per ha) EconomicAgricultural employment (agricultural working units) SocialSoil organic matter (%) EnvironmenNitrate leaching (kg/ha) EnvironmenCrop diversity (number of crops) Environmen

Table 2 provides an example of indicators which can beassessed with the Prototype 2 models.

3.2.2. Model chain and stand-alone model components

SEAMLESS-IF contains the following models (i.e., thebackbone models – Fig. 3): the agricultural sector modelSEAMCAP that simulates supply–demand relationshipsin the EU-25 for agricultural commodities; SEAMCAPderives information on price–supply relationships fromfarm system models (FSSIM) through an econometricmeta-model (EXPAMOD). The farm system models inturn simulate farm behaviour and use agricultural activities(e.g., crop rotations) assessed through a mechanistic simu-lation model for agricultural production and externalities(APES). Two predefined model chains are distinguished,i.e., APES-FSSIM, the linkage between a biophysical sim-ulation model and a bio-economic farm model, andFSSIM-EXPAMOD-SEAMCAP, the linkage of a bio-eco-nomic farm model through an econometric procedure to amarket model.

3.2.2.1. Agricultural production and externalities simulator(APES). APES is a modular, deterministic simulationmodel targeted at estimating the biophysical behaviour ofagricultural production systems in response to weather,

and the level of organisation to which they refer; in the final column the

Level of Organisation Model

EU, nation, NUTS-2 SEAMCAPEU, nation, NUTS-2 SEAMCAPEU, nation, NUTS-2 SEAMCAPEU, nation, NUTS-2, farm type SEAMCAP, FSSIMFarm type, NUTS-2 FSSIMEU, nation, NUTS-2, farm type SEAMCAP, FSSIM

tal Farm type, NUTS-2 FSSIM, APEStal Farm type, NUTS-2 FSSIM, APEStal Farm type FSSIM

FSSIM-MP

FSSIM-AM

APES

EXPAMOD

Production technology+ externalities

Farm response models

Extrapolation + Aggregation:Suppy elasticities

Agricultural commodities

Indicators

FSSIM-MP

SEAMCAP

EXPAMOD

Fig. 3. Backbone model chain of SEAMLESS-IF for field, farm andmarket level analysis, from the bottom to the top of the figure,respectively. See text for explanation of the various model componentsand their linkages.


soils and agro-management options. Using mostly model-ling approaches already made available by research andpreviously tested in other simulation tools (cf. Van Itter-sum and Donatelli, 2003), APES runs at a daily time-stepin the communication among components and simulatesone-dimensional fluxes at field scale. The simulationapproaches usually embody process knowledge about bio-physical relationships. APES computes the yields, bothaverages and variability across years, as well as inputs suchas irrigation water and externalities of crop rotations.APES itself consists of several components representingland uses (crops, grassland, vineyards, orchards andagro-forestry), soil water, carbon and nitrogen, soil ero-sion, pesticide fate, management activities (Donatelliet al., 2006c) and a component used to generate/estimatesynthetic weather (Donatelli et al., 2005, 2006a,b; Carliniet al., 2006) (Fig. 4). APES has been designed to allow fur-ther extension of this list of components if required. Thecriteria to select modelling components within APES arebased on the need of: (1) accounting for specific processesto simulate soil–land use interactions; (2) input data torun simulations, which may be a constraint at EU scale;(3) simulation of agricultural production activities of inter-est and (4) simulation of alternative management optionsand their impact on the system.

3.2.2.2. Farm system simulator (FSSIM). FSSIM is a bio-economic farm model (Janssen and van Ittersum, 2007)developed to quantify the integrated agricultural, environ-mental, economic and policy aspects of farming systems(Louhichi et al., 2006). FSSIM is developed to assessthe response of the major farm types (as defined througha typology (see Section 3.2.3)) across the EU in responseto policies and agro-technological development. FSSIMincludes a data module for agricultural management,FSSIM-AM, which computes the technical coefficientsand costs for ranges of current and alternative agricul-tural activities, and FSSIM-MP, the mathematical pro-

Soil-Water

Pesticides

C-Nitrogen

Weather

Agro-fores

Cr

Gras

Orchard/ vineyaAgriculturalmanagement

Simulation

engine

APES

Fig. 4. The agricultural production and externa

gramming part aims to capture resource, socio-economicand policy constraints and the farmer’s major objectives(Thompson, 1982; Deybe and Flichman, 1991; Wossinket al., 1992). FSSIM-MP is a comparative, static mathe-matical programming model with a non-linear objectivefunction representing expected income and risk aversiontowards price and yield variations. It assumes that pricesare exogenous to the farmer and these prices are providedthrough simulation by the SEAMCAP agricultural sectormodel.

The agricultural activities assessed in FSSIM-AM repre-sent the different ways in which farmers can grow arablecrops (crop rotations), livestock and perennials (grassland,orchards, vineyards) in terms of their inputs (nutrients,labour, etc.) yields, costs and externalities. The productiontechnology of each single activity is of the Leontief-type(Leontief, 1986), i.e., characterized by fixed input and out-put coefficients. However, the combination of differentcrop rotations defines non-linear, multi-output, multi-inputtechnologies (Hazell and Norton, 1986). Current activitiesare largely based on surveys and databases that monitorthe agricultural activities of sampled farms across theEU-25. Alternative activities are systematically generatedthrough a set of tools within FSSIM-AM using agronomicknowledge rules (Dogliotti et al., 2003; Hengsdijk and vanIttersum, 2003). FSSIM-AM can be linked to APES toassess productivity and externalities.

3.2.2.3. Agricultural sector model (SEAMCAP). SEAM-CAP is a version of CAPRI (Common Agricultural PolicyRegionalized Impact) integrated in SEAMLESS-IF, i.e.,an agricultural sector model of the EU (Heckelei andBritz, 2001; Britz et al., 2006). It is a comparative staticequilibrium model, solved by iterating supply and marketmodules. The supply side of SEAMCAP consists of non-linear programming models at NUTS-2 level, allowingdirect implementation of most policy measures withhighly differentiated sets of agricultural activities.

try

ops

ses

rd a production enterprise component

a soil component

lities simulator (APES) and its components.

AM

Biodiversity

Agriculturalemployment

GTAP

Developingcountry models

FSSIM-MP

FSSIM-

APES

Visual landscape attributes

Environmentalattributes

Global model

Territorial

SEAMCAP

EXPAMOD

Fig. 5. An overview of all quantitative models in SEAMLESS-IF.Compared with Fig. 3 (backbone models) four types of models have beenadded: territorial models, agricultural employment assessment, GTAP(global market model) and models for developing countries.


Allocation is based on profit maximising behaviour andcalibrated multi-product cost functions. SEAMCAP alsoprovides nutrient balances and gas emissions with globalwarming potentials using a matrix of coefficients linkedto the levels of the activities. Prices are exogenous inthe supply module and are provided by the market mod-ule of SEAMCAP that searches, in an iterative procedure,for the set of prices that equilibrate supply and demandon EU and international markets for all considered agri-cultural outputs.

The regional supply modules (ca. 300 regions in the EU-25) represent activities of all farmers at farm type level cap-tured by the Economic Accounts for Agriculture (EAA).The programming models are a kind of hybrid approach,as they combine a Leontief-technology for variable costscovering a low and high yield variant for the different pro-duction activities with a non-linear cost function (positivemathematical programming) which captures otherwisenon-accounted cost effects such as limited labour and cap-ital capacities, aggregation errors and multi-output interac-tions (rotation effects). The non-linear cost function allowsfor perfect calibration of the models to base year levels anda smooth simulation response to changing prices and poli-cies as can be typically observed at the aggregate level. Themodels capture in high detail the premiums paid under theCommon Agricultural Policy (CAP), set-aside obligationsand milk quotas.

The SEAMCAP market module endogenously adjustsEU- and international prices to achieve market equilib-rium. It also allows the assessment of the impact of a largeset of bi- and multilateral trade policy instruments. Thesub-module for marketable agricultural outputs is a spa-tial, non-stochastic global multi-commodity model forabout 40 primary and processed agricultural products, cov-ering about 40 countries or country blocks in 18 tradingblocks. Bilateral trade flows and attached prices are mod-elled based on the Armington assumption (Armington,1969). The behavioural functions for supply, feed, process-ing and human consumption apply flexible functionalforms where calibration algorithms ensure full compliancewith micro-economic theory including curvature. Theparameters are synthetic, i.e., to a large extent taken fromthe literature and other modelling systems. This sub-mod-ule delivers prices used in the supply module and allowsfor market analysis at global, EU and national scale,including a welfare analysis.

3.2.2.4. Model components anticipated to be integrated in

later versions. Future versions of SEAMLESS-IF are antic-ipated to include further model components currentlyunder development, as also illustrated in Fig. 5. First, ter-ritorial modelling capabilities are developed that enableassessment of environmental and biodiversity indicators,as well as visual quality of the landscape at regional andlandscape levels. These components make use of resultsof the farm type modelling and the spatially allocated farmtypes in NUTS-2 regions. Second, an agricultural labour

input estimation procedure is added. This econometricapproach allows post-model analysis of labour effects fordifferent production structures as given by the SEAMCAPmodel. Also, a structural change (of farm types) model isdeveloped (Zimmermann et al., 2006). Third, GTAP willbe linked to SEAMCAP. GTAP is an existing global trademodel (computable general equilibrium model) includingglobal databases of production and trade, to analyse theinteractions between EU policies and the rest of the world(Hertel, 1997; Van Tongeren et al., 2001). Finally, develop-ing country models are developed, i.e., a computerized gen-eral equilibrium model at national level linked to farmhousehold models (FSSIM) to allow for assessing effectsof EU policies on agricultural production, environmentalimpacts, poverty and rural development in developingcountries.

3.2.3. Data and typologies

SEAMLESS provides a pan-European integrated data-base, including comprehensive datasets on biophysicalvariables (climate, soils, land use, topography), farmingand farm management, crops and livestock, socio-eco-nomic aspects (prices, employment, production data,trade flows, income, etc.) and policies (international andEC policies, as well as national and regional policies).Important sources are the European soil map, climatedata from the MARS (Monitoring Agriculture withRemote Sensing) database, Farm Accountancy Data Net-work (FADN), Eurostat and the GTAP databases (Her-tel, 1997).

It is a major objective of the project to include data thatcan be distributed freely. This means that it is not alwayspossible to include the original datasets in SEAMLESS-IF due to property rights and, in some cases, disclosurerules. For spatial data, it is therefore in some cases neces-sary to transform the original data from vector data to griddata or to a lower spatial resolution. For thematic data, itis likewise possible to distribute some data only in aggre-gated format, for example, by building typologies of farmsrather than distributing single farm data.


Another major objective is to link the different types ofdata, for example, data from farm statistics, data on thebiophysical environment and socio-economic data. This isdone by making all data spatially explicit for a spatialframework combining administrative borders and biophys-ical characteristics to regions with assumed homogenousbiophysical conditions for crop production. The frame-work starts with the so-called NUTS-2 regions that aredivided into climate zones (2.2 per region on average), withhomogenous temperature, rainfall, etc. The climate zonesagain are divided into agri-environmental zones (5.9 perregion on average), with homogenous soil conditions suchas texture and depth of soil. Many (e.g., socio-economic)data are of course not available at the level of agri-environ-mental zones, hence data available only at NUTS-2 regionare assumed homogenous for all biophysical units withinthat region. A specific achievement and exception in rela-tion to this is the development of a statistical procedureto allocate the information from farm economic statistics(i.e., FADN) available for administrative regions to bio-physical units with similar conditions for crop production(Elbersen et al., 2006). This allows the mapping of farmmodel (FSSIM) results in a region and hence the upscalingof economic and environmental results at farm level to aregional level.

The variety of environments, conditions and farmingsystems across the EU-25 is enormous. Typologies assistin simplifying this vast amount of information into coher-ent groups that share the same characteristics. In SEAM-LESS typologies have been developed for (1) farmingsystems (based on farm size, intensity and specialisation/land use – Andersen et al., 2007); (2) agro-environments(based on environmental (climate) zones, soil quality (top-soil organic carbon) and suitability for agriculture (incl.slope) – Hazeu et al., 2006), and (3) socio-economic regions(based on criteria such as population density, employmentand income).

Typologies assist the sampling of data or regions con-sidered in a specific study. For instance, the bio-economicfarm models require a considerable amount of data thatis not available consistently across all regions in Europe.Therefore, a number of sample regions were identifiedthat represent different types of biophysical conditionsand the variety of farm types across the EU-25. Thedeveloped typologies can also be used to provide a spatialcontext for the assessment of indicators. The assessmentof indicators can then take into account the heterogeneityin biophysical, economic and social conditions across theEU-25. Such an approach can, for example, be used toidentify hot spot areas where changes in specific indica-tors coincide with a specific vulnerability to thesechanges, e.g., a rise in livestock density in areas sensitiveto leaching.

The datasets are annotated by metadata (ISO-stan-dards) and by ontologies defined by the modellers. Thisenables future users to get interactively insight in the qual-ity of the stored data and to identify their appropriate use.

4. Illustration of the use of SEAMLESS-IF through a policy

test case

In this section, we illustrate how a specified integratedassessment problem can be implemented in SEAMLESS-IF (Fig. 1) and the type of indicators the framework is ableto generate. Prototype versions of the model componentspresented in Fig. 3 are available, but data availabilityand model parameterisation do not yet allow applicationof APES, FSSIM and EXPAMOD to an adequate sampleof regions and farm types to test and demonstrate the fullfunctionality in terms of policy-relevant results. Instead wedemonstrate results of SEAMCAP and FSSIM (Section4.1) and explain how a fully operational model chain wouldhandle the four challenges presented in the introduction ofthe paper (Section 4.2). The example is simple as comparedto more complex assessments combining trade liberalisa-tion and environmental policies and technological changesthat SEAMLESS-IF is designed to address in its finalversion.

4.1. The policy case and initial results

The policy test case is the integrated assessment of atrade liberalisation proposal by the so-called G20 groupof developing countries at the current Doha Round ofthe World Trade Organisation (G20, 2005). In the pre-modelling phase the policy case is defined in terms of timehorizon, exogenous and endogenous driving forces thatdefine the scenarios, spatial and temporal dimension andindicators (Belhouchette et al., 2007). The pre-modellingphase (Fig. 1) is completed in close interaction with the pol-icy experts. We take the year 2013 as time horizon of theassessment in order to remain consistent with the latestavailable EU-outlook on agricultural markets (EC, 2006)which is used to parameterize the scenarios. The baselinescenario for 2013 is interpreted as a projection in time cov-ering the most probable future development of the Euro-pean agricultural policy, with the Luxemburg Agreementson Common Agricultural Policy Reform as the core, andincluding all future changes already foreseen in the currentdomestic, EU and international legislation (e.g., sugar mar-ket reform). The aim is to provide a baseline that is used asa reference point for counterfactual analysis. The baselinescenario should capture the complex interrelations betweentechnological, structural and preference changes related toagricultural production and commodities world-wide incombination with changes in policies, population andnon-agricultural markets. The policy scenario differs fromthe baseline scenario only in terms of implementing theG20 proposal on the reduction of tariffs for agriculturalproducts and the additional abolition of subsidised exportsby the EU. In the final steps of the pre-modelling phase, thespatial scale of the assessment problem is defined, in thiscase the EU with a spatial resolution of NUTS-2 regions,and a set of indicators at the relevant scales. The MidiPyrenees region is used as an example of a NUTS-2 region


represented by three farm types to explore impacts at farmlevel. The policy case is illustrated with some economicindicators (product prices, meat production, farm incomeand premiums) and environmental indicators (nitrateleaching and soil organic matter).

In the modelling-phase the scenario parameters areimplemented in the models. For our example, based onavailable projections of exogenous driving forces, e.g.,growth of GDP and demographic changes, and extrapola-tion of existing trends SEAMCAP is calibrated to theEU-outlook on production quantities by adjusting behav-ioural parameters to obtain the baseline results (Britzet al., 2006). The baseline of the farm model FSSIM usesconsistent (with SEAMCAP) changes of some exogenousvariables (e.g., technological change and farm premiums)relative to the base year 2001 for which FSSIM andSEAMCAP were calibrated. Alternative agriculturalactivities and management options (e.g., conservationagriculture and agro-forestry) can be assessed at farmlevel, but in this example agricultural activities arerestricted to crop rotations and agro-management cur-rently observed in the region for the three farm types.For this specific assessment problem, the policy scenariois implemented in SEAMCAP only as the proposed policyenters the agricultural system at the market level. Theapplication of SEAMCAP under the policy scenario pro-vides changes of agricultural market variables such asprices and corresponding production and consumptionsquantities. Price changes relative to the baseline are animportant impact of the trade liberalisation scenario con-sidered here and subsequently assessed at farm level usingFSSIM. Fig. 6 illustrates the relative EU producer pricechanges for major plant and animal product groups (Ade-nauer and Heckelei, unpublished).

In general, trade liberalisation of agricultural productsimpacts differently on different commodities. Prices ofproducts where the original degree of protection is rela-

0.6

0.7

0.8

0.9

1Cereals

Oilseeds

Other arable field crops

Beef

Pork meat

Poultry meat

Baseline Scenario

PolicyScenario

Fig. 6. Relative producer price changes of major agricultural productgroups at EU level. Comparison of G20 proposal (policy scenario) withthe baseline for 2013.

tively small (cereals, oilseeds or pork meat), do notdecrease much, whereas highly protected products like beefand dairy show larger price reductions. Consequently, wesimulate an overall significant reduction of productionquantities of beef. However, the decrease will be differenti-ated by region due to variations in the development ofprofitability of products competing for limited resourcessuch as land. Fig. 7 shows the relative reduction of regionalbeef meat supply in the EU-25 NUTS-2 regions and theBalkan states. Overall, results per commodity are rathersmall in relative terms. However, aggregation across com-modities reveals a considerable impact of the G20 pro-posal, as agricultural income is reduced significantly dueto decreasing producer prices for almost all commodities(data not shown).

Next, FSSIM simulates consequences of the pricechanges due to the liberalisation proposal, in terms of thesupply of commodities at farm level, as well as the associ-ated production plans (crop and livestock systems), inputuse and a range of externalities including nitrogen surplusand emissions, pesticide use and impact and irrigationwater use. The alternative agricultural activities have beenderived from the agricultural management module ofFSSIM that can be linked to APES to simulate agriculturalproduction and externalities. As APES has not beenparameterized yet for all crops, it has been replaced in thisexample by CropSyst (Stockle et al., 2003) previouslyparameterized for the Midi-Pyrenees region (Belhouchetteet al., unpublished). Table 3 provides an example of a sim-ulation for the three farm types of Midi-Pyrenees, showinghow each farm type responds to the policy and baseline sce-nario, in comparison with the base year (Louhichi andBelhouchette, unpublished). Compared to the base year2001 farm type 1 showed minor changes in crop productionplans in 2013, while the other farm types reacted byincreasing fallow at the expense of irrigated crops (e.g.,maize) especially on shallow soils. Specific changes in areaper crop and agro-management (irrigation or not) are con-sistent with the changes of premiums per crop type between2001 and 2013 and with the resource endowments of eachfarm type. In the baseline scenario, for all farm types, thefarm income decreased with ca. 20% in 2013, comparedto 2001, mainly because of reduction of premiums. Theimpact on nitrate leaching (average at farm level weighedby area per crop) is marginal in absolute terms, as wasthe effect on soil organic matter. The policy scenario usedin this example had no significant effect on the arable farmactivities (crop rotations and agro-management per soiltype), in comparison with the baseline scenario. Conse-quently the assessment indicators were hardly affected,apart from a slight reduction of the farm income for allfarm types (Table 3) This is due to the absence of furtherchange in premiums in the policy scenario, compared tothe CAP reform already included in the baseline scenario,and due to the limited impact of the policy proposal(G20) on the price of the major arable products as simu-lated by the SEAMCAP model (Fig. 6). Table 4 shows

Fig. 7. Relative change (%) of beef production across EU-25 regions as a result of the policy scenario (compared to the baseline scenario).


the aggregated response of all three farm types at Midi-Pyrenees level, illustrating the type of indicators that canbe assessed at NUTS-2 level.

4.2. SEAMLESS-IF and four identified challenges of

integrated assessment

The presented results have been derived with stand-alone versions of SEAMCAP and FSSIM. We will nowdescribe how a fully operational SEAMLESS-IF allowstacking the four challenges of integrated assessment asintroduced in the first section of the paper. First, SEAM-LESS-IF allows a consistent multi-scale analysis. The prin-cipal idea of the approach is to use the detailed informationfrom farm simulations (FSSIM) to assess price–supplyrelationships which are used in the agricultural sectormodel (SEAMCAP). This is done by making the regionalsupply modules of CAPRI behave like the aggregate ofthe FSSIM models of the same region. Given the ca. 300regions and many farm types (3–10 per region) within the

EU-25 it is virtually impossible to run FSSIM for all farmtypes across the EU-25. Instead, we propose to run FSSIMonly for a sample of 23 out of the 300 regions. EXPAMODallows for extrapolation and aggregation of resultsobtained from FSSIM models from a sample of regionsto all EU regions in SEAMCAP. Thus, the sampled regionsmust adequately represent the diversity of farm types, soiland climate differences across the entire EU-25. The meth-odology envisaged to map the supply behaviour of farmmodels (FSSIM) to the market model (SEAMCAP) com-prises the following sequence of steps implemented in aneconometric meta-model (EXPAMOD): (1) simulation ofFSSIM supply response to price variations to obtainprice-quantity data sets; (2) estimation of supply responsesas functions of price variations, farm characteristics, regio-nal soil and climate conditions, which are available for theEU; (3) extrapolation of supply response to other farmtypes and NUTS-2 regions; (4) aggregation of supplyresponse to the level of SEAMCAP regions (administrativeunits) and SEAMCAP product categories; (5) calibration

Table 3Impact of WTO G20 proposal (policy scenario) on cropping systems and some economic and environmental aspects of the farm types of the Midi Pyreneesregion simulated by the FSSIM model (Louhichi and Belhouchette, unpublished)

Impacts Farm Type1 (2330 farms) Farm Type 2 (990 farms) Farm Type 3 (1376 farms)

Base year[2001]

Baselinescenario [2013](%)

Policy scenario[2013] (%)

Base year[2001]



Base year[2001]



Economic indicators

Farm income(€/ha)

674 �20 �25 751 �15 �19 625 �21 �24

Premiums(k€/farm)

41.7 �14 �14 35.1 �20 �20 51.8 �22 �22

Environmental indicators

Nitrateleaching(kg/ha)

8.1a 9 11 8.8 13 15 8.0 �10 �11

Soil organicmatter (%)

2.1 �1 �2 2.0 �2 �2 1.9 2 2

Cropping system

Maize (ha) 35.1 �10 �13 27.1 �40 �44 6.2 �63 �67Sunflower

(ha)14.3 4 6 12.6 �29 �30 34.0 �10 �9

Soybean (ha) 3.0 4 6 3.7 �50 �51 7.8 �70 �74Durum wheat

(ha)17.3 �4 �3 11.4 �31 �32 31.6 �28 �28

Soft wheat(ha)

13.1 10 12 12.3 �28 �31 13.1 �2 �5

Other crops(ha)b

19.8 8 10 15.5 �1 �3 19.1 �4 �5

Fallow (ha) 11.4 6 4 18.9 124 133 11.5 196 205

For the scenarios the impacts are given as the relative deviation of the indicator in 2013 compared to 2001.a Amount of nitrate leached between sowing and harvest of the arable crops. Average value per farm weighted by the area of each crop on the various

soil types.b Other crops are oats, barley, canola, peas, tobacco, apple orchards, vineyards or grasslands, depending on the farm type. Each of these crops showed

little change compared to the base year.

Table 4Impact of WTO G20 proposal on arable farming systems (average acrossfarm types) in Midi Pyrenees

Indicators (Unit) Baseyear[2001]

Baselinescenario[2013] (%)

WTO G20 proposal–policy scenario[2013] (%)

Farm income (€/ha) 672 19 �23Premiums (K€ per farm) 43.9 �18 �18Nitrate leaching (kg/ha) 8.2 3 4Soil organic matter (%) 2.0 �0.2 �0.3

For the scenarios the impacts are given as the relative deviation of theindicator in 2013 compared to 2001. Indicators at regional level are theaverage of the corresponding indicators for each farm type (see Table 3)weighted by the total number of farms per type (for economic indicators)or by the total area represented by the farm types (for environmentalindicators).


of regional supply modules in SEAMCAP to aggregatedsupply response. This method is currently tested.

Second, SEAMLESS-IF allows the assessment of eco-nomic, environmental and social indicators, as well as insti-tutional constraints. Examples of economic andenvironmental indicators have been provided already inTable 3. An example of a social indicator is income varia-

tion (‘equity’) across farm types and regions and effects ofpolicies on agricultural employment. Both can readily becomputed from FSSIM output (income variation) andthe post-model analysis for SEAMCAP (agriculturalemployment – Section 3.2.2). Qualitative or semi-quantita-tive pre- or post-model analyses are an essential part ofintegrated assessment in SEAMLESS-IF to assess institu-tional aspects. Crucial institutional aspects include theroles and positions of the main political actors, the avail-able options (or, action alternatives) for the main stake-holders (e.g., water board, farmers, etc.), the type ofpolicy measure (incentive-based instruments, command-and-control, etc.), the way of policy implementation (e.g.,via administrative authorities at various levels), and thedecision maker’s competencies and capabilities (e.g.,income level, communication capacity, social capital).The Procedure for Institutional Compatibility Assessment(PICA) (Schleyer et al., 2007) allows assessment of suchinstitutional aspects. The SEAMLESS-IF approach ini-tially assumes that the appropriate and required institu-tions are in place to introduce new policies or agro-technologies and to let them be effective. PICA questionsthese assumptions and provides a procedure to test them


and to assess the institutional (in)-compatibility of policyoptions. PICA clusters policy options in generic policytypes (e.g., types and area of intervention), links these typesto crucial institutional aspects, selects and develops institu-tional indicators to evaluate the crucial institutional aspectsand derives statements about the probable effectiveness ofthe policy option from an institutional perspective.

Third, SEAMLESS-IF is designed with stand-alonecomponents. APES, FSSIM and SEAMCAP can be (re-)used as stand-alone models and they can be linked inSEAMLESS-IF to assess new policy proposals. Somemay only need some of the components (e.g., policies witha regional focus which do not require a full market analy-sis, only need APES and FSSIM). Each of the componentscan be further developed such that they represent state-of-the-art science.

Fourth, the common ontology allows the consistentconceptual linkage of the components. A ‘winter wheat’crop has the same unambiguous meaning throughout thevarious components (e.g., FSSIM and SEAMCAP), what-ever the scientific background of the user, and ‘yield of win-ter wheat’ has one consistent unit. The use of OpenMIallows the technical linkages of components, even thoughthey have been programmed in different languages, forinstance C# (APES), JAVA and GAMS (FSSIM, EXPA-MOD, CAPRI).

5. Science and society perspectives

In this final section, we discuss some considerations inthe development of the framework, both from the perspec-tive of scientific and user (society) community.

5.1. Science perspectives

We have designed the SEAMLESS integrated frame-work from the notion that agricultural science (and naturalresource management science in general) requires integra-tive frameworks to overcome current fragmentation inresearch methods and tools from a methodological andtechnical point of view. The four ambitions introduced inthe first section of the paper and discussed above in thecontext of the illustration, i.e., overcoming the micro–macro gap, the bias in integrated assessments, the poorre-use of models and hindrances in technical linkage, havebeen addressed by proposing an integrated framework thatallows re-use of past investments, and the integration ofnew knowledge through synthesis in stand-alone modeland data components capturing state-of-the-art science. Itallows concentrating on linking different hierarchical levels,e.g., the micro and macro analysis through extrapolatingthe price–supply relationships, rather than on the model-ling of processes at a specific scale. This proposed upscalingmethodology has yet to be tested – if successful it will beexplored whether the method is also applicable to upscaleenvironmental externalities of farming systems assessedwith bio-economic farm models.

Evidently there are risks involved in developing method-ologies such as proposed in this paper. Two types of riskare eminent. First, we have opted for one out of a rangeof methods for integrated assessment, i.e., linking fairlycomplex model components which are targeted at specifichierarchical levels of agricultural systems. It has yet to beshown that the selected method is sufficiently generic, flex-ible and operational. For specific problems different typesof meta-modelling or systems dynamic modelling (VanGigch, 1991; Sterman, 2001) may well be superior in termsof focusing on the relevant processes at the proper level ofdetail and in terms of data availability for that specificproblem. Also, the proposed methodology bears an inher-ent risk in carrying too much detail from lower hierarchicallevels to a next. Yet, in our view the proposed method is theway forward to meet the requirements of being able to re-use models and a framework for new questions that mayarise and to easily allow incorporation of new knowledgeand insights. Second, there is a clear tension between trans-parency, usability and flexibility. Flexibility of the systemposes significant challenges on modularity of the models,separating models from data and graphical user interfaces,and the use of software approaches which challenge thetransparency and user-friendliness of the framework.Graphical user interfaces and training modules are beingdeveloped to overcome many of these issues, but duringthe development phase the collaboration of scientists isclearly challenged by this issue and usability of the frame-work will remain an important focal point in the final twoyears of the project.

5.2. Society perspectives

SEAMLESS-IF was developed and evolved in interac-tion with the Directorate General Research of the EC.Since its start we have actively engaged with potential usersfrom the EC. We also, in a less structured way, interactwith representatives from national ministries and nationaland international institutions. Recurring themes in theinteractions are threefold.

Firstly, users’ appreciation was high for a frameworkthat aims at being generic and flexible: they foresee thatpolitical issues change between now and five years and theyfind it appealing if the framework is able to handle differentissues or that it can be expanded efficiently to address newissues. They also appreciate the different types of results theframework may produce, ranging from more classical cost-effectiveness of policy measures to the comprehensiveassessment of proposed policies through a wide range ofindicators and trade-offs between such indicators. Sec-ondly, they repeatedly bring up the issue of maintenanceand usability of the framework, referring to both organisa-tional, data, modelling and funding issues. Thirdly, for aframework such as SEAMLESS-IF credibility and infor-mation about sensitivity and uncertainty is of key impor-tance. We serve credibility by aiming for transparencythrough a knowledge base and the use of meta-information


and ontologies which facilitate communication of frame-work and model assumptions, as well as tracking and tracingof variables and parameters throughout the framework. Theindividual sources being used in SEAMLESS-IF must havebeen peer reviewed. And, we are seeking an extended peerreview (Ravetz, 2005) through interaction with the users.Open source rather than closed source as the standard forSEAMLESS-IF is important in stimulating use and peerreview. Quality assurance procedures are an important con-tribution to credibility as well (Refsgaard et al., 2005).

5.3. Concluding comments

The paper has presented how SEAMLESS-IF addressesfour challenges as to quantitative scientific tools for inte-grated assessment, i.e., the micro–macro gap, the bias inintegrated assessments, the poor re-use of models and hin-drances in technical linkage of models. The developmentof prototype versions of SEAMLESS-IF has been crucialto demonstrate, both to scientists involved in the projectand to users, that the concept of linking model componentsfor use in integrated assessment is feasible. SEAMLESS tar-gets at a working version of the integrated framework by2009 for its prime users in the European Commission. Bythat time it is possible to evaluate whether the tools areeffective in improving the Impact Assessment proceduresin the EC. At the same time the software infrastructure ofthe project is anticipated to provide an open source meansto facilitate linkage and integration of models and otherknowledge sources from different domains and pro-grammed in different environments and languages. Finally,the different components of SEAMLESS-IF are designed tohave stand-alone value. These components can be used fortargeted applications or serve as a starting point for furtherscientific development. As such, we aim that the integratedframework facilitates synthesis of scientific knowledge inthe domain of agriculture and its environment beyond thespecific setting of the project and we therefore anticipatethat this initial publication of its rationale and design willstimulate interaction and improve the development.

Acknowledgements

The work presented in this publication is funded by theSEAMLESS integrated project, EU 6th Framework Pro-gramme for Research Technological Development andDemonstration, Priority 1.1.6.3. Global Change and Eco-systems (European Commission, DG Research, ContractNo. 010036-2). We gratefully acknowledge all SEAMLESSparticipants who contributed to the development of thefirst prototypes. We thank Ing. Eelco Meuter for preparingfigures and further support.

References

Andersen, E., Elbersen, B., Godeschalk, F., Verhoog, D., 2007. Farm

management indicators and farm typologies as a basis for assessments

in a changing policy environment. Journal of Environmental Man-

agement 82, 353–362.

Arfini, F. (Ed.), 2005. Modelling agricultural policies: state of the art and

new challenges. In: Proceedings of the 89th European seminar of the

European Association of Agricultural Economists. Monte Universita

Parma, Parma.

Argent, R.M., 2004. An overview of model integration for environmental

applications–components, frameworks and semantics. Environmental

Modelling Software 19, 219–234.

Argent, R.M., Rizzoli, A.E., 2004. Deveopment of multi-framework

model components. In: Pahl-Wostl, C., Schmidt, S., Rizzoli, A.E.,

Jakeman, A.J. (Eds.), Complexity and Integrated Resources Manage-

ment, Transactions of the 2nd Biennial Meeting of the International

Environmental Modelling and Software Society, iEMSs, Osnabruck,

Germany.

Argent, R.M., Voinov, A., Maxwell, T., Cuddy, S.M., Rahman, J.M.,

Seaton, S., Vertessy, R.A., Braddock, R.D., 2006. Comparing mod-

elling frameworks – a workshop approach. Environmental Modelling

Software 21, 895–910.

Armington, P.S., 1969. A theory of demand for products distinguished by

place of production. IMF Staff Papers 18, 159–178.

Belhouchette, H., Therond, O., Adenauer, M., Kuiper, M., Bigot, G.,

Alkan Olsson, J., Wery, J., 2007. D6.2.3.3 – Documentation of

baseline and policy options to be assessed with Prototype 2 and 3.

SEAMLESS Integrated Project, EU 6th Framework Programme.

<www.SEAMLESS-IP.org>, 112 pp.

Britz, W., Perez, I., Zimmermann, A., Kempen, M., Heckelei, T., 2006.

PD3.5.1 – Definition of the CAPRI Core Modelling System. SEAM-

LESS Integrated Project, EU 6th Framework Programme.

<www.SEAMLESS-IP.org>.

Brooke, A., Kendrick, D., Meeraus, A., Raman, R., 2006. A User’s Guide

of GAMS. GAMS Development Corporation, Washington, DC.

Carlini, L., Bellocchi, G., Donatelli, M., 2006. Rain, a software compo-

nent to generate synthetic precipitation data. Agronomy Journal 98,

1312–1317.

Dalgaard, T., Hutchings, N.J., Porter, J.R., 2003. Agroecology, scaling

and interdisciplinarity. Agriculture, Ecosystems Environment 100, 39–

51.

Deybe, D., Flichman, G., 1991. A regional agricultural model using a

plant growth simulation program as activities generator. Agricultural

Systems 37, 369–385.

Dogliotti, S., Rossing, W.A.H., van Ittersum, M.K., 2003. ROTAT, a tool

for systematically generating crop rotations. European Journal of

Agronomy 19, 239–250.

Donatelli, M., Bellocchi, G., Carlini, L., Colauzzi, M., 2005. CLIMA: a

component-based weather generator frameworks. In: Zerger, A.,

Argent, R.M. (Eds.), MODSIM 2005 International Congress on

Modelling and Simulation. Modelling and Simulation Society of

Australia and New Zealand, Melbourne, Australia, pp. 627–633.

Donatelli, M., Bellocchi, G., Carlini, L., 2006a. Sharing knowledge via

software components: models on reference evapotranspiration. Euro-

pean Journal of Agronomy 24, 186–192.

Donatelli, M., Carlini, L., Bellocchi, G., 2006b. A software component for

estimating solar radiation. Environmental Modelling and Software 21,

411–416.

Donatelli, M., Van Evert, F.K., Di Guardo, A., Adam, M., Kansou, K.,

2006c. A component to simulate agricultural management. In: Voinov,

A., Jakeman, A., Rizzoli, A.E. (Eds.), iEMSs Third Biannual Meeting,

Summit on Environmental Modelling and Software. International

Environmental Modelling and Software Society, Burlington, USA.

EC, 2001. A sustainable Europe for a better world: a European strategy

for sustainable development. European Commission’s proposal for the

Gothenburg European Council, COM 264.

EC, 2005a. Impact Assessment Guidelines. SEC(2005) 791.

EC, 2005b. The commissions strategic objectives 2005–2009. European

Commission, COM 12.

EC, 2006. Prospects for Agricultural Markets and Income 2006–2013.

European Commission.

http://www.SEAMLESS-IP.org

http://www.SEAMLESS-IP.org


Elbersen, B.S., Kempen, M., van Diepen, C.A., Andersen, E., Hazeu,

G.W., Verhoog, A.D., 2006. Protocols for spatial allocation of farm

types. SEAMLESS Integrated Project, EU 6th Framework Pro-

gramme, Report No. 19. <http://www.seamless-ip.org/Reports/Re-

port_19_PD4.7.1.pdf>.

Ewert, F., van Keulen, H., van Ittersum, M.K., Giller, K.E., Leffelaar,

P.A., Roetter, R.P., 2006a. Multi-scale analysis and modelling of

natural resource management. In: Voinov, A., Jakeman, A., Rizzoli,

A.E. (Eds.), Proceedings of the iEMSs Third Biannual Meeting

‘‘Summit on Environmental Modelling and Software’’. International

Environmental Modelling and Software Society, Burlington, USA,

Burlington, USA <http://www.iemss.org/iemss2006/papers/s9/

128_Ewert_0.pdf>.

Ewert, F.A., van Ittersum, M.K., Bezlepkina, I., Oude Lansink,

A.G.J.M., Andersen, E., Backlund, A.K., Brouwer, F.M., Flichman,

G., Heckelei, T., Alkan Olsson, J., Olsson, L., Rizzoli, A.E.,

Theesfeld, I., Wery, J., Wien, J.E., Elbersen, B.S., Donatelli, M.,

Martinson, L., Perez, I., Svensson, M.G.E., van Diepen, C.A., van

der Wal, T., Wolf, J., 2006b. Methodological concepts for integrated

assessment of agricultural and environmental policies in SEAM-

LESS-IF. SEAMLESS Integrated Project, EU 6th Framework

Programme, Report No. 18. <http://www.seamless-ip.org/Reports/

Report_18_PD1.3.8.pdf>.

G20, 2005. G20 proposal on market access – October 12, 2005. <http://

www.g-20.mre.gov.br/conteudo/proposals_marketaccess.pdf>.

Gijsbers, P.J.A., Gregersen, J.B., 2005. OpenMI: a glue for model

integration. In: Zerger, A., Argent, R.M. (Eds.), MODSIM 2005

International Congress on Modelling and Simulation. Modelling and

Simulation Society of Australia and New Zealand, pp. 648–654.

Gijsbers, P.J.A., Wien, J.E., Verweij, P., Knapen, R., 2006. Advances in

the OpenMI. In: Proceedings of 7th International Conference on

Hydroinformatics, HIC 2006. Nice, France, pp. 72-81.

Hansen, J.W., Jones, J.W., 2000. Scaling-up crop models for climate

variability applications. Agricultural Systems 66, 43–72.

Harris, G., 2002. Integrated assessment and modeling – science for

sustainability. In: Costanza, R., Joergensen, S.E. (Eds.), Understand-

ing and Solving Environmental Problems in the 21st Century. Elsevier,

pp. 5–17.

Hazell, P.B.R., Norton, R.D., 1986. Mathematical Programming for

Economic Analysis in Agriculture. MacMillan Publishing Company,

London, UK.

Hazeu, G.W., Elbersen, B.S., van Diepen, C.A., Metzger, M.J., 2006.

Regional typologies of ecological and biophysical context. SEAM-

LESS Integrated Project, EU 6th Framework Programme, Report No.

14. <http://www.seamless-ip.org/Reports/Report_14_PD4.3.3.pdf>.

Heckelei, T., Britz, W., 2001. Concept and explorative application of an

EU-wide regional agricultural sector model (CAPRI-Projekt). In:

Heckelei, T., Witzke, H.P., Henrichsmeyer, W. (Eds.), Proceedings of

the 65th EAAE Seminar. Vauk Verlag, Kiel, Germany, Bonn

University, pp. 281–290.

Heckelei, T., Witzke, H.P., Henrichsmeyer, W. (Eds.), 2001. Agricultural

sector modelling and policy information systems. Proceedings of the

65th seminar of the European Association of Agricultural Economists.

Vauk Verlag, Kiel.

Hengsdijk, H., van Ittersum, M.K., 2003. Formalizing agro-ecological

engineering for future-oriented land use studies. European Journal of

Agronomy 19, 549–562.

Hertel, T.W. (Ed.), 1997. Global Trade Analysis: Modeling and Appli-

cations. Cambridge University Press, Cambridge, New York.

Hillyer, C., Bolte, J., van Evert, F., Lamaker, A., 2003. The ModCom

modular simulation system. European Journal of Agronomy 18, 333–

343.

Janssen, S., van Ittersum, M.K., 2007. Assessing farm innovations and

responses to policies: a review of bio-economic farm models. Agricul-

tural Systems 94, 622–636.

Kropff, M.J., Jones, J.W., Van Laar, G., 2001. Advances in systems

approaches for agricultural development. Agricultural Systems 70,

353–354.

Kruseman, G., 2000. Bio-Economic Household Modelling for Agricul-

tural Intensification, PhD thesis, Wageningen University, The Neth-

erlands.

Leavesley, G.H., Markstrom, S.L., Brewer, M.S., Viger, R.J., 1996. The

modular modelling system (MMS) – the physical process modelling

component of a database-centered decision support system for water

and power management. Water, Air, and Soil Pollution 90, 303–311.

Leontief, W., 1986. Input–Output Economics. Oxford University Press,

New York.

Louhichi, K., Janssen, S.J.C., Li, H., Zander, P., Stokstad, G., Kanel-

lopoulos, A., van Ittersum, M.K., van der Maden, E., Hengsdijk, H.,

Flichman, G., Blanco Fonseca, M., Rizzoli, A.E., 2006. D3.3.6 – A

first version of FSSIM for the arable farming sector to be used within

Prototype 1 of SEAMLESS-IF. SEAMLESS Integrated Project, EU

6th Framework Programme. <www.seamless-ip.org>.

McIntosh, B.S., Giupponi, C., Voinov, A., Smith, C., Matthews, K.B.,

Monticino, M., Kolkman, M.J., Crossman, N., van Ittersum, M.,

Haase, D., Haase, A., Mysiak, J., Groot, J.C.J., Sieber, S., Verweij, P.,

Quinn, N., Waeger, P., Gaber, N., Hepting, D., Scholten, H., Sulis, A.,

van Delden, H., Gaddis, E., Assaf, H., 2007. State of the art and

futures in environmental modelling and software. In: Jakeman, A.J.,

Voinov, A., Rizzoli, A.E., Chen, S. (Eds.), iEMSs book, in prepara-

tion.

Parker, P., Letcher, R., Jakeman, A., Beck, M.B., Harris, G., Argent,

R.M., Hare, M., Pahl-Wostl, C., Voinov, A., Janssen, M., Sullivan, P.,

Scoccimarro, M., Friend, A., Sonnenshein, M., Baker, D., Matejicek,

L., Odulaja, D., Deadman, P., Lim, K., Larocque, G., Tarikhi, P.,

Fletcher, C., Put, A., Maxwell, T., Charles, A., Breeze, H., Nakatani,

N., Mudgal, S., Naito, W., Osidele, O., Eriksson, I., Kautsky, U.,

Kautsky, E., Naeslund, B., Kumblad, L., Park, R., Maltagliati, S.,

Girardin, P., Rizzoli, A.E., Mauriello, D., Hoch, R., Pelletier, D.,

Reilly, J., Olafsdottir, R., Bin, S., 2002. Progress in integrated

assessment and modelling. Environmental Modelling Software 17,

209–217.

Parson, E.A., 1995. Integrated assessment and environmental policy

making. Energy Policy 23, 463–475.

Rahman, J.M., Seaton, S.P., Cuddy, S.M., 2004. Making frameworks

more useable: using model introspection and metadata to develop

model processing tools. Environmental Modelling Software 19, 275–

284.

Ravetz, J.R., 2005. The post-normal sciences of precaution. Water Science

Technology 52, 11–17.

Refsgaard, J.C., Henriksen, H.J., Harrar, W.G., Scholten, H., Kassahun,

A., 2005. Quality assurance in model based water management –

review of existing practice and outline of new approaches. Environ-

mental Modelling Software 20, 1201–1215.

Rizzoli, A.E., Davis, J.R., Abel, D.J., 1998. Model and data integration

and re-use in environmental decision support systems. Decision

Support Systems 24, 127–144.

Rizzoli, A.E., Donatelli, M., Athanasiadis, I.N., Villa, F., Huber, D.,

2007. Semantic links in integrated modelling frameworks. Mathemat-

ics and Computers in Simulation, in press.

Rotmans, J., van Asselt, M.B.A., 1996. Integrated assessment: growing

child on its way to maturity. An editorial essay. Climatic Change 34,

327–336.

Schleyer, C., Theesfeld, I., Hagedorn, K., Aznar, O., Callois, J.M., et al.,

2007. An approach towards an operational tool to apply institutional

analysis for the assessment of policy feasibility within SEAMLESS-IF.

SEAMLESS Integrated Project, EU 6th Framework Programme,

Report No. 29. <www.seamless-ip.org/Reports/Report_29_D2.4.2.

pdf>.

Sissoko, K., 1998. Et demain I’Agriculture? Options techniques et mesures

politiques pour un developpement agricole durable en Afrique

subsaharienne Cas du Cercle de Koutiala en zone sud du Mali. PhD,

Wageningen University, The Netherlands.

Spangenberg, J.H., Pfahl, S., Deller, K., 2002. Towards indicators for

institutional sustainability: lessons from an analysis of Agenda 21.

Ecological Indicators 2, 61–77.

http://www.seamless-ip.org/Reports/Report_19_PD4.7.1.pdf


http://www.iemss.org/iemss2006/papers/s9/128_Ewert_0.pdf

http://www.iemss.org/iemss2006/papers/s9/128_Ewert_0.pdf



http://www.g-20.mre.gov.br/conteudo/proposals_marketaccess.pdf

http://www.g-20.mre.gov.br/conteudo/proposals_marketaccess.pdf


http://www.seamless-ip.org

http://www.seamless-ip.org/Reports/Report_29_D2.4.2.pdf

http://www.seamless-ip.org/Reports/Report_29_D2.4.2.pdf


Sterman, J.D., 2001. System dynamics modeling: tools for learning in a

complex world. California Management Review 43, 8–25.

Stockle, C.O., Donatelli, M., Nelson, R., 2003. CropSyst, a cropping

systems simulation model. European Journal of Agronomy 18, 289–307.

Szyperski, C., Gruntz, D., Murer, S., 2002. Component Software –

Beyond Object-Oriented Programming. ACM press, New York.

Thompson, A.M.M., 1982. A Farm-Level Model to Evaluate the Impacts

of Current Energy Policy Options. Lincoln College, Canterbury.

Tidwell, J., 2006. Designing Interfaces. O’Reilly.

Van der Wal, T., Knapen, R., Svensson, M., Athanasiadis, I., Rizzoli,

A.E., 2005. Trade-offs in the design of cross-disciplinary software

system. In: Zerger, A., Argent, R.M. (Eds.), MODSIM 2005 Interna-

tional Congress on Modelling and Simulation. Modelling and Simu-

lation Society of Australia and New Zealand, Melbourne, pp. 732–737

<http://www.mssanz.org.au/modsim05/papers/van_der_wal.pdf>.

Van Gigch, J.P., 1991. System Design Modeling and Metamodeling.

Plenum Press, New York.

Van Ittersum, M.K., Donatelli, M., 2003. Modelling cropping systems –

highlights of the symposium and preface to the special issues.

European Journal of Agronomy 18, 187–197.

Van Tongeren, F., Van Meijl, H., Surry, Y., 2001. Global models applied

to agricultural and trade policies: a review and assessment. Agricul-

tural Economics 26, 149–172.

Verburg, P.H., Rounsevell, M.D.A., Veldkamp, A., 2006. Scenario-based

studies of future land use in Europe. Agriculture, Ecosystems

Environment 114, 1–6.

Villa, F., Athanasiadis, I., 2006. PD4.2.1 – Guidelines for populating the

demo knowledge database. SEAMLESS Integrated Project, EU 6th

Framework Programme. <www.SEAMLESS-IP.org>.

Villa, F., Donatelli, M., Rizzoli, A.E., Krause, P., Kralisch, S., van Evert,

F.K., 2006. Declarative modelling for architecture independence and

data/model integration: a case study. In: Voinov, A., Jakeman, A.,

Rizzoli, A.E. (Eds.), iEMSs Third Biannual Meeting ‘‘Summit on

Environmental Modelling and Software’’. International Environmen-

tal Modelling and Software Society, Burlington, USA <http://

www.iemss.org/iemss2006/papers/s5/278_Villa_1.pdf>.

Wossink, G.A.A., De Koeijer, T.J., Renkema, J.A., 1992. Environmental–

Economic Policy assessment: a farm economic approach. Agricultural

Systems 39, 421–438.

Zeigler, B.P., 1987. Hierarchical, modular discrete-event modelling in an

object-oriented environment. Simulation 45, 219–230.

Zimmermann, A., Heckelei, T., Perez, I., 2006. Working Paper –

Literature Reveiw of Approaches to Estimate Structural Change.

SEAMLESS Integrated Project, EU 6th Framework Programme,

Report No. 16. <http://www.seamless-ip.org/Reports/Report_16_

PD3.6.6.pdf>.

http://www.mssanz.org.au/modsim05/papers/van_der_wal.pdf

http://www.iemss.org/iemss2006/papers/s5/278_Villa_1.pdf

http://www.iemss.org/iemss2006/papers/s5/278_Villa_1.pdf



overview journal article

Documents