huub scholten and adrie j.m. beulens wageningen university - netherlands
DESCRIPTION
Testing ontological support for multidisciplinary model-based problem-solving for water management. Huub Scholten and Adrie J.M. Beulens Wageningen University - Netherlands. Index. Introduction Problems to solve HarmoniQuA - MoST Project Ontological KB and Knowledge Editor - PowerPoint PPT PresentationTRANSCRIPT
Laxenburg, 29-31 August 2005 Huub Scholten - CSM/KSS2005 - test MoST 1
Testing ontological support for multidisciplinary model-based
problem-solving for water management
Huub Scholten and Adrie J.M. Beulens
Wageningen University - Netherlands
Laxenburg, 29-31 August 2005
Huub Scholten - CSM/KSS2005 - test MoST 2
Index Introduction Problems to solve HarmoniQuA - MoST
Project Ontological KB and Knowledge Editor Modelling Support Tool
Verification - validation Mental Modelling Model Testing: criteria and results Conclusions
Laxenburg, 29-31 August 2005
Huub Scholten - CSM/KSS2005 - test MoST 3
Introduction
Model-based problem solving is seen as a process Integrated problem-solving (with socio-economics
aspects) multidisciplinary problems higher demands to QA
QA through Process Knowledge Base
(here for modelling, instantiated for water management) Process support tool
Laxenburg, 29-31 August 2005
Huub Scholten - CSM/KSS2005 - test MoST 4
Problems to solve Growing interest for QA in modelling with a multitude of
reasons: ambiguous terminology; a lack of mutual understanding between key-players, malpractice in regard to input data; inadequate model set-up; insufficient calibration/validation; model use outside of its intended scope; insufficient knowledge on some processes; miscommunication of the modeller to the end-user; overselling of model capabilities; confusion on how to use model results in decision making; a lack of documentation and transparency of modelling processes.
Laxenburg, 29-31 August 2005
Huub Scholten - CSM/KSS2005 - test MoST 5
HarmoniQuA - the project
Period 1-1-2002 – 31-12-2005 (4 years)
2.57 million Euro (1.65 million Euro financed by EC)
12 partners in 10 countries with expertise in 7 domains of model
based water management
www.HarmoniQuA.org
Laxenburg, 29-31 August 2005
Huub Scholten - CSM/KSS2005 - test MoST 6
HarmoniQuA - partners
WU, The Netherlands GEUS, Denmark NTUA, Athens, Greece CEH-Wallingford, UK WL | Delft Hydraulics,The
Netherlands Cemagref, France BfG, Germany SMHI, Sweden VITUKI Plc, Hungary Uni Dortmund, Germany LNEC, Portugal DHI Hydroinform a.s.,
Czech Republic
Laxenburg, 29-31 August 2005
Huub Scholten - CSM/KSS2005 - test MoST 7
HarmoniQuA - aimsDeveloping a methodology and support for modelling in water
management according to the Water Framework Directive
consisting of:
1. Knowledge Base (KB): Guidance on how to model for water management Glossary of terms
2. Modelling Support Tool (MoST): Providing guidance from the KB Monitoring what modelling teams do Helping generating fancy reports
3. Training website for professionals and students: Screen-recording movies on MoST Presentations, exercises, background information
Model StudyPlan
Data andConceptualisation
ModelSet-up
Calibrationand
Validation
Simulation and
Evaluation
Model TeamSingle/multiple domain
MoST
Reporting and client review take place in each step
User
Knowledge BaseGuidelinesSoftware capabilitiesGlossaryDomains:
GroundwaterPrecipitation-runoffHydrodynamicsFlood forecastingWater qualityBiota (ecology)Socio-economics
GuidanceGeneric + specific for:- model domain- user- job complexity
ReportingSpecific for typesof users
MonitoringGeneric + specific for:- model domain- user- job complexity
AdviseFrom previousmodel projects
ModelArchive
Model journal, Project A
Model journal, Project B
Model journal, Project C
Model journal, Project D
MoS
T (
KB
+ to
ol)
over
view
Laxenburg, 29-31 August 2005
Huub Scholten - CSM/KSS2005 - test MoST 9
Knowledge Base Content (what to do and how to do it) in increasing
specialization Knowledge on processes (co-operating in teams)
Specialised for modelling
Instantiated for water management
Implemented in an ontology (e.g. smart database; Protégé2000, (http://protege.stanford.edu) Protégé2000 is for knowledge engineering experts and too difficult
for domain experts
HarmoniQuA’s Knowledge Base Editor act as front-end for Protégé2000 and deals with authorisation issues
Provided to users by MoST (Modelling Support Tool)
Laxenburg, 29-31 August 2005
Huub Scholten - CSM/KSS2005 - test MoST 10
Modelling Support Tool (MoST)
Provides guidance from the KB to users
Can set-up and support the execution of multidisciplinary
modelling projects
Monitors the work of all persons involved in a modelling project
Reports to specific audiences
Laxenburg, 29-31 August 2005
Huub Scholten - CSM/KSS2005 - test MoST 11
Verification – validation of KBS
Verification: determining that the KBS is built according to its specifications.
Validation: determining that the KBS actually fulfils the purpose for which it was intended.
Testing: more generic term, referring to verification and validation.
Laxenburg, 29-31 August 2005
Huub Scholten - CSM/KSS2005 - test MoST 12
Mental Modelling Model If problem owner has a problem, he/she can organize a
multidisciplinary team, consisting of team members, who have different disciplinary backgrounds and fulfil different roles in that team.
The multidisciplinary team ‘solve’ the problem at hand in a process with best professional practices.
The process is carried out in a modelling project, consisting of one or more subprojects, belonging to one or more domains/disciplines.
Subprojects: are single domain subproject or multidomain subproject; consist of steps.
Steps consist of tasks. Tasks consists of activities. Tasks and activities can use methods and tools. What teams have to do can be made explicit in a knowledge base.
Laxenburg, 29-31 August 2005
Huub Scholten - CSM/KSS2005 - test MoST 13
Structured testing
Implemented ontological KBS (1) KB structure (11) KB content (related to water management) (12)
Process decomposition (121)Content of decomposition elements (122)
MoST (13) Training material (14)
KBS technology (2)
Laxenburg, 29-31 August 2005
Huub Scholten - CSM/KSS2005 - test MoST 14
KB structure (11)Criteria
Correctness, completeness, consistency, granularity.
Method(s) Discussions with (1) internal experts, (2) internal testers and
(3) external reviewers. Using the structure to fill KB with the knowledge.
Result(s)
Initially discussions were fruitful, later not.
Laxenburg, 29-31 August 2005
Huub Scholten - CSM/KSS2005 - test MoST 15
KB content: process decomposition (121)Criteria
Correctness, completeness, redundancy, consistency, transparency, granularity.
Method(s) Discussions with (1) internal experts, (2) internal testers and
(3) external reviewers. Using the KB in 2x10 test cases.
Result(s) Demands for changing the process decomposition came from
project partners and from the wider modelling society. These were related to (1) order of tasks, (2) dependencies of tasks and (3) is some piece of work a task or an activity.
Achieving agreement was difficult, but implementing changes was easy and not time consuming.
Laxenburg, 29-31 August 2005
Huub Scholten - CSM/KSS2005 - test MoST 16
KB content: decomposition elements (122)Criteria
Correctness, completeness, redundancy, consistency, meaningfulness.
Method(s)Reading and using by (1) internal testers (not involved in filling KB, (2) professional modellers in test cases, (3) external reviewers.
Result(s)Long list of errors, wishes, comments, but all appreciated the guidance from the KB.
Laxenburg, 29-31 August 2005
Huub Scholten - CSM/KSS2005 - test MoST 17
Modelling Support Tool (13)Criteria
Correctness, reliability, according to requirements, adequate for professionals, appropriate for students.
Method(s)Using MoST by (1) internal testers (not involved in filling KB),
(2) professional modellers in test cases, (3) external reviewers.
Result(s)Incremental (prototyping) development of MoST in 3 phases, with a test after phase 1 and phase 2 long list of ‘bugs’ more support needed for multidisciplinary teams + more flexibility (working
on 2 decomposition levels), because activity-level is often too detailed.
Laxenburg, 29-31 August 2005
Huub Scholten - CSM/KSS2005 - test MoST 18
Testing training material (14)Criteria
Correctness, usefulness professionals, usefulness students.
Method(s)Using MoST in (internal and external) workshops/seminars and university courses.
Result(s)Many small changes, but training website and especially screen-recording movies on MoST appeared to flatten the (steep) learning curve.
Laxenburg, 29-31 August 2005
Huub Scholten - CSM/KSS2005 - test MoST 19
Testing re-use of technology (2)Criteria
Is the KB ontological structure reusable and is the tool MoST useful for other processes?
Method(s)Applying the presented technology for other processes, i.e. within the EC-funded project Aquastress (aiming to mitigate water stress problems).
Result(s) Applying the ontological KB structure for the Aquastress project,
did not show any technical problem. MoST could show and use the new KB without a problem.
Laxenburg, 29-31 August 2005
Huub Scholten - CSM/KSS2005 - test MoST 20
Conclusion Testing is a continuous process. Peer reviewed scientific output = testing. Applying our technology for other processes in which
multidisciplinary teams have to perform a complex job, shows that the technology has a wide field of application
Public website www.HarmoniQuA.org free software training material etc.