open source tools for uncertainty enabling the model web benjamin proß university of münster...
TRANSCRIPT
![Page 1: Open Source Tools for Uncertainty Enabling the Model Web Benjamin Proß University of Münster FOSS4G, Denver, 12-16 Sept 2011](https://reader035.vdocuments.us/reader035/viewer/2022070404/56649f335503460f94c508e2/html5/thumbnails/1.jpg)
Open Source Tools for Uncertainty Enabling
the Model Web
Benjamin ProßUniversity of Münster
FOSS4G, Denver, 12-16 Sept 2011
![Page 2: Open Source Tools for Uncertainty Enabling the Model Web Benjamin Proß University of Münster FOSS4G, Denver, 12-16 Sept 2011](https://reader035.vdocuments.us/reader035/viewer/2022070404/56649f335503460f94c508e2/html5/thumbnails/2.jpg)
• UncertWeb
• Data Encodings/Profiles
• Web Services
• Other Tools
Overview
![Page 3: Open Source Tools for Uncertainty Enabling the Model Web Benjamin Proß University of Münster FOSS4G, Denver, 12-16 Sept 2011](https://reader035.vdocuments.us/reader035/viewer/2022070404/56649f335503460f94c508e2/html5/thumbnails/3.jpg)
UncertWeb Facts
• Uncertainty-enabled Model Web, FP7 Project (http://www.uncertweb.org/)
• Feb 2010 – Jan 2013• 8 Partners
• Aston University• Italian National Research Council• Food and Environment Research Agency• Joint Research Centre• Norwegian Institute for Air Research• Eindhoven University of Technology• University of Muenster• Wageningen University
![Page 4: Open Source Tools for Uncertainty Enabling the Model Web Benjamin Proß University of Münster FOSS4G, Denver, 12-16 Sept 2011](https://reader035.vdocuments.us/reader035/viewer/2022070404/56649f335503460f94c508e2/html5/thumbnails/4.jpg)
The UncertWeb concept
• The “model web”.
• When chaining services of limited or unknown quality, uncertainty must be accounted for if rational decisions are to be made.
ECMWFensembles
GEMSensembles
Emissions data
Air quality forecastmodel
Modelparameters
Uncertaintyenabled air quality
forecast
Air qualitysensor network
Short range airquality forecasts
Meteorologicalsensor network
ECMWF model
GEMS models
Near real-time airquality nowcasts
Land use data
Albatross individualactivity simulator
Historical activitydata
Traffic model
Individualexposure
assessment
Emissions model
GISinfrastructure
data
Demographicdata
“UncertWeb develops mechanisms, standards, tools and test-beds for accountable uncertainty propagation in web service chains.”
![Page 5: Open Source Tools for Uncertainty Enabling the Model Web Benjamin Proß University of Münster FOSS4G, Denver, 12-16 Sept 2011](https://reader035.vdocuments.us/reader035/viewer/2022070404/56649f335503460f94c508e2/html5/thumbnails/5.jpg)
Definitions
• Uncertainty– Model inputs
• E.g. through measurement errors of sensors • Taking outputs of other models as inputs
– Models itself• By aproximating reality, models introduce errors,
and should inform the user about (annotate its output with) the degree this happens
![Page 6: Open Source Tools for Uncertainty Enabling the Model Web Benjamin Proß University of Münster FOSS4G, Denver, 12-16 Sept 2011](https://reader035.vdocuments.us/reader035/viewer/2022070404/56649f335503460f94c508e2/html5/thumbnails/6.jpg)
• Biodiversity and climate change
• Land-use response to climatic and economic change
• Short term uncertainty-enabled forecasts for local air quality
• Individual activity in the environment
Application Scenarios
![Page 7: Open Source Tools for Uncertainty Enabling the Model Web Benjamin Proß University of Münster FOSS4G, Denver, 12-16 Sept 2011](https://reader035.vdocuments.us/reader035/viewer/2022070404/56649f335503460f94c508e2/html5/thumbnails/7.jpg)
Data encodings
![Page 8: Open Source Tools for Uncertainty Enabling the Model Web Benjamin Proß University of Münster FOSS4G, Denver, 12-16 Sept 2011](https://reader035.vdocuments.us/reader035/viewer/2022070404/56649f335503460f94c508e2/html5/thumbnails/8.jpg)
• UncertML is a dictionary and encoding for uncertain information.– JSON and XML encoding– Provides support for distributions, statistics
and realisations.– Aim to cover a very wide range of uses: SWE,
SBML, Semantic Web
Uncertainty Model Language (UncertML)
![Page 9: Open Source Tools for Uncertainty Enabling the Model Web Benjamin Proß University of Münster FOSS4G, Denver, 12-16 Sept 2011](https://reader035.vdocuments.us/reader035/viewer/2022070404/56649f335503460f94c508e2/html5/thumbnails/9.jpg)
Profiles
![Page 10: Open Source Tools for Uncertainty Enabling the Model Web Benjamin Proß University of Münster FOSS4G, Denver, 12-16 Sept 2011](https://reader035.vdocuments.us/reader035/viewer/2022070404/56649f335503460f94c508e2/html5/thumbnails/10.jpg)
• Profiles:• specification or standard consisting of a set of
references to one or more base standards and/or other profiles, and the identification of any chosen conformance test classes, conforming subsets, options and parameters of those base standards, or profiles necessary to accomplish a particular function
• In other words:– Restricts a base standard to suite a certain application
purpose
Why Profiling?
![Page 11: Open Source Tools for Uncertainty Enabling the Model Web Benjamin Proß University of Münster FOSS4G, Denver, 12-16 Sept 2011](https://reader035.vdocuments.us/reader035/viewer/2022070404/56649f335503460f94c508e2/html5/thumbnails/11.jpg)
• Vector Data:– GML Profile: Restriction of geometry types– ISO 19139 extension for uncertainty– O&M Profile: Spatio-temporal and result
restrictions
• Raster Data:– NetCDF-U: extension of NetCDF for
uncertainty values
UncertWeb Profiles
![Page 12: Open Source Tools for Uncertainty Enabling the Model Web Benjamin Proß University of Münster FOSS4G, Denver, 12-16 Sept 2011](https://reader035.vdocuments.us/reader035/viewer/2022070404/56649f335503460f94c508e2/html5/thumbnails/12.jpg)
• Goal: provide easy-to-use and simplified profiles
• GML Profile used to encode vector-based geometries and features:
• Restrictions on Geometries• Restrictions on Metadata
• O&M Profile:• Uses GML Profile• Restrictions on result types• Common ways to encode uncertainties
UW Profiles – Vector Data
![Page 13: Open Source Tools for Uncertainty Enabling the Model Web Benjamin Proß University of Münster FOSS4G, Denver, 12-16 Sept 2011](https://reader035.vdocuments.us/reader035/viewer/2022070404/56649f335503460f94c508e2/html5/thumbnails/13.jpg)
• Spatial Geometries– Point– LineString– Polygon– Grid
• Multigeometries– Collection for each of the geometries defined above (e.g.
Multipoint, MultiLineString, etc.)
• Temporal Geometries:– TimeInstant, TimePeriod
• Uncertainty Property Type
GML Profile
![Page 14: Open Source Tools for Uncertainty Enabling the Model Web Benjamin Proß University of Münster FOSS4G, Denver, 12-16 Sept 2011](https://reader035.vdocuments.us/reader035/viewer/2022070404/56649f335503460f94c508e2/html5/thumbnails/14.jpg)
ISO 19139 Extension
• Extends DQ_QuantitativeResult with DQ_UncertaintyResult
• Used for resultQuality in O&M
![Page 15: Open Source Tools for Uncertainty Enabling the Model Web Benjamin Proß University of Münster FOSS4G, Denver, 12-16 Sept 2011](https://reader035.vdocuments.us/reader035/viewer/2022070404/56649f335503460f94c508e2/html5/thumbnails/15.jpg)
O&M Profile - restrictions
• Spatial restrictions• to SamplingFeature as defined in O&M
• Geometries of SamplingFeatures as defined in GML Profile
• Temporal Restrictions• TimeInstant and TimePeriod according to GML
profile
• resultQuality has to be ISO 19139 data quality
• Includes extension with Uncertainty result
![Page 16: Open Source Tools for Uncertainty Enabling the Model Web Benjamin Proß University of Münster FOSS4G, Denver, 12-16 Sept 2011](https://reader035.vdocuments.us/reader035/viewer/2022070404/56649f335503460f94c508e2/html5/thumbnails/16.jpg)
O&M Profile – Observation Subtypes
• Measurement:• result is double with uom info
• BooleanObservation• DiscreteNumericObservation
• Result is integer• TextObservation
• Result is String• UncertaintyObservation
• Result is UncertML AbstractUncertainty• ReferenceObservation
• Result is reference to e.g. file on a server
![Page 17: Open Source Tools for Uncertainty Enabling the Model Web Benjamin Proß University of Münster FOSS4G, Denver, 12-16 Sept 2011](https://reader035.vdocuments.us/reader035/viewer/2022070404/56649f335503460f94c508e2/html5/thumbnails/17.jpg)
O&M Profile - Collections
• Collection for each observation subtype– MeasurementCollection– BooleanObservationCollection– …
![Page 18: Open Source Tools for Uncertainty Enabling the Model Web Benjamin Proß University of Münster FOSS4G, Denver, 12-16 Sept 2011](https://reader035.vdocuments.us/reader035/viewer/2022070404/56649f335503460f94c508e2/html5/thumbnails/18.jpg)
• Network Common Data Format
• support the creation, access, and sharing of array-oriented scientific data
• Structure: Header + Body:– Header defines variables– Body contains binary-encoded variable values
NetCDF-U
![Page 19: Open Source Tools for Uncertainty Enabling the Model Web Benjamin Proß University of Münster FOSS4G, Denver, 12-16 Sept 2011](https://reader035.vdocuments.us/reader035/viewer/2022070404/56649f335503460f94c508e2/html5/thumbnails/19.jpg)
NetCDF-U Header Example
URLs to UncertMLDictionary
![Page 20: Open Source Tools for Uncertainty Enabling the Model Web Benjamin Proß University of Münster FOSS4G, Denver, 12-16 Sept 2011](https://reader035.vdocuments.us/reader035/viewer/2022070404/56649f335503460f94c508e2/html5/thumbnails/20.jpg)
• Goal: provide easy-to-use lightweight Java API for Information Models
• Is used in Web Service implementations for encoding/decoding of inputs and outputs
UncertWeb Java APIs
![Page 21: Open Source Tools for Uncertainty Enabling the Model Web Benjamin Proß University of Münster FOSS4G, Denver, 12-16 Sept 2011](https://reader035.vdocuments.us/reader035/viewer/2022070404/56649f335503460f94c508e2/html5/thumbnails/21.jpg)
Web Services
![Page 22: Open Source Tools for Uncertainty Enabling the Model Web Benjamin Proß University of Münster FOSS4G, Denver, 12-16 Sept 2011](https://reader035.vdocuments.us/reader035/viewer/2022070404/56649f335503460f94c508e2/html5/thumbnails/22.jpg)
Web Services and Models
• Models not developed with automation in mind.– Can require a user interface.
• Wrappers enable them to be called with code (e.g. Java) in the service interface.– Write input files, run, parse output files into usable objects.– Mechanisms for running each model may vary substantially.
![Page 23: Open Source Tools for Uncertainty Enabling the Model Web Benjamin Proß University of Münster FOSS4G, Denver, 12-16 Sept 2011](https://reader035.vdocuments.us/reader035/viewer/2022070404/56649f335503460f94c508e2/html5/thumbnails/23.jpg)
Service interface overview
• Open Geospatial Consortium (OGC):– WPS for processing (i.e. models).
• Including utility services for translating between uncertainty types.
– WCS, WFS, WMS, SOS for data access.
• W3C Web Services (WS):– SOAP for information exchange.– WSDL for service description.
![Page 24: Open Source Tools for Uncertainty Enabling the Model Web Benjamin Proß University of Münster FOSS4G, Denver, 12-16 Sept 2011](https://reader035.vdocuments.us/reader035/viewer/2022070404/56649f335503460f94c508e2/html5/thumbnails/24.jpg)
• UncertWeb Proxy Service (UPS)– Proxy for (not uncertainty-enabled) model-WPS– Executes Monte Carlo simulations
• Uncertainty Transformation Service (UTS)– Transforms Uncertainties– E.g. distributions to realisations
• Spatio Temporal Aggregation Service (STAS)– Transforms different data into a common scale
Use of WPS
![Page 25: Open Source Tools for Uncertainty Enabling the Model Web Benjamin Proß University of Münster FOSS4G, Denver, 12-16 Sept 2011](https://reader035.vdocuments.us/reader035/viewer/2022070404/56649f335503460f94c508e2/html5/thumbnails/25.jpg)
Uncertainty Transformation Service
• Transforms Uncertainties– E.g. distributions to realisations
• Can handle:– UncertML (UncertML parser/encoder)– NetCDF-U (UncertWeb binary
parser/encoder)– O&M
• R-backend
![Page 26: Open Source Tools for Uncertainty Enabling the Model Web Benjamin Proß University of Münster FOSS4G, Denver, 12-16 Sept 2011](https://reader035.vdocuments.us/reader035/viewer/2022070404/56649f335503460f94c508e2/html5/thumbnails/26.jpg)
UncertWeb Proxy Service
• MonteCarloSimulation process• Inputs:
– Identifier of the process to be executed by the model-WPS– Uncertain inputs for the process– Static inputs for the process– URL of the model-WPS– Output uncertainty type (e.g. distribution or realisations)– Number of realisations (i.e. monte carlo runs)
• Outputs:– The specified uncertainty type
![Page 27: Open Source Tools for Uncertainty Enabling the Model Web Benjamin Proß University of Münster FOSS4G, Denver, 12-16 Sept 2011](https://reader035.vdocuments.us/reader035/viewer/2022070404/56649f335503460f94c508e2/html5/thumbnails/27.jpg)
UPS
Monte Carlo Service (UPS)
Monte Carlo Service (UPS)
Samples Realisations
Model Service (WPS)
Model Service (WPS)
Model Implementation
Model Implementation
1 Sample 1 Realisation
1 Sample 1 Realisation
LOOP
![Page 28: Open Source Tools for Uncertainty Enabling the Model Web Benjamin Proß University of Münster FOSS4G, Denver, 12-16 Sept 2011](https://reader035.vdocuments.us/reader035/viewer/2022070404/56649f335503460f94c508e2/html5/thumbnails/28.jpg)
UPS + UTS
Monte Carlo Service (UPS)
Monte Carlo Service (UPS)
PDF PDF
Model Service (WPS)
Model Service (WPS)
Model Implementation
Model Implementation
1 Sample 1 Realisation
1 Sample 1 Realisation
LOOP
Uncertainty Transformatio
n(UTS)
Uncertainty Transformatio
n(UTS)
Realisations
PDFUncertainty Transformatio
n(UTS)
Uncertainty Transformatio
n(UTS)
Samples
![Page 29: Open Source Tools for Uncertainty Enabling the Model Web Benjamin Proß University of Münster FOSS4G, Denver, 12-16 Sept 2011](https://reader035.vdocuments.us/reader035/viewer/2022070404/56649f335503460f94c508e2/html5/thumbnails/29.jpg)
Local Air Quality Model Chain
• Demo chain with UncertWeb components
Uncertainty enabled Austal
model(UPS + WPS +
UTS)
Uncertainty enabled Austal
model(UPS + WPS +
UTS)
UncertML
JSON
O&M
Web-basedVisualisation
client
Web-basedVisualisation
client
INTAMAP service (WPS)
INTAMAP service (WPS)
Air quality observations
(SOS)
Air quality observations
(SOS)
UncertML+ GML
Overlay Service (WPS +
UTS)
Overlay Service (WPS +
UTS)
UncertML
realisations
Interpolation of background concentration
Estimation of air pollutionfrom local emissions at point locations
Adding both outputs to final concentration map
![Page 30: Open Source Tools for Uncertainty Enabling the Model Web Benjamin Proß University of Münster FOSS4G, Denver, 12-16 Sept 2011](https://reader035.vdocuments.us/reader035/viewer/2022070404/56649f335503460f94c508e2/html5/thumbnails/30.jpg)
USOS/UWCS
• Can handle:– UncertWeb O&M profile (USOS)– NetCDF-U (UWCS)
![Page 31: Open Source Tools for Uncertainty Enabling the Model Web Benjamin Proß University of Münster FOSS4G, Denver, 12-16 Sept 2011](https://reader035.vdocuments.us/reader035/viewer/2022070404/56649f335503460f94c508e2/html5/thumbnails/31.jpg)
SOAP/WSDL
• SOAP protocol.– Use in combination with information models
(GML, O&M).– Standard fault messages.
• WSDL service description.– Available operations.– Required structure of messages, both request
and response.– Where to find the service.
![Page 32: Open Source Tools for Uncertainty Enabling the Model Web Benjamin Proß University of Münster FOSS4G, Denver, 12-16 Sept 2011](https://reader035.vdocuments.us/reader035/viewer/2022070404/56649f335503460f94c508e2/html5/thumbnails/32.jpg)
SOAP/WSDL
• Advantages:– Widely adopted by the rest of the web.– Tool support for generating code.– Compatibility with workflow software
(Taverna, Kepler) and orchestration engines.
• Disadvantages:– Lack of semantics.– Not fully adopted by the OGC.
![Page 33: Open Source Tools for Uncertainty Enabling the Model Web Benjamin Proß University of Münster FOSS4G, Denver, 12-16 Sept 2011](https://reader035.vdocuments.us/reader035/viewer/2022070404/56649f335503460f94c508e2/html5/thumbnails/33.jpg)
JSON
• A lightweight format for exchanging data.
• Preferred over XML for JavaScript client development.
• Used by Google, Facebook, Twitter...
![Page 34: Open Source Tools for Uncertainty Enabling the Model Web Benjamin Proß University of Münster FOSS4G, Denver, 12-16 Sept 2011](https://reader035.vdocuments.us/reader035/viewer/2022070404/56649f335503460f94c508e2/html5/thumbnails/34.jpg)
Applying the technology
• Web service supporting two interfaces: SOAP/WSDL and JSON.
• Web service interface sits separately from model.• Wrappers need to be developed so the interface
can communicate with the model.
![Page 35: Open Source Tools for Uncertainty Enabling the Model Web Benjamin Proß University of Münster FOSS4G, Denver, 12-16 Sept 2011](https://reader035.vdocuments.us/reader035/viewer/2022070404/56649f335503460f94c508e2/html5/thumbnails/35.jpg)
Other tools
![Page 36: Open Source Tools for Uncertainty Enabling the Model Web Benjamin Proß University of Münster FOSS4G, Denver, 12-16 Sept 2011](https://reader035.vdocuments.us/reader035/viewer/2022070404/56649f335503460f94c508e2/html5/thumbnails/36.jpg)
• Composition as a Service
• Offers functionality to– Create uncertainty enabled model chains– Execute them
CaaS
![Page 37: Open Source Tools for Uncertainty Enabling the Model Web Benjamin Proß University of Münster FOSS4G, Denver, 12-16 Sept 2011](https://reader035.vdocuments.us/reader035/viewer/2022070404/56649f335503460f94c508e2/html5/thumbnails/37.jpg)
Visualization tool
• Different visualizations of uncertainties in spatio-temporal data
![Page 38: Open Source Tools for Uncertainty Enabling the Model Web Benjamin Proß University of Münster FOSS4G, Denver, 12-16 Sept 2011](https://reader035.vdocuments.us/reader035/viewer/2022070404/56649f335503460f94c508e2/html5/thumbnails/38.jpg)
Elicitator
• Expert elicitation of uncertainties
• E.g. parameters of distribution functions– Mean and standard deviation of a Gaussian
normal distibution
![Page 39: Open Source Tools for Uncertainty Enabling the Model Web Benjamin Proß University of Münster FOSS4G, Denver, 12-16 Sept 2011](https://reader035.vdocuments.us/reader035/viewer/2022070404/56649f335503460f94c508e2/html5/thumbnails/39.jpg)
Sensitivity Analysis tool
• Performs sensitivity analysis– Identify the influence of the different inputs to
the model output
![Page 40: Open Source Tools for Uncertainty Enabling the Model Web Benjamin Proß University of Münster FOSS4G, Denver, 12-16 Sept 2011](https://reader035.vdocuments.us/reader035/viewer/2022070404/56649f335503460f94c508e2/html5/thumbnails/40.jpg)
Links
• http://www.uncertweb.org/
• http://uncertml.org/
• https://svn.52north.org/svn/geostatistics/main/uncertweb
![Page 41: Open Source Tools for Uncertainty Enabling the Model Web Benjamin Proß University of Münster FOSS4G, Denver, 12-16 Sept 2011](https://reader035.vdocuments.us/reader035/viewer/2022070404/56649f335503460f94c508e2/html5/thumbnails/41.jpg)
Thank you
Questions?
The research leading to these results has received funding from the European Union Seventh Framework Programme (FP7/2007-2013) under grant agreement n° [248488].