practical issues of sensor web implementation and gridification kussul n., korbakov m., kravchenko o
TRANSCRIPT
Practical issues of Sensor Web Practical issues of Sensor Web implementation and implementation and gridificationgridification
Kussul N., Korbakov M., Kravchenko O.
OutlookOutlook
• Sensor Web: overview
• Test case: floodings
• SensorML: experience
• Sensor Observation Service: experience
• Sensor Web: gridification
• Our plans
Sensor Web: the purposeSensor Web: the purpose
• Integration of heterogeneous sensors into the information infrastructure
• Sensors discovery and data access
• Composition of dataflows between system components
• Events triggering by sensors conditions
OGC O&M Observations & Measurements Approved
SensorML Sensor Model Language Approved
TransducerML Transducer Model Language Approved
OGC SOS Sensor Observations Service Approved
OGC SPS Sensor Planning Service Approved
OGC SAS Sensor Alert Service In progress
OGC WNS Web Notification Services In progress
OpenGIS StandardsOpenGIS Standards
• SW Enablement working group at OGC have developed a number of standards governing different aspects of Sensor Web
Test CaseTest Case
• The task under study is flooding in different regions of world (requested by Red Cross )
• Particular test case is floodings in Mozambique
Test case: data flowTest case: data flow
Internet
RSGSESA LAADS
Data Storage
Envisat, ERS-2 Radarsat MODIS
USGS EarthExplorer
Landsat
RSGS Grid
Envisat/ASAR WSM&GM
Processing subsystem
Optical-based floodextent extraction
SAR-based flood extentextraction
Water bodiescartography
Visualization subsystem
UMN MapServerWeb-server/OpenLayers
Internet
ESA G-POD
UA Space Grid
InterGrid
Users
EUMETCastMSG, MetOp
Data integration
Test Case: data sourcesTest Case: data sources
• ASAR
• MODIS
• MERIS
• LandSat
• DEM
Weather model
SOS
SOS
SPS
Hydrologicalmodel
SOS
Sim
ulat
ion
data
Sim
ulat
ion
data
Order
SOS
Weatherstation
SOS
Hydrologicalstation
Mea
sure
men
ts
Mea
sure
men
ts
SAS
Test Case: SW perspectiveTest Case: SW perspective
Test Case: MozambiqueTest Case: Mozambique
http://floods.ikd.kiev.ua
Test Case: MozambiqueTest Case: Mozambique
SensorMLSensorML
• Sensor modeling language is the cornerstone of all SW services
• It provides comprehensive description of sensor parameters and capabilities
• It can be used for describing different kind of sensors:– Stationary or dynamic– Remote or in-situ– Physical measurements or simulations
SensorML: exampleSensorML: example
..............<inputs> <InputList> <input name="ambiantTemperature"> <swe:Quantity definition= "urn:ogc:def:phenomenon:temperature"/> </input> <input name="atmosphericPressure"> <swe:Quantity definition= "urn:ogc:def:phenomenon:pressure"/> </input> <input name="windSpeed"> <swe:Quantity definition= "urn:ogc:def:phenomenon:windSpeed"/> </input></InputList></inputs>..............
.............<outputs> <OutputList> <output name="weatherMeasurements"> <swe:DataGroup> <swe:component name="time"> <swe:Time definition="urn:ogc:def:phenomenon:time“ uom="urn:ogc:def:unit:iso8601"/> </swe:component> <swe:component name="temperature"> <swe:Quantitydefinition="urn:ogc:def:phenomenon:temperature uom="urn:ogc:def:unit:celsius"/> </swe:component> <swe:component name="barometricPressure"> <swe:Quantity definition="urn:ogc:def:phenomenon:pressure“ uom="urn:ogc:def:unit:bar" scale="1e-3"/> </swe:component> <swe:component name="windSpeed"> <swe:Quantity definition="urn:ogc:def:phenomenon:windSpeed“ uom="urn:ogc:def:unit:meterPerSecond"/> </swe:component> </swe:DataGroup> </output> </OutputList></outputs>.............
SensorML: WRF modelSensorML: WRF model
• Modeling and simulation are very important parts of environmental monitoring
• Sensor Web infrastructure should be able to integrate modeling data in convenient way
• We have tried to describe weather modeling process using WRF numerical model in terms of SensorML
An example of single model input in SensorML:
<sml:input name="QVAPOR"> <swe:DataArray definition="urn:ogc:def:phenomenon:time"> <swe:elementCount> <swe:Count definition="urn:ogc:def:property:OGC:numberOfPixels"><swe:value>1</swe:value></swe:Count> </swe:elementCount> <swe:elementType name=""> <swe:DataArray definition="urn:ogc:def:phenomenon:altitude"> <swe:elementCount> <swe:Count definition="urn:ogc:def:property:OGC:numberOfPixels"><swe:value>30</swe:value></swe:Count> </swe:elementCount> <swe:elementType name=""> <swe:DataArray definition="urn:ogc:def:phenomenon:latitude"> <swe:elementCount> <swe:Count definition="urn:ogc:def:property:OGC:numberOfPixels"><swe:value>202</swe:value></swe:Count> </swe:elementCount> <swe:elementType name=""> <swe:DataArray definition="urn:ogc:def:phenomenon:longtitude"> <swe:elementCount> <swe:Count definition="urn:ogc:def:property:OGC:numberOfPixels"><swe:value>219</swe:value></swe:Count> </swe:elementCount> <swe:elementType name=""> <swe:Quantity definition="urn:ogc:def:phenomenon:QVAPOR"><swe:uom code="kg_kg-1"/></swe:Quantity> </swe:elementType> </swe:DataArray> </swe:elementType> </swe:DataArray> </swe:elementType> </swe:DataArray> </swe:elementType> </swe:DataArray></sml:input>
SensorML: WRF modelSensorML: WRF model
Axis descriptionsAxis descriptions
Value description
SensorML: WRF modelSensorML: WRF model
• There are nearly 50 inputs and 20 outputs for basic WRF configuration
• Each of them requires quite significant amount of XML code to be properly described
• It would be great if next revision of SensorML will include some elements for simpler description of multidimensional data
• Another negative issue is inconsistency between SML specification, published XML schemas and educational materials
Sensor Observation ServiceSensor Observation Service
• We have studied two possible implementations of Sensor Observation Service (SOS) for serving temperature sensors data
• Implementations under study were:– UMN Mapserver v5 (http://mapserver.gis.umn.edu/)– 52North SOS (http://52north.org/)
• The conclusion of study follows: there isn’t (yet) really good and reliable solution for serving data through SOS protocol
• However for some cases 52North’s implementation provides good experience
Sensor Observation ServiceSensor Observation Service
• UMN Mapserver (as SOS server)– Pros:
• Very good and reliable abstraction for different data sources (raster files, spatial databases, WFS, etc)
• Simple application model (CGI executable)• Wide set of features beside SOS• Open software
– Cons:• SOS support is declared but far from being working
implementation• Poor documentation on SOS topic• Strange plans for future development (automatic
SensorML generation)
Sensor Observation ServiceSensor Observation Service
• 52North SOS– Pros:
• SOS implementation is stable and complete• Platform-independent (Java-based)• A part of wider SW implementations stack (SPS, SAS)• Open software• Source code is clean and easily reusable
– Cons:• No data abstraction: the only data source is relational
database of specific structure• Database structure is far from optimal (strings as primary
keys, missed indexes, etc)• Complex application model (Java web application)
Sensor Observation ServiceSensor Observation Service
• We have used 52North implementation for building a testbed SOS server:– http://web.ikd.kiev.ua:8080/52nsos/sos
• Server is providing data of temperature sensors over Ukraine and South Africa region
• Data comes from PostGIS database with some tweaks to make is compatible with 52North database structure (VIEWS, index tables, etc)
• Performance is quite good for our DB. Yet, for other DBs such adaptations could lead to unacceptable drops in performance
Sensor Observation ServiceSensor Observation Service
• The database accessed through SOS contains more than 2 millions records
• Served by PostgreSQL with PostGIS extensions• We have used a number of VIEWS and stored
procedures to adapt it to 52North• Queries are complex:
SELECT observations."time" AS time_stamp, "procedure".procedure_id, feature_of_interest.feature_of_interest_id, phenomenon.phenomenon_id, offering.offering_id, '' AS text_value, observations.t AS numeric_value, '' AS mime_type, observations.oid AS observation_id FROM observations, "procedure", proc_foi, feature_of_interest, proc_off, offering_strings offering, foi_off, phenomenon, proc_phen, phen_off WHERE "procedure".procedure_id::text = proc_foi.procedure_id::text AND proc_foi.feature_of_interest_id::text = feature_of_interest.feature_of_interest_id AND "procedure".procedure_id::text = proc_off.procedure_id::text AND proc_off.offering_id::text = offering.offering_id::text AND foi_off.offering_id::text = offering.offering_id::text AND foi_off.feature_of_interest_id::text = feature_of_interest.feature_of_interest_id AND proc_phen.procedure_id::text = "procedure".procedure_id::text AND proc_phen.phenomenon_id::text = phenomenon.phenomenon_id::text AND phen_off.phenomenon_id::text = phenomenon.phenomenon_id::text AND phen_off.offering_id::text = offering.offering_id::text AND observations.wmoid::text = feature_of_interest.feature_of_interest_id;
Sensor Observation ServiceSensor Observation Service
• Despite of complex queries and large datasets SOS service shows linear scalability with respect to query depth:
0
200
400
600
800
1000
1200
1400
1600
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24
Months
ms
Sensor Observation ServiceSensor Observation Service
Sensor Observation ServiceSensor Observation Service
• Example of single SOS measurement...
<om:Measurement gml:id="o255136"> <om:samplingTime> <gml:TimeInstant xsi:type="gml:TimeInstantType"> <gml:timePosition>2005-04-14T04:00:00+04</gml:timePosition> </gml:TimeInstant> </om:samplingTime> <om:procedure xlink:href="urn:ogc:object:feature:Sensor:WMO:33506"/> <om:observedProperty xlink:href="urn:ogc:def:phenomenon:OGC:1.0.30:temperature"/> <om:featureOfInterest> <sa:Station gml:id="33506"> <gml:name>WMO33506</gml:name> <sa:sampledFeature xlink:href=""/> <sa:position> <gml:Point> <gml:pos srsName="urn:ogc:crs:epsg:4326">34.55 49.6</gml:pos> </gml:Point> </sa:position> </sa:Station> </om:featureOfInterest> <om:result uom="celsius">10.9</om:result> </om:Measurement>
Sensor Observation ServiceSensor Observation Service
• ... and the whole time serie of observations
<om:result>2005-03-14T21:00:00+03,33506,-5@@2005-03-15T00:00:00+03,33506,-5.2@@2005-03-15T03:00:00+03,33506,-5.5@@2005-03-15T06:00:00+03,33506,-4.6@@2005-03-15T09:00:00+03,33506,-2.2@@2005-03-15T12:00:00+03,33506,1.7@@2005-03-15T15:00:00+03,33506,1.7@@2005-03-15T18:00:00+03,33506,2.4@@2005-03-15T21:00:00+03,33506,-0.7@@2005-03-16T00:00:00+03,33506,-1.4@@2005-03-16T03:00:00+03,33506,-1.1@@2005-03-16T06:00:00+03,33506,-1.1@@2005-03-16T09:00:00+03,33506,-1.3@@2005-03-16T12:00:00+03,33506,0.5@@2005-03-16T15:00:00+03,33506,1.7@@2005-03-16T18:00:00+03,33506,1.5@@</om:result>
Gridification: rationaleGridification: rationale
• Sensor Web services like SOS, SPS and SAS can benefit from integration with Grid platform like Globus Toolkit
• Advantages includes:– Convenient way for implementation of notifications
and event triggering– Sensors discovery through Index Service– High-level access to XML description– Reliable data transfer for large datasets– Enforcement of data and services access policies
Gridification: implementationGridification: implementation
• We have developed a testbed SOS service using the Globus Toolkit platform
• For now, service works as proxy translating and redirecting user request to usual SOS-server
Intranet
SOS Server DatabaseGrid Server/SOS Service
Gridification: implementationGridification: implementation
• We have developed a testbed SOS service using the Globus Toolkit platform
• For now, service works as proxy translating and redirecting user request to usual SOS-server
• Next version should have in-service implementation of SOS-server functionality
Intranet
SOS Server DatabaseGrid Server/SOS Service
Gridification: problemsGridification: problems
• The main problem of implementation of OGC Grid service lies in complexity of XML schema used
• According to OGC SOAP Interoperability Experiment, none of available SOAP binding tools were able to parse OGC schemas completely (year 2003)
• Situation haven’t improved significantly till now
• The main problem of complexity is GML data types
Gridification: problemsGridification: problems
• This problems could be solved by using custom serializers for services XML data or by discarding high-level access to XML data in service implementation
• Custom serialization is complex in implementation and debugging
• Discarding high-level OO access to XML is applicable for services acting like proxies to other XML-generating facilities
• Let’s hope that situation will improve from both sides
Gridification: eventsGridification: events
• GT4 middleware provides Trigger Service that can be used for triggering events basing specific conditions
• We have implemented additional request to SOS Grid service that registers specific observation for periodic update and exports it
• After registering observation user can pass it to the Trigger Service together with specific condition (in form of XPath expression) and an action to be fired
Our plansOur plans
Our future works include:
• Implementation of Mozambique test case in terms of Sensor Web
• To participate in IC "Space and Major Disasters“ with architectural proposals
• To provide stable Grid-based implementation of Sensor Web services
• To collaborate with International Red Cross organization within it’s tasks
Our plans: Red Cross tasksOur plans: Red Cross tasks
ThemeTheme--Based Flood Product Generation for I FRCBased Flood Product Generation for I FRC1
From portal select desired theme(s) and area of interest
Wizard picks appropriate workflow for desired result
Wizard
Mozambique
Disaster Management Information System (DMIS)
Workflows
Estimated rainfall accumulation and flood prediction model
Flood Model
Selected workflow automatically activates needed assets and models
Baseline water level, flood waters and predicted flooding
Thank you!