Download - DIR workshop ontology stream data access
Date: 16/02/2011
Ontology-based Access to Streaming Data Sources
Jean-Paul Calbimonte
Ontology Engineering GroupFacultad de Informática, Universidad Politécnica de Madrid.
Data Intensive Research WorkshopEdinburgh, UK
Introduction & Scope
2
• Streaming Data(t9, a1, a2, ... , an)(t8, a1, a2, ... , an)(t7, a1, a2, ... , an)......(t1, a1, a2, ... , an)......
Streaming Data
• Continuously appended data• Potentially infinite• Time-stamped tuples• Continuous queries• Changes of values over time• Latest used in queries
• Ubiquitous data capture• Data processing • Cheap• Noisy, Unreliable• Low computational, power resources, storage
• Sensor technologies
Enabling Ontology-based Access to Streaming Data Sources
• Applications in security surveillance, healthcare provision, environmental monitoring, you name it.
Motivation
3Enabling Ontology-based Access to Streaming Data Sources
Emergency planner
Flood risk alert: South East England
...
...
...
Real-time data
Meteorological forecasts
Flood defences data
Other sources
• Detect conditions likely to cause a flood• Present data model in terms of the user domain: e.g. Flood risk assessment
Example:• “provide me with the wind speed observations average over the last minute in
the Solent region, if it is higher than the average of the last 2 to 3 hours”
Wave,Wind,Tide
Motivation
4Enabling Ontology-based Access to Streaming Data Sources
• Ontologies can be used as such a common model• Achieve logical transparency in access to data.• Hide to the user where and how data are stored.• Present to the user a conceptual view of the data.• Use a semantically rich formalism for the conceptual
view.
[Calvanese2007]
• Need to provide solution for:• Establish mappings between ontological models and
streaming data source schemas• Access streaming data sources through queries over
ontology models
Background – Ontology-based Data Access
5Enabling Ontology-based Access to Streaming Data Sources
Ontology-based Data Access
R2O + ODEMapsterD2RQSquirrelRDFRDBToOntoRelational.OWLSPASQLVirtuosoMASTRO
Query/Data Transformation
Ontological Models
Relational Schemas
Mapping Document
Generate Semantic Web content from existing relational data sources
e.g. SPARQL
e.g. SQL
Linked DataRDF
Web of Data
6Enabling Ontology-based Access to Streaming Data Sources
Background – Querying Relational Data Streams
7Enabling Ontology-based Access to Streaming Data Sources
Streaming Data
STREAMAurora/BorealisCougarTinyDBSNEE
e1
e2e1
e2 e3
e2e1 e3
e1
e4
t t+1 t+2 t+3 t+4 t+5
WINDOW [tnow TO tnow-2] SLIDE 1
Transform infinite sequence of tuples to bounded bag
Window-to-Stream operators: convert stream of windows to stream of tuples
...
...
CQLSNEEqlTinyQL
Query engines
Query languages
Event Streams/Acquisitional Streams
Ontology-based Streaming Data Access
8Enabling Ontology-based Access to Streaming Data Sources
Querytranslation
Query Evaluator
Clie
nt Stream-to-Ontologymappings
SPARQLStream (Og)
[tuples]
Stream Engine (S3)
Ontology-based Streaming Data Access Service
RelationalDB (S2)
Sensor Network (S1)
RDF Store(Sm)
SPARQLStream algebra(S1 S2 Sm)
Datatranslation
q
[triples]
SNEEql
Ontology-based Streaming Data Access
9Enabling Ontology-based Access to Streaming Data Sources
• Mappings from relational streams to ontological concepts• Extend stored data schema mappings• Study translation semantics
• Provide with a stream query language at ontological level• Use notion of RDF stream• Extend SPARQL• Window operator, window-to-stream operators
SPARQLStream
10Enabling Ontology-based Access to Streaming Data Sources
• RDF-Stream ......( <si-1,pi-1, oi-1>, ti-1 ),( <si, pi, oi>, ti ),( <si+1,pi+1, oi+1>, ti+1 ),......
Example:• “provide me with the wind speed observations over the last minute in the Solent Region ”
cd:Observation
xsd:double
cd:observationResult......( <ssg4e:Obs1,rdf:type, cd:Observation>, ti ),( <ssg4e:Obs1,cd:observationResult,”34.5”>, ti ),( <ssg4e:Obs2,rdf:type, cd:Observation>, ti+1 ),( <ssg4e:Obs2,cd:observationResult,”20.3”>, ti+1 ),......
STREAM <http://www.semsorgrid4env.eu/ccometeo.srdf>
SPARQLStream
11Enabling Ontology-based Access to Streaming Data Sources
Example:• “provide me with the wind speed observations over the last minute in the Solent Region ”
cd:Observation
xsd:double
cd:observationResult
PREFIX cd: <http://www.semsorgrid4env.eu/ontologies/CoastalDefences.owl#>PREFIX sb: <http://www.w3.org/2009/SSN-XG/Ontologies/SensorBasis.owl#> PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> SELECT ?windspeed ?windtsFROM STREAM <http://www.semsorgrid4env.eu/ccometeo.srdf> [ NOW – 1 MINUTE TO NOW MINUTES ] WHERE { ?WindObs a cd:Observation;
cd:observationResult ?windspeed; cd:observationResultTime ?windts;cd:observedProperty ?windProperty;cd:featureOfInterest ?windFeature.
?windFeature a cd:Feature;cd:locatedInRegion cd:SolentCCO.
?windProperty a cd:WindSpeed. }
cd:Feature
cd:featureOfInterest
cd:Property
cd:observedProperty
cd:locatedInRegion
cd:Region
Ontology-based Streaming Data Access
12Enabling Ontology-based Access to Streaming Data Sources
Querytranslation
Query Evaluator
Clie
nt Stream-to-Ontologymappings
SPARQLSream (Og)
[tuples]
Stream Engine (S3)
Ontology-based Streaming Data Access Service
RelationalDB (S2)
Sensor Network (S1)
RDF Store(Sm)
SPARQLSream algebra(S1 S2 Sm)
Datatranslation
q
[triples]
SNEEql
S2O Mappings
13Enabling Ontology-based Access to Streaming Data Sources
<conceptmap-def id="Observation_wind"name="http://www.semsorgrid4env.eu/ontologies/CoastalDefences.owl#Observation" virtualStream="http://www.semsorgrid4env/ccometeo.srdf"><uri-as>
<operation oper-id="concat"><arg-restriction on-param="string1">
<has-value>http://www.semsorgrid4env.eu/data#ObservationWind</has-value></arg-restriction><arg-restriction on-param="string2">
<has-column>meteostream.DateTime</has-column></arg-restriction>
</operation></uri-as>
<attributemap-def name="http://www.semsorgrid4env.eu/ontologies/CoastalDefences.owl#observationResult" dataType="xsd:double"><selector><aftertransform><operation oper-id="constant"><arg-restriction on-param="const-val"><has-column>meteostream.Hs</has-column>
</arg-restriction></operation>
</aftertransform></selector></attributemap-def>
cd:Observation
xsd:double
cd:observationResult
meteostream
Datetime: longHs : floatLon: floatLat: float
S2O Mappings
14Enabling Ontology-based Access to Streaming Data Sources
cd:Observation
xsd:double
cd:observationResult envdata_hornsea
Datetime: longHs : floatLon: floatLat: float
envdata_westbay
Datetime: longHs : floatLon: floatLat: float
envdata_milford
Datetime: longHs : floatLon: floatLat: float
envdata_rhylflats
Datetime: longHs : floatLon: floatLat: float
U
U
U
<union name="meteostream"><map value="http://www.semsorgrid4env.eu/ontologies/
CoastalDefences.owl#SolentCCO"><extent name="envdata_hornsea"></extent><extent name="envdata_rhylflats"></extent><extent name="envdata_milford"></extent><extent name="envdata_westbay"></extent>
</map> </union>
• Union of extents-streams
Ontology-based Streaming Data Access
15Enabling Ontology-based Access to Streaming Data Sources
Querytranslation
Query Evaluator
Clie
nt Stream-to-Ontologymappings
SPARQLSream (Og)
[tuples]
Stream Engine (S3)
Ontology-based Streaming Data Access Service
RelationalDB (S2)
Sensor Network (S1)
RDF Store(Sm)
SPARQLSream algebra
Datatranslation
q
[triples]
SNEEql
Query Translation
16Enabling Ontology-based Access to Streaming Data Sources
• Queries: SELECT ?y WHERE{ ?x a cd:Observation;
cd:observationResult ?y. }
SELECT ?yFROM STREAM < STREAM <http://www.semsorgrid4env.eu/ccometeo.srdf> [ NOW – 1 MINUTE TO NOW MINUTES ] >WHERE{ ?x a cd:Observation;
cd:observationResult ?y. }
Query Translation
17Enabling Ontology-based Access to Streaming Data Sources
• Mappings: query algebra expression
over sources
cd:Observation
xsd:double
cd:observationResult
envdata_milford
Datetime: longHs : floatLon: floatLat: float
envdata_milford
DatetimeHs
Query Translation
18Enabling Ontology-based Access to Streaming Data Sources
envdata_rhylflats
DateTimeHs
cd:Observation
xsd:double
cd:observationResult
envdata_hornsea
Datetime: longHs : floatLon: floatLat: float
envdata_westbay
Datetime: longHs : floatLon: floatLat: float
envdata_milford
Datetime: longHs : floatLon: floatLat: float
envdata_rhylflats
Datetime: longHs : floatLon: floatLat: float
U
U
U
envdata_milford
DateTimeHs
envdata_hornsea
DateTimeHs
envdata_westbay
DateTimeHs
DateTimeHs
Ontology-based Streaming Data Access
19Enabling Ontology-based Access to Streaming Data Sources
Querytranslation
Query Evaluator
Clie
nt Stream-to-Ontologymappings
SPARQLSream (Og)
[tuples]
Stream Engine (S3)
Ontology-based Streaming Data Access Service
RelationalDB (S2)
Sensor Network (S1)
RDF Store(Sm)
SPARQLSream algebra
Datatranslation
q
[triples]
SNEEql
vv
vv
Query Execution
20
PREFIX cd: <http://www.semsorgrid4env.eu/ontologies/CoastalDefences.owl#>PREFIX sb: <http://www.w3.org/2009/SSN-XG/Ontologies/SensorBasis.owl#> PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> SELECT ?waveheight ?wavets ?lat ?lonFROM STREAM <http://www.semsorgrid4env/ccometeo.srdf> WHERE { ?WaveObs a cd:Observation;
cd:observationResult ?waveheight; cd:observationResultTime ?wavets;cd:observationResultLatitude ?lat;cd:observationResultLongitude ?lon;cd:observedProperty ?waveProperty;cd:featureOfInterest ?waveFeature.
?waveFeature a cd:Feature;cd:locatedInRegion cd:SouthEastEnglandCCO.
?waveProperty a cd:WaveHeight. }
(SELECT Lon,timestamp,Hs,Lat FROM envdata_rhylflats) UNION (SELECT Lon,timestamp,Hs,Lat FROM envdata_hornsea) UNION (SELECT Lon,timestamp,Hs,Lat FROM envdata_milford) UNION (SELECT Lon,timestamp,Hs,Lat FROM envdata_chesil) UNION (SELECT Lon,timestamp,Hs,Lat FROM envdata_perranporth) UNION (SELECT Lon,timestamp,Hs,Lat FROM envdata_westbay) UNION (SELECT Lon,timestamp,Hs,Lat FROM envdata_pevenseybay)
envdata_rhylflats
Timestamp: longHs : floatLon: floatLat: float
envdata_hornsea
Observation
WaveHeightProperty
observedProperty
hasObservationResult
xsd:float
locatedInRegion
Ontologies Streams
S2O Mapping
envdata_milford
envdata_chesil
envdata_westbay
Region
Feature
SPARQLStreamSNEEql
Enabling Ontology-based Access to Streaming Data Sources
Ontology-based Streaming Data Access
21Enabling Ontology-based Access to Streaming Data Sources
Querytranslation
Query Evaluator
Clie
nt Stream-to-Ontologymappings
SPARQLSream (Og)
[tuples]
Stream Engine (S3)
Ontology-based Streaming Data Access Service
RelationalDB (S2)
Sensor Network (S1)
RDF Store(Sm)
SPARQLSream algebra
Datatranslation
q
[triples]
SNEEql
Data Translation
• Data translation:Tagged tuples SPARQL bound variables
<ns9:sparql><ns9:head>
<ns9:variable name="waveheight"/><ns9:variable name="wavets"/>
</ns9:head><ns9:results>
<ns9:result><ns9:binding name="waveheight">
<ns9:literal datatype="http://www.w3.org/2001/XMLSchema#double">4.850</ns9:literal></ns9:binding><ns9:binding name="wavets">
<ns9:literal datatype="http://www.w3.org/2001/XMLSchema#long">1272588663</ns9:literal></ns9:binding>
</ns9:result><ns9:result><ns9:binding name="waveheight">
<ns9:literal datatype="http://www.w3.org/2001/XMLSchema#double">2.1230</ns9:literal></ns9:binding><ns9:binding name="wavets">
<ns9:literal datatype="http://www.w3.org/2001/XMLSchema#long">1272587400</ns9:literal></ns9:binding>
</ns9:result></ns9:results></ns9:sparql>
waveheight wavets4.850 12725886632.1230 1272587400
R2RML Mappings
• W3C RDB2RDF Working Group
• :waveObsResult rr:predicateMap [ rr:predicate ssg:observationResult ]; • rr:objectMap [ rr:column "Hs" ].• :waveObsTime rr:predicateMap [ rr:predicate ssg:observationResultTime ]; • rr:objectMap [ rr:column "timestamp" ].• :waveFoISea rr:predicateMap [ rr:predicate ssg:featureOfInterest ]; • rr:objectMap [ rr:object ssg:Sea ].• :waveObsProp rr:predicateMap [ rr:predicate ssg:observedProperty ]; • rr:objectMap [ rr:object ssg:WaveHeight ]. •• :MilfordWaveObservation a rr:TriplesMapClass;• rr:SQLQuery "";• rr:subjectMap [ rr:column "envdata_milford.DateTime"; rr:class ssg:Observation; rr:graph
ssg:ccometeo.srdf ];• rr:tableName "envdata_milford";• rr:predicateObjectMap :waveObsResult;• rr:predicateObjectMap :waveObsTime;• rr:predicateObjectMap :waveFoISea;• rr:predicateObjectMap :waveObsProp;
23Enabling Ontology-based Access to Streaming Data Sources
SemSorGrid4Env EU Project
1. Universidad Politécnica de Madrid, (UPM,
Spain)
2. University of Manchester (UNIMAN, UK)
3. National and Kapodistrian University of
Athens (NKUA, Greece)
4. University of Southampton (SOTON, UK)
5. Deimos Space SLU (DMS, Spain)
6. EMU Ltd. (EMU, UK)
7. TechIdeas (TI, Spain)
24
3
3
1
24Enabling Ontology-based Access to Streaming Data Sources
Project Challenges
• Integrated information space • Discovery new sensor networks• Integrate with existing ones • Integrate possibly other data sources
(e.g., historical databases)
020406080
100
1ertrim.
3ertrim.
EsteOesteNorte
s e n s o r n e tw o rk s
le g a c y d a ta s o u rc e s
s e m a n tic d a ta in te g ra tio n a n d q u e ry in g
th in a p p lic a tio n s (m a s h u p s )
re g is trie s
m id d le w a re
• Rapid development • flexible and user-centric decision
support systems • Use data from multiple
autonomous independently deployed sensor networks and other applications.
25Enabling Ontology-based Access to Streaming Data Sources
Implementation
26Enabling Ontology-based Access to Streaming Data Sources
• Design, implement and deploy a Semantic Integration Service
• Extend existing ontology-based data integration models to take into account sensor networks streaming data, semantic heterogeneity and quality of service
Implementation
27Enabling Ontology-based Access to Streaming Data Sources
Integration Query Service
IntegrationInterface
QueryInterface
PullInterface
Translator
ClientSNEE-WS
S2O mappingsrepository
SNEEql
IntegrateAs (DataResourceAddressList, S2O document)
DataResourceAddress
SPARQLQueryFactory (DataResourceName, SPARQLSTR )
DataResourceAddressaddQuery
GetStreamNewestItem(DataResourceName )
DataSetgetResultSetrepeat
CCO-WS
Implementation
• Flood Warning Application
28Enabling Ontology-based Access to Streaming Data Sources
- Wave Height updated every 10 min - Live data from sensor in buoys - Over 30 deployments
Conclusions
Ontology-base data access• Define stream extensions for R2O• Define SPARQLStream language syntax and semantics• Define S2O mappings-based translation semantics
Implementation• Enable engine support for « S2O » documents,
SPARQLStream queries• Enabled engine support for SNEEql translation and
connection• Limited to non-distributed scenario initially
29Enabling Ontology-based Access to Streaming Data Sources
30
Future Work
• Ontology-based data access• SPARQL construct expressions, aggregates, projected
operators• Implement adapters for other streaming sources• Add query rewriting algorithms
• Ontology-based streaming data integration• Horizontal & vertical integration• Integrate streaming + stored data• RDF data sources integration
• Streaming query optimization• Analyze cost models• Streaming sources statistics and metadata
• Quantitative evaluation
30Enabling Ontology-based Access to Streaming Data Sources
31Enabling Ontology-based Access to Streaming Data Sources
Planet Data
• EU FP7 Network of Excellence
• Working with EPFL Global Sensor Network
• SwissExperiment Data
Co-authors of this work
• Oscar Corcho – Universidad Politécnica de Madrid• Alasdair Gray – The University of Manchester
• Supported by the SemSorGrid4Env FP7-223913
32Enabling Ontology-based Access to Streaming Data Sources
Thanks!
33
Ontology-based Access toStreaming Data Sources
Jean-Paul [email protected]
Enabling Ontology-based Access to Streaming Data Sources