xsparql credible workshop

www.insight)centre.org.

XML Data Mediation usingXSPARQL

Nuno Lopes, Laleh Kazemzadeh

October, 2013

Why use RDF for data integration (I)

Flexibility

1 Representation

Reuse any vocabulariesNo schema required

2 Combining

Easily combine different datasetsRDF merge is simple

3 Sharing

Linked DataBuilt on web technologies (HTTP, URIs)

1 / 21

Flexibility

1 Representation

Reuse any vocabulariesNo schema required

2 Combining

3 Sharing

1 / 21

Flexibility

1 RepresentationReuse any vocabulariesNo schema required

2 Combining

3 Sharing

1 / 21

Flexibility

2 CombiningEasily combine different datasetsRDF merge is simple

3 Sharing

1 / 21

Flexibility

2 CombiningEasily combine different datasetsRDF merge is simple

3 SharingLinked DataBuilt on web technologies (HTTP, URIs)

1 / 21

Why use RDF for data integration (II)

Global Identifiers

× ×√

Schema-less

×√ √

Self-Describing

× × /√ √

Graph-Based

× × /√ √

2 / 21

Global Identifiers ×

Schema-less ×

√ √

Self-Describing ×

× /√ √

Graph-Based ×

× /√ √

2 / 21

Global Identifiers × ×

Schema-less ×√

Self-Describing × × /√

Graph-Based × × /√

2 / 21

Global Identifiers × ×√

Schema-less ×√ √

Self-Describing × × /√ √

Graph-Based × × /√ √

2 / 21

RDF-based Data Integration

DataTransformation

DataRepresentation

3 / 21

DataTransformation

DataRepresentation

3 / 21

DataTransformation

DataRepresentation

3 / 21

DataTransformation

DataRepresentation

3 / 21

DataTransformation

DataRepresentation

3 / 21

Data Transformation: Query Languages

SQLselect address from personwhere name = "Nuno"

XQueryfor $person in doc("people.xml")return $person//address

SPARQLselect $addressfrom <nunolopes.org/..>where { :nuno foaf:address

$address }

relation

solution sequence

sequence of items

SPA QLML

4 / 21

$address }

relation

solution sequence

sequence of items

SPA QLML

4 / 21

$address }

relation

solution sequence

sequence of items

SPA QLML

4 / 21

$address }

relation

solution sequence

sequence of items

SPA QLML

4 / 21

XSPARQL

SPA QLML

RTransformation language between RDB,XML, and RDF

Syntactic extension of XQuerySemantics based on XQuery’s semantics

Why based on XQuery?Expressive languageUse as scripting language

Arbitrary Nesting of expressions

5 / 21

XSPARQL

SPA QLML

RTransformation language between RDB,XML, and RDFSyntactic extension of XQuery

Semantics based on XQuery’s semantics

5 / 21

XSPARQL

SPA QLML

RTransformation language between RDB,XML, and RDFSyntactic extension of XQuerySemantics based on XQuery’s semantics

5 / 21

XSPARQL

SPA QLML

5 / 21

XSPARQL

SPA QLML

Why based on XQuery?Expressive languageUse as scripting languageArbitrary Nesting of expressions

5 / 21

Same Language for each Format

XSPARQLfor address as $address from peoplewhere name = "Nuno"return $address

for $person in doc("people.xml")return $person//address

XSPARQLfor $address from <nunolopes.org/..>where { :nuno foaf:address $address }return $address

for var in Exprlet var := Exprwhere Exprorder by Exprreturn Expr

for SelectSpecfrom RelationListwhere WhereSpecListreturn Expr

for varlistfrom DatasetClausewhere { pattern }return Expr

6 / 21

XSPARQLfor $person in doc("people.xml")return $person//address

6 / 21

Usecase: Inparanoid

7 / 21

Usecase: Inparanoid

7 / 21

Usecase: Inparanoid

7 / 21

Usecase: Predicting Protein-Protein Interactions

8 / 21

Creating RDF with XSPARQL

for $gene in doc("H.sapiens-M.musculus.xml")//genelet $id := fn:data($gene/@id)let $geneId :=fn:data($gene/@geneId)let $protId :=fn:data($gene/@protId)let $uri := fn:concat("http://bioinfo.deri.ie/inparanoid/", $id)construct { <{$uri}> purl:identifier {$id};

:geneID {$geneId};:protID {$protId} . }

constructclause generatesRDF

Arbitrary XSPARQLexpressions in subject,predicate, and object

Query Result:1 dct:identifier "1" .:1 bioinfo:geneID "ENSG00000155657" .gene:ENSG00000155657 rdf:type bioinfo:Gene .:1 bioinfo:protID "ENSP00000364178" .protein:ENSP00000364178 rdf:type bioinfo:protein .gene:ENSG00000155657 bioinfo:source_database "Ensembl" .protein:ENSG00000155657 bioinfo:organism <http://purl.uniprot.org/taxonomy/9606> .

9 / 21

Usecase: Combining Inparanoid with BridgeDB

10 / 21

Integration Query Example

for $gene in doc("H.sapiens-M.musculus.xml")//genelet $id := fn:data($gene/@id)let $geneId :=fn:data($gene/@geneId)let $protId :=fn:data($gene/@protId)let $uri := fn:concat("http://bioinfo.deri.ie/inparanoid/", $id)for $link $idRight from linkwhere idRight = $geneIdconstruct { <{$uri}> purl:identifier {$id};

:geneID {$geneId};:protID {$protId};:link {$idRight} . }

More involved XSPARQL queries: RDB2RDFDirect Mapping: ∼130 LoCR2RML: ∼290 LoC

11 / 21

Cloudspaces

CloudSpace Platform offersInfrastructure for the intersection of:

Linked DataBig DataCloud Computing

12 / 21

Cloudspaces & Sindice

CloudspacesA collection of tool to help users Extract, Transform & Load big

data sets.

Cloudspaces is based on Sindice, allowing user-defined Linked Datapipelines to be used in the Cloud

SindiceLinked Data "Search Engine"High availability Sparql Endpoint:

12 Billion TriplesCan load 100 million triples a day (updated daily)

13 / 21

Cloudspaces & Sindice

CloudspacesA collection of tool to help users Extract, Transform & Load big

data sets.

Cloudspaces is based on Sindice, allowing user-defined Linked Datapipelines to be used in the Cloud

SindiceLinked Data "Search Engine"High availability Sparql Endpoint:

12 Billion TriplesCan load 100 million triples a day (updated daily)

13 / 21

Overview

XSPARQL

Usecases & Other features

Beyond XSPARQL

Conclusions14 / 21

SPARQL 1.1

Data integration features in SPARQL 1.1AggregatesSubqueriesFederation ExtensionsNegationExpressions in the SELECT clauseProperty PathsAssignment

14 / 21

SPARQL 1.1

Data integration features in SPARQL 1.1AggregatesSubqueriesFederation ExtensionsNegationExpressions in the SELECT clauseProperty PathsAssignment

14 / 21

SPARQL 1.1: Federation extensions

PREFIX dbpedia2: <http://dbpedia.org/property/>PREFIX foaf: <http://xmlns.com/foaf/0.1/>

SELECT ?N ?MyBFROM <http://polleres.net/foaf.rdf>WHERE { [ foaf:birthday ?MyB ].

SERVICE <http://dbpedia.org/sparql> {SELECT ?N WHERE { [ dbpedia2:born ?B; foaf:name ?N ]. }

}FILTER ( Regex(Str(?B),str(?MyB)) )

Problemlimits from SPARQL endpoints prevent this query from working!

15 / 21

XSPARQL: endpoint/service queries

XSPARQL endpoint

prefix dbprop: <http://dbpedia.org/property/>prefix foaf: <http://xmlns.com/foaf/0.1/>prefix : <http://xsparql.deri.org/bday#>

let $MyB := for * from <http://polleres.net/foaf.rdf>where { [ foaf:birthday $B ]. }return $B

for * from <http://dbpedia.org/>endpoint <http://dbpedia.org/sparql>where { [ dbprop:born $B; foaf:name $N ].

filter ( regex(str($B),str($MyB)) ) }construct { :axel :sameBirthDayAs $N }

16 / 21

XSPARQL: endpoint/service queries

XSPARQL endpoint

prefix dbprop: <http://dbpedia.org/property/>prefix foaf: <http://xmlns.com/foaf/0.1/>prefix : <http://xsparql.deri.org/bday#>

let $MyB := for * from <http://polleres.net/foaf.rdf>where { [ foaf:birthday $B ]. }return $B

for * from <http://dbpedia.org/>endpoint <http://dbpedia.org/sparql>where { [ dbprop:born $B; foaf:name $N ].

filter ( regex(str($B),str($MyB)) ) }construct { :axel :sameBirthDayAs $N }

16 / 21

Converting Logainm dump to RDF

SPA QLML

∼ 1.3M triples

Data provided in XML

Translated to RDF using XSPARQL

Exposed using Openlink Virtuoso

17 / 21

SPA QLML

∼ 1.3M triples

17 / 21

SPA QLML

∼ 1.3M triples

17 / 21

Overview

XSPARQL

Usecases & Other features

Beyond XSPARQL

Conclusions18 / 21

XSPARQLVizhttp://deri-srvgal33.nuig.ie:8080/XsparqlViz/

18 / 21

Annotated RDF(S) Domains

Annotations refer to a specific domain

Temporal:nuno :address :Galway . [2008,2012]

Fuzzy:nuno :address :Dublin . 0.9

Access Control:nuno :address :Galway . [nl]

19 / 21

Annotated RDF(S) AC Query

ap address Galway.

ap birthday ”24/03”].nl address Galway.

nl birthday ”23/12”].

AnQL AC QuerySELECT * WHERE { $person :birthday $birthday .q] }

$person $birthdayap “24/03”nl “23/12”

20 / 21

ap address Galway.

ap birthday ”24/03” : [[ap]].

nl address Galway.

nl birthday ”23/12” : [[nl ]].

AnQL AC QuerySELECT * WHERE { $person :birthday $birthday :[[nl]] . }

20 / 21

ap address Galway.

ap birthday ”24/03” : [[ap]].

nl address Galway.

nl birthday ”23/12” : [[nl ]].

AnQL AC QuerySELECT * WHERE { $person :birthday $birthday :[[nl]] . }

20 / 21

Conclusions

You can use XSPARQL to easily merge data from differentsources in a common language

Annotated RDF can provide Access Control over RDF data

Useful linksXSPARQL http://xsparql.deri.org/

XSPARQLViz http://deri-srvgal33.nuig.ie:8080/XsparqlViz/

logainm http://data.logainm.ie/

Thank you! Questions?

21 / 21

Conclusions

You can use XSPARQL to easily merge data from differentsources in a common languageAnnotated RDF can provide Access Control over RDF data

21 / 21

Conclusions

21 / 21

Conclusions

21 / 21

xsparql credible workshop

Education

address xquery

use rdf

nuno return

expr return expr

linked data

nuno xquery

xml data mediation

xsparql spa ql