programming with lod - kaistsemanticweb.kaist.ac.kr/2012lodsummer/lecture/lecture...programming with...
TRANSCRIPT
Programming with LOD
Fumihiro Kato
Research Organization of Information and SystemsTransdisciplinary Research Integration Center
National Institute of Informatics
1
Linked Data Principles
1. Use URIs as names for things
2. Use HTTP URIs so that people can look up those names.
3. When someone looks up a URI, provide useful information, using the standards (RDF, SPARQL)
4. Include links to other URIs, so that they can discover more things.
2
http://dbpedia.org/resource/South_Korea3
1. request /resource/South_Korea
2. return an HTML document for /resource/South_Korea
http://dbpedia.orgWeb Browser
4
1. request /resource/South_Korea
2. return an RDF document for /resource/South_Korea
http://dbpedia.orgLOD Agent@prefix dbpedia-owl: <http://dbpedia.org/ontology/> .@prefix dbpedia: <http://dbpedia.org/resource/> .dbpedia:Seoul dbpedia-owl:country dbpedia:South_Korea .dbpedia:JnC dbpedia-owl:hometown dbpedia:South_Korea .@prefix dbpprop: <http://dbpedia.org/property/> .dbpedia:JnC dbpprop:origin dbpedia:South_Korea .dbpedia:Lee_Jae_Jin dbpedia-owl:hometown dbpedia:South_Korea ; dbpprop:origin dbpedia:South_Korea .dbpedia:Kang_Sung_Hoon dbpedia-owl:hometown dbpedia:South_Korea ; dbpprop:origin dbpedia:South_Korea .dbpedia:Northern_Asia-Pacific_Division_of_Seventh-day_Adventists dbpedia-owl:location dbpedia:South_Korea ; dbpedia-owl:regionServed dbpedia:South_Korea ; dbpprop:location dbpedia:South_Korea .dbpedia:Korean_Amateur_Radio_League dbpedia-owl:regionServed dbpedia:South_Korea ; dbpprop:regionServed dbpedia:South_Korea .dbpedia:Asia-Pacific_Association_for_International_Education dbpprop:headquarters dbpedia:South_Korea ; dbpedia-owl:headquarter dbpedia:South_Korea .dbpedia:The_Korean_Chemical_Society dbpedia-owl:location dbpedia:South_Korea ; dbpprop:location dbpedia:South_Korea .dbpedia:Korean_e-Sports_Association dbpedia-owl:regionServed dbpedia:South_Korea .dbpedia:Miss_Asia_Pacific_World dbpprop:headquarters dbpedia:South_Korea ; dbpedia-owl:headquarter dbpedia:South_Korea .dbpedia:Daegu_Stadium dbpedia-owl:location dbpedia:South_Korea .dbpedia:Chuncheon_Civic_Stadium dbpedia-owl:location dbpedia:South_Korea .
5
RDF Libraries
• Java: Jena, Sesame
• C: Redland
• C#: dotNetRDF
• Ruby: RDF.rb
• PHP: ARC2, Graphite
• Python: rdflib
• Perl: PerlRDF
• JavaScript: rdfQuery, rdfstore-js
• ....
See http://www.w3.org/2001/sw/wiki/Tools
6
RDF.rb
• http://rdf.rubyforge.org
• http://github.com/ruby-rdf
• $ gem install linkeddata equivalent-xml
7
Reading RDF
1 #!/usr/bin/env ruby 2 # -*- coding: utf-8 -*- 3 4 require 'linkeddata' 5 6 RDF::Reader.open("http://dbpedia.org/resource/South_Korea") do |reader| 7 reader.each do |statement| 8 puts statement.inspect 9 end10 end
8
Reading RDF
.......#<RDF::Statement:0x3fd362ca7338(<http://dbpedia.org/resource/South_Korea> <http://dbpedia.org/ontology/PopulatedPlace/areaTotal> 100209.22997701018 .)>#<RDF::Statement:0x3fd362ca7338(<http://dbpedia.org/resource/South_Korea> <http://dbpedia.org/ontology/PopulatedPlace/areaTotal> 100209.22997701018 .)>#<RDF::Statement:0x3fd362cabd70(<http://dbpedia.org/resource/South_Korea> <http://dbpedia.org/ontology/PopulatedPlace/areaTotal> 100210.0 .)>#<RDF::Statement:0x3fd362cabd70(<http://dbpedia.org/resource/South_Korea> <http://dbpedia.org/ontology/PopulatedPlace/areaTotal> 100210.0 .)>#<RDF::Statement:0x3fd3623772cc(<http://dbpedia.org/resource/South_Korea> <http://dbpedia.org/ontology/PopulatedPlace/populationDensity> 491.0 .)>#<RDF::Statement:0x3fd3623772cc(<http://dbpedia.org/resource/South_Korea> <http://dbpedia.org/ontology/PopulatedPlace/populationDensity> 491.0 .)>#<RDF::Statement:0x3fd362c797bc(<http://dbpedia.org/resource/South_Korea> <http://dbpedia.org/ontology/PopulatedPlace/populationDensity> 490.73584350744875 .)>#<RDF::Statement:0x3fd362c797bc(<http://dbpedia.org/resource/South_Korea> <http://dbpedia.org/ontology/PopulatedPlace/populationDensity> 490.73584350744875 .)>#<RDF::Statement:0x3fd362c59c28(<http://dbpedia.org/resource/South_Korea> <http://dbpedia.org/ontology/abstract> Corea del Sud, of icialment la República de Corea) és un país de l'Àsia que ocupa la regió meridional de la península de Corea. Al nord, limita amb Corea del Nord, al qual estava unit fins el 1945. A l'oest, creuant la mar Groga, es troba la República Popular de Xina, i a l'est, creuant l'Estret de Corea, es troba el Japó. La capital i la ciutat més gran n'és Seül, l'àrea metropolitana de la qual és una de les tres més poblades del món. La llengua oficial de Corea del Sud és el coreà, i és, actualment, la desena llengua més parlada del món. El nom del país, en coreà, significa "La Gran Nació del Poble Coreà", hom es refereix al país sovint com la "Terra de la Calma del Matí". llegendari Dangun Kanggeom. L'evidència arqueològica suggereix que els coreans han habitat la península des del Paleolític Baix. Després de la unificació dels Tres Regnes Coreans sota el comandament de Silla el 668 dC, Corea va romandre com a nació unificada sota les dinasties de Goryeao i Joseaon fins la caiguda de l'Imperi Coreà el 1910. Després de la divisió i de l'establiment de la república moderna de Corea del Sud, el país ha tingut que reconstruir-se després del període d'explotació japonesa i de la Guerra de Corea. No obstant això, l'estàndard de vida, així com l'economia, van créixer de manera sorprenent des de la dècada de 1980. El 1987, després de diverses manifestacions nacionals, el govern autoritari es va col·lapsar, i des de llavors, s'hi han celebrat eleccions democràtiques cada cinc anys. Avui dia, Corea del Sur és un dels Estats pioners de la democràcia a Àsia; els ciutadans coreans gaudeixen d'un dels nivells més alts de......
1 #!/usr/bin/env ruby 2 # -*- coding: utf-8 -*- 3 4 require 'linkeddata' 5 6 RDF::Reader.open("http://dbpedia.org/resource/South_Korea") do |reader| 7 reader.each do |statement| 8 puts statement.inspect 9 end10 end
9
Accessing statements
1 #!/usr/bin/env ruby 2 # -*- coding: utf-8 -*- 3 4 require 'linkeddata' 5 6 RDF::Reader.open("http://dbpedia.org/resource/Korea") do |reader| 7 reader.each do |statement| 8 puts "s: #{statement.subject}, p: #{statement.predicate}, o: #{statement.object}" 9 end10 end
10
RDF Serialization
• N-Triples
• RDF/XML
• Turtle (N3)
• TriX
• RDFa
• JSON-LD
• RDF/JSON
• ...
11
Reading text/n3
1 #!/usr/bin/env ruby 2 # -*- coding: utf-8 -*- 3 4 require 'linkeddata' 5 6 RDF::Reader.open("http://dbpedia.org/data/South_Korea.n3", :format => "n3") do |reader| 7 reader.each do |statement| 8 puts statement.inspect! 9 end10 end
12
Writing RDF
1 #!/usr/bin/env ruby 2 # -*- coding: utf-8 -*- 3 4 require 'linkeddata' 5 6 RDF::Writer.open("writing.ttl") do |writer| 7 subject = RDF::URI("http://dbpedia.org/resource/South_Korea") 8 predicate = RDF::RDFS.label 9 object = RDF::Literal("韓国", :language => :ja)10 writer << RDF::Steatement.new(subject, predicate, object)11 end
13
Loading and Merging RDF
1 #!/usr/bin/env ruby 2 # -*- coding: utf-8 -*- 3 4 require 'linkeddata' 5 6 graph = RDF::Graph.new 7 graph.load("http://dbpedia.org/resource/South_Korea") 8 puts graph.size 9 graph.load("http://dbpedia.org/resource/Daejeon")10 puts graph.size
14
Querying RDF 1 #!/usr/bin/env ruby 2 # -*- coding: utf-8 -*- 3 4 require 'linkeddata' 5 6 graph = RDF::Graph.new 7 graph.load("http://dbpedia.org/resource/Daejeon") 8 graph.query([RDF::URI("http://dbpedia.org/resource/Daejeon"), 9 RDF::URI("http://dbpedia.org/ontology/country"),10 nil]).each do |statement|11 puts statement.object12 end
dbpedia:Daejeon
dbpedia-owl:country
dbpedia:South_Korea
dbpedia-owl:Cityrdf:type
Daejeon
rdfs:label36.350101
geo:lat
15
Defining a vocabulary 1 #!/usr/bin/env ruby 2 # -*- coding: utf-8 -*- 3 4 require 'linkeddata' 5 6 DBP = RDF::Vocabulary.new("http://dbpedia.org/resource/") 7 DBP_ONT = RDF::Vocabulary.new("http://dbpedia.org/ontology/") 8 9 graph = RDF::Graph.new10 graph.load(DBP.Daejeon)11 graph.query([DBP.Daejeon, DBP_ONT.country, nil]).each do |statement|12 puts statement.object13 end
dbpedia:Daejeon
dbpedia-owl:country
dbpedia-owl:Cityrdf:type
Daejeon
rdfs:label36.350101
geo:lat
dbpedia-owl:South_Korea
16
RDB
Query
DB
Model RDF Model
SQL
Relational Model
RDF Store
SPARQL
17
SPARQL
• SPARQL Protocol and RDF Query Language
• W3C Standards
• SQL-like syntax
• Graph Patterns
• Result Formats: XML, JSON, (CSV, TSV)
• $ gem install sparql
18
Using SPARQL in RDF.rb
1 #!/usr/bin/env ruby 2 # -*- coding: utf-8 -*- 3 4 require 'linkeddata' 5 6 graph = RDF::Graph.new 7 graph.load("http://dbpedia.org/resource/Daejeon") 8 9 sse = SPARQL.parse("""SELECT ?o 10 WHERE {11 <http://dbpedia.org/resource/Daejeon> <http://dbpedia.org/ontology/country> ?o .12 }""")13 14 sse.execute(graph).each do |solution|15 puts solution[:o]16 end
19
Querying a SPARQL Endpoint
1 #!/usr/bin/env ruby 2 # -*- coding: utf-8 -*- 3 4 require 'linkeddata' 5 6 client = SPARQL::Client.new("http://dbpedia.org/sparql") 7 8 query = """SELECT ?o 9 WHERE {10 <http://dbpedia.org/resource/Daejeon> <http://dbpedia.org/ontology/country> ?o .11 }"""12 13 client.query(query).each do |solution|14 puts solution[:o]15 end
20
dbpedia:Seoul
Seoul
dbpedia:South_Korea
dbpedia-owl:coutnry
rdfs:label
dbpedia-owl:capital
freebase:Seoul
owl:sameAs
rdfs:labelSouth Koreadbpedia-owl:City
rdf:type
dbpedia:Samsung_Electronics
dbpedia-owl:location
Samsung Electronics
rdfs:labelhttp://global.samsungtomorrow.com/
foaf:homepage
1969-01-01
dbpedia-owl:Companyrdf:type
dbpprop:foundingYear
dbpedia-owl:Country
rdf:type
dbpedia-owl:Lee_Kun-hee
dbpedia-owl:owner
삼성전자
dbpedia-owl:location
rdfs:label
21
dbpedia:Seoul
Seoul
dbpedia:South_Korea
dbpedia-owl:coutnry
rdfs:label
dbpedia-owl:capital
freebase:Seoul
owl:sameAs
rdfs:labelSouth Koreadbpedia-owl:City
rdf:type
dbpedia:Samsung_Electronics
dbpedia-owl:location
Samsung Electronics
rdfs:labelhttp://global.samsungtomorrow.com/
foaf:homepage
1969-01-01
dbpedia-owl:Companyrdf:type
dbpprop:foundingYear
dbpedia-owl:Country
rdf:type
dbpedia-owl:Lee_Kun-hee
dbpedia-owl:owner
삼성전자
dbpedia-owl:location
rdfs:label
PREFIX dbpedia: <http://dbpedia.org/resource/>PREFIX dbpedia-owl: <http://dbpedia.org/ontology/>PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
SELECT ?location ?labelWHERE { dbpedia:Samsung_Electronics dbpedia-owl:location ?location . ?location rdfs:label ?label .}
22
23
24
Querying a SPARQL Endpoint 1 #!/usr/bin/env ruby 2 # -*- coding: utf-8 -*- 3 4 require 'linkeddata' 5 6 client = SPARQL::Client.new("http://dbpedia.org/sparql") 7 8 query = """PREFIX dbpedia: <http://dbpedia.org/resource/> 9 PREFIX dbpedia-owl: <http://dbpedia.org/ontology/>10 PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>11 12 SELECT ?location ?label13 WHERE {14 dbpedia:Samsung_Electronics dbpedia-owl:location ?location .15 ?location rdfs:label ?label .16 }"""17 18 client.query(query).each do |solution|19 puts solution.inspect20 end
25
dbpedia:Seoul
Seoul
dbpedia:South_Korea
dbpedia-owl:coutnry
rdfs:label
dbpedia-owl:capital
freebase:Seoul
owl:sameAs
rdfs:labeldbpedia-owl:City
rdf:type
dbpedia:Samsung_Electronics
dbpedia-owl:location
Samsung Electronics
rdfs:labelhttp://global.samsungtomorrow.com/
foaf:homepage
1969-01-01
dbpedia-owl:Companyrdf:type
dbpprop:foundingYear
dbpedia-owl:Country
rdf:type
dbpedia-owl:Lee_Kun-hee
dbpedia-owl:owner
삼성전자
dbpedia-owl:location
rdfs:label
PREFIX dbpedia: <http://dbpedia.org/resource/>PREFIX dbpedia-owl: <http://dbpedia.org/ontology/>PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
SELECT ?location ?labelWHERE { dbpedia:Samsung_Electronics dbpedia-owl:location ?location . ?location a dbpedia-owl:City ; rdfs:label ?label .}
South Korea
26
27
dbpedia:Seoul
Seoul
dbpedia:South_Korea
dbpedia-owl:coutnry
rdfs:label
dbpedia-owl:capital
freebase:Seoul
owl:sameAs
rdfs:labeldbpedia-owl:City
rdf:type
dbpedia:Samsung_Electronics
dbpedia-owl:location
Samsung Electronics
rdfs:labelhttp://global.samsungtomorrow.com/
foaf:homepage
1969-01-01
dbpedia-owl:Companyrdf:type
dbpprop:foundingYear
dbpedia-owl:Country
rdf:type
dbpedia-owl:Lee_Kun-hee
dbpedia-owl:owner
삼성전자
dbpedia-owl:location
rdfs:label
PREFIX dbpedia: <http://dbpedia.org/resource/>PREFIX dbpedia-owl: <http://dbpedia.org/ontology/>PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
SELECT ?location ?labelWHERE { dbpedia:Samsung_Electronics dbpedia-owl:location ?location . ?location a dbpedia-owl:City ; rdfs:label ?label . FILTER(lang(?label) = ‘ko’)}
South Korea
28
29
SPARQL 1.0
• CRUD
• SELECT
• ASK
• CONSTRUCT
• DESCRIBE
30
SELECT
クエリパターン変数の値を取得
SELECT *WHERE { ?s ?p ?o .}LIMIT 100OFFSET 100
31
ASK
PREFIX dbpedia: <http://dbpedia.org/resource/>PREFIX dbpedia-owl: <http://dbpedia.org/ontology/>
ASK WHERE { dbpedia:Daejeon ?p ?o .}
32
CONSTRUCT
PREFIX foaf: <http://xmlns.com/foaf/0.1/>PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
CONSTRUCT { ?s rdfs:label ?name .}WHERE { ?s foaf:name ?name .}
33
DESCRIBE
DESCRIBE <http://dbpedia.org/resource/Tokyo>
34
• http://www.w3.org/wiki/SparqlEndpoints
http://www.w3.org/wiki/SparqlEndpointshttp://www.w3.org/wiki/SparqlEndpoints35
• http://www.w3.org/wiki/SparqlEndpoints
http://www.w3.org/wiki/SparqlEndpointshttp://labs.mondeca.com/sparqlEndpointsStatus/36
http://ckan.net/package?res_format=api%2Fsparqlhttp://thedatahub.org/group/lodcloud37
http://lod.ac/apps/lodac_location/38
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>PREFIX geo: <http://www.w3.org/2003/01/geo/wgs84_pos#>
SELECT DISTINCT ?link ?title ?lat ?long WHERE { ?link rdfs:label ?title; geo:lat ?lat; geo:long ?long . FILTER ( ?lat > #{sw_lat} && ?lat < #{ne_lat} && ?long > #{sw_long} && ?long < #{ne_long} && lang(?title) = "ja" )}
39
RDB
Query
DB
Model RDF Model
SQL
Relational Model
RDF Store
SPARQL
40
RDF Store
• Triplestore / Quadstore / Graph DB
• SPARQL Endpoint
• Options
• RDFS/OWL Inference
• Full text indexing
• Clustering
• See http://en.wikipedia.org/wiki/Triplestore
Bigdata®
41
http://virtuoso.openlinksw.com/dataspace/dav/wiki/Main/42
Virtuoso Open Source• http://virtuoso.openlinksw.com/dataspace/dav/wiki/Main/
• install binary
• install from source$ tar xvzf virtuoso-opensource-6.1.6.tar.gz
$ cd virtuoso-opensource-6.1.6
$ ./configure
$ make
$ make install
$ /usr/local/virtuoso-opensource/bin/virtuoso-t +configfile /usr/local/virtuoso-opensource/var/lib/virtuoso/db/virtuoso.ini43
http://localhost:8890/conductor/
44
http://localhost:8890/conductor/
45
46
47
Storage Adapter• Memory
• Sesame
• Virtuoso
• DataObjects (SQLite3, PostgreSQL)
• RedStore
• 4Store
• MongoDB
• Cassandra
• Talis
• SPARQL
49
Virtuoso Adapter$ gem install rdf-virtuoso
1 #!/usr/bin/env ruby 2 # -*- coding: utf-8 -*- 3 4 require 'linkeddata' 5 require 'rdf/virtuoso' 6 7 REPO_URI = "http://localhost:8890" 8 USER = "dba" 9 PASS = "dba" # this should be changed10 DBP = RDF::Vocabulary.new("http://dbpedia.org/resource/")11 12 repo = RDF::Virtuoso::Repository.new(REPO_URI, :username => USER,13 :password => PASS, :auth_method => 'digest')14 query = RDF::Virtuoso::Query.select.where([:s, :p, DBP.South_Korea])15 result = repo.select(query)16 17 result.each do |solution|18 puts solution.inspect19 end
50
Summary
• how to consume LOD
• RDF
• SPARQL
51