change tracking in knowledge organization systems with ......three basic steps to an actionable...
TRANSCRIPT
![Page 1: Change Tracking in Knowledge Organization Systems with ......Three basic steps to an actionable skos-history Start with one SKOS file per version. 1) Create the deltas - insertions](https://reader033.vdocuments.us/reader033/viewer/2022042007/5e70ae76fbd42672be50f7c4/html5/thumbnails/1.jpg)
Change Tracking in Knowledge Organization Systems with skos-history
Joachim Neubert & Osma SuominenZBW – Leibniz Information Centre for Economics, Kiel/Hamburg &The National Library of Finland, Helsinki
DCMI/ASIST/AIMS Webinar Series:Generic Tools and Methods for SKOS-based Concept Schemes16.3.2016
![Page 2: Change Tracking in Knowledge Organization Systems with ......Three basic steps to an actionable skos-history Start with one SKOS file per version. 1) Create the deltas - insertions](https://reader033.vdocuments.us/reader033/viewer/2022042007/5e70ae76fbd42672be50f7c4/html5/thumbnails/2.jpg)
Agenda
User questions and requirements Getting a grip on changes: Overview Creating a version store Generic queries Dataset-specific adaption of queries
skos-history in use Application at the National Library of Finland Application for STW Thesaurus for Economics
Outlook: Future work and the skos-history project
Page 2
![Page 3: Change Tracking in Knowledge Organization Systems with ......Three basic steps to an actionable skos-history Start with one SKOS file per version. 1) Create the deltas - insertions](https://reader033.vdocuments.us/reader033/viewer/2022042007/5e70ae76fbd42672be50f7c4/html5/thumbnails/3.jpg)
What users want to know …
Page 3
… when we publish a new KOS version:
What‘s new? What has changed?
![Page 4: Change Tracking in Knowledge Organization Systems with ......Three basic steps to an actionable skos-history Start with one SKOS file per version. 1) Create the deltas - insertions](https://reader033.vdocuments.us/reader033/viewer/2022042007/5e70ae76fbd42672be50f7c4/html5/thumbnails/4.jpg)
Use cases for extended change information
Page 4
Human indexers wanting to learn about new and deprecated concepts
Human indexers (and supporting applications) re-indexing large sets of documents
People maintaining mappings to other vocabularies, and applications supporting them
People maintaining a derived subset of a KOS Vocabulary-based automatic or semi-automatic indexing applications Search applications utilizing the KOS
![Page 5: Change Tracking in Knowledge Organization Systems with ......Three basic steps to an actionable skos-history Start with one SKOS file per version. 1) Create the deltas - insertions](https://reader033.vdocuments.us/reader033/viewer/2022042007/5e70ae76fbd42672be50f7c4/html5/thumbnails/5.jpg)
Agenda
User questions and requirements Getting a grip on changes: Overview Creating a version store Generic queries Dataset-specific adaption of queries
skos-history in use Application at the National Library of Finland Application for STW Thesaurus for Economics
Outlook: Future work and the skos-history project
Page 5
![Page 6: Change Tracking in Knowledge Organization Systems with ......Three basic steps to an actionable skos-history Start with one SKOS file per version. 1) Create the deltas - insertions](https://reader033.vdocuments.us/reader033/viewer/2022042007/5e70ae76fbd42672be50f7c4/html5/thumbnails/6.jpg)
Overview: getting a grip on changes
Provided that we have no access to the KOS maintenance system where the changes take place originally, or can’t extend it to report this changes comprehensively.
Dataset versioning + skos-history approach
=> should work on every SKOS vocabulary
Page 6
![Page 7: Change Tracking in Knowledge Organization Systems with ......Three basic steps to an actionable skos-history Start with one SKOS file per version. 1) Create the deltas - insertions](https://reader033.vdocuments.us/reader033/viewer/2022042007/5e70ae76fbd42672be50f7c4/html5/thumbnails/7.jpg)
Scope of vocabulary versioning
Versioning the concept scheme, not each individual concept URIs for the concepts remain stable over the different versions Distinct versions of a vocabulary, or at least timestamped dumps,
must be available Support for a continuous flow of changes, e.g., the LoC Subject
Headings, or the concepts of the GND, is currently not provided
Page 7
![Page 8: Change Tracking in Knowledge Organization Systems with ......Three basic steps to an actionable skos-history Start with one SKOS file per version. 1) Create the deltas - insertions](https://reader033.vdocuments.us/reader033/viewer/2022042007/5e70ae76fbd42672be50f7c4/html5/thumbnails/8.jpg)
Three basic steps to an actionable skos-history
Start with one SKOS file per version.1) Create the deltas - insertions and deletions - between every two
version files. (Via a raw diff of sorted ntriples files, or via SPARQL MINUS in a triple store.This gives you thousands and thousands of differences -added or deleted triples -, even excluding bnodes.)
2) Load the version files and the insertions and deletions into a triplestore as named graphs.
3) Add metadata about the versions and the deltas in a separate „version history graph“.
Page 8
https://github.com/jneubert/skos-history/blob/master/bin/load_versions.sh
![Page 9: Change Tracking in Knowledge Organization Systems with ......Three basic steps to an actionable skos-history Start with one SKOS file per version. 1) Create the deltas - insertions](https://reader033.vdocuments.us/reader033/viewer/2022042007/5e70ae76fbd42672be50f7c4/html5/thumbnails/9.jpg)
Agenda
User questions and requirements Getting a grip on changes: Overview Creating a version store Generic queries Dataset-specific adaption of queries
skos-history in use Application at the National Library of Finland Application for STW Thesaurus for Economics
Outlook: Future work and the skos-history project
Page 9
![Page 10: Change Tracking in Knowledge Organization Systems with ......Three basic steps to an actionable skos-history Start with one SKOS file per version. 1) Create the deltas - insertions](https://reader033.vdocuments.us/reader033/viewer/2022042007/5e70ae76fbd42672be50f7c4/html5/thumbnails/10.jpg)
Hands on: Create a version store for skos-history
Requirements:
SPARQL 1.1 compliant service or repository (‘triple store’), accessible in read/write modehttps://github.com/NatLibFi/Skosmos/wiki/InstallTutorial#install-jena-fuseki
An environment for executing bash scripts for the data load script (any Linux should do, Cygwin may).
Tutorial: https://github.com/jneubert/skos-history/wiki/TutorialCode of scripts and queries: also on GitHub
Page 10
![Page 11: Change Tracking in Knowledge Organization Systems with ......Three basic steps to an actionable skos-history Start with one SKOS file per version. 1) Create the deltas - insertions](https://reader033.vdocuments.us/reader033/viewer/2022042007/5e70ae76fbd42672be50f7c4/html5/thumbnails/11.jpg)
Load a version store: config file for JEL
Page 11
Configuration for Fuseki (https://github.com/jneubert/skos-history/blob/master/bin/jel.config);see also configuration for Sesame (https://github.com/jneubert/skos-history/blob/master/bin/jel.sesame.config)
![Page 12: Change Tracking in Knowledge Organization Systems with ......Three basic steps to an actionable skos-history Start with one SKOS file per version. 1) Create the deltas - insertions](https://reader033.vdocuments.us/reader033/viewer/2022042007/5e70ae76fbd42672be50f7c4/html5/thumbnails/12.jpg)
Load a version store: load_versions.sh script
Page 12
![Page 13: Change Tracking in Knowledge Organization Systems with ......Three basic steps to an actionable skos-history Start with one SKOS file per version. 1) Create the deltas - insertions](https://reader033.vdocuments.us/reader033/viewer/2022042007/5e70ae76fbd42672be50f7c4/html5/thumbnails/13.jpg)
Load a version store: load_versions.sh script
Page 13
![Page 14: Change Tracking in Knowledge Organization Systems with ......Three basic steps to an actionable skos-history Start with one SKOS file per version. 1) Create the deltas - insertions](https://reader033.vdocuments.us/reader033/viewer/2022042007/5e70ae76fbd42672be50f7c4/html5/thumbnails/14.jpg)
Page 14
Example endpoint:http://zbw.eu/beta/sparql/stwv/query
Version History Graph, discoverable viafix URI, e.g.: http://zbw.eu/stw/version
![Page 15: Change Tracking in Knowledge Organization Systems with ......Three basic steps to an actionable skos-history Start with one SKOS file per version. 1) Create the deltas - insertions](https://reader033.vdocuments.us/reader033/viewer/2022042007/5e70ae76fbd42672be50f7c4/html5/thumbnails/15.jpg)
Version History Graph, published as HTML/RDFa
Page 15http://zbw.eu/stw/version
![Page 16: Change Tracking in Knowledge Organization Systems with ......Three basic steps to an actionable skos-history Start with one SKOS file per version. 1) Create the deltas - insertions](https://reader033.vdocuments.us/reader033/viewer/2022042007/5e70ae76fbd42672be50f7c4/html5/thumbnails/16.jpg)
Vocabularies used for the plumbing
dc:/dcterms:Dublin Core, as usual the base for everything
void: http://rdfs.org/ns/void#Vocabulary of interlinked datasets
sd: http://www.w3.org/ns/sparql-service-description#SPARQL service description
delta: http://www.w3.org/2004/delta#Differences between RDF graphs
dsv: http://purl.org/iso25964/DataSet/Versioning#Version history records (providing version identifier and date) and a pointer to the current version – outside the actual version data
sh: http://purl.org/skos-history/Scheme and concept version deltas
Page 16
![Page 17: Change Tracking in Knowledge Organization Systems with ......Three basic steps to an actionable skos-history Start with one SKOS file per version. 1) Create the deltas - insertions](https://reader033.vdocuments.us/reader033/viewer/2022042007/5e70ae76fbd42672be50f7c4/html5/thumbnails/17.jpg)
What’s the benefit?
A database of all versions of a KOS and all deltas between versions – which can be queried in parallel!
Page 17
![Page 18: Change Tracking in Knowledge Organization Systems with ......Three basic steps to an actionable skos-history Start with one SKOS file per version. 1) Create the deltas - insertions](https://reader033.vdocuments.us/reader033/viewer/2022042007/5e70ae76fbd42672be50f7c4/html5/thumbnails/18.jpg)
Agenda
User questions and requirements Getting a grip on changes: Overview Creating a version store Generic queries Dataset-specific adaption of queries
skos-history in use Application at the National Library of Finland Application for STW Thesaurus for Economics
Outlook: Future work and the skos-history project
Page 18
![Page 19: Change Tracking in Knowledge Organization Systems with ......Three basic steps to an actionable skos-history Start with one SKOS file per version. 1) Create the deltas - insertions](https://reader033.vdocuments.us/reader033/viewer/2022042007/5e70ae76fbd42672be50f7c4/html5/thumbnails/19.jpg)
Page 19http://zbw.eu/beta/sparql-lab/?queryRef=https://api.github.com/repos/jneubert/skos-history/contents/sparql/added_concepts.rq
Query for added concepts
![Page 20: Change Tracking in Knowledge Organization Systems with ......Three basic steps to an actionable skos-history Start with one SKOS file per version. 1) Create the deltas - insertions](https://reader033.vdocuments.us/reader033/viewer/2022042007/5e70ae76fbd42672be50f7c4/html5/thumbnails/20.jpg)
Newly inserted concepts – results
Page 20
![Page 21: Change Tracking in Knowledge Organization Systems with ......Three basic steps to an actionable skos-history Start with one SKOS file per version. 1) Create the deltas - insertions](https://reader033.vdocuments.us/reader033/viewer/2022042007/5e70ae76fbd42672be50f7c4/html5/thumbnails/21.jpg)
Reports operating on standard SKOS structures
Page 21
https://github.com/jneubert/skos-history/tree/master/sparql
![Page 22: Change Tracking in Knowledge Organization Systems with ......Three basic steps to an actionable skos-history Start with one SKOS file per version. 1) Create the deltas - insertions](https://reader033.vdocuments.us/reader033/viewer/2022042007/5e70ae76fbd42672be50f7c4/html5/thumbnails/22.jpg)
Reports … (continued)
Page 22
![Page 23: Change Tracking in Knowledge Organization Systems with ......Three basic steps to an actionable skos-history Start with one SKOS file per version. 1) Create the deltas - insertions](https://reader033.vdocuments.us/reader033/viewer/2022042007/5e70ae76fbd42672be50f7c4/html5/thumbnails/23.jpg)
Changed notations
Page 23
http://zbw.eu/beta/sparql-lab/?queryRef=https://api.github.com/repos/jneubert/skos-history/contents/sparql/changed_notations.rq
![Page 24: Change Tracking in Knowledge Organization Systems with ......Three basic steps to an actionable skos-history Start with one SKOS file per version. 1) Create the deltas - insertions](https://reader033.vdocuments.us/reader033/viewer/2022042007/5e70ae76fbd42672be50f7c4/html5/thumbnails/24.jpg)
New concepts, split from old ones
Page 24
Labels moved to added concepts:
http://zbw.eu/beta/sparql-lab/?queryRef=https://api.github.com/repos/jneubert/skos-history/contents/sparql/labels_moved_to_added_concepts.rq
![Page 25: Change Tracking in Knowledge Organization Systems with ......Three basic steps to an actionable skos-history Start with one SKOS file per version. 1) Create the deltas - insertions](https://reader033.vdocuments.us/reader033/viewer/2022042007/5e70ae76fbd42672be50f7c4/html5/thumbnails/25.jpg)
Change history of a concept: “Personnel selection”
Page 25
http
://zb
w.eu
/bet
a/sp
arql
-lab/
?que
ryR
ef=h
ttps:
//api
.gith
ub.c
om/re
pos/
jneu
bert/
skos
-his
tory
/con
tent
s/sp
arql
/con
cept
_del
tas.
rq
![Page 26: Change Tracking in Knowledge Organization Systems with ......Three basic steps to an actionable skos-history Start with one SKOS file per version. 1) Create the deltas - insertions](https://reader033.vdocuments.us/reader033/viewer/2022042007/5e70ae76fbd42672be50f7c4/html5/thumbnails/26.jpg)
Agenda
User questions and requirements Getting a grip on changes: Overview Creating a version store Generic queries Dataset-specific adaption of queries
skos-history in use Application at the National Library of Finland Application for STW Thesaurus for Economics
Outlook: Future work and the skos-history project
Page 26
![Page 27: Change Tracking in Knowledge Organization Systems with ......Three basic steps to an actionable skos-history Start with one SKOS file per version. 1) Create the deltas - insertions](https://reader033.vdocuments.us/reader033/viewer/2022042007/5e70ae76fbd42672be50f7c4/html5/thumbnails/27.jpg)
GND subjects by subject category – query
Page 27 http
s://g
ithub
.com
/jneu
bert/
skos
-his
tory
/blo
b/m
aste
r/spa
rql/s
wds
kos/
adde
d_co
ncep
ts_b
y_ca
tego
ry.rq
![Page 28: Change Tracking in Knowledge Organization Systems with ......Three basic steps to an actionable skos-history Start with one SKOS file per version. 1) Create the deltas - insertions](https://reader033.vdocuments.us/reader033/viewer/2022042007/5e70ae76fbd42672be50f7c4/html5/thumbnails/28.jpg)
GND subjects by subject category – results
Page 28
![Page 29: Change Tracking in Knowledge Organization Systems with ......Three basic steps to an actionable skos-history Start with one SKOS file per version. 1) Create the deltas - insertions](https://reader033.vdocuments.us/reader033/viewer/2022042007/5e70ae76fbd42672be50f7c4/html5/thumbnails/29.jpg)
STW deprecated concepts – query
Page 29 http
s://g
ithub
.com
/jneu
bert/
skos
-his
tory
/blo
b/m
aste
r/spa
rql/s
tw/d
epre
cate
d_co
ncep
ts_b
y_ca
tego
ry.rq
![Page 30: Change Tracking in Knowledge Organization Systems with ......Three basic steps to an actionable skos-history Start with one SKOS file per version. 1) Create the deltas - insertions](https://reader033.vdocuments.us/reader033/viewer/2022042007/5e70ae76fbd42672be50f7c4/html5/thumbnails/30.jpg)
STW deprecated concepts – result
Page 30
![Page 31: Change Tracking in Knowledge Organization Systems with ......Three basic steps to an actionable skos-history Start with one SKOS file per version. 1) Create the deltas - insertions](https://reader033.vdocuments.us/reader033/viewer/2022042007/5e70ae76fbd42672be50f7c4/html5/thumbnails/31.jpg)
Agenda
User questions and requirements Getting a grip on changes: Overview Creating a version store Generic queries Dataset-specific adaption of queries
skos-history in use Application at the National Library of Finland Application for STW Thesaurus for Economics
Outlook: Future work and the skos-history project
Page 31
![Page 32: Change Tracking in Knowledge Organization Systems with ......Three basic steps to an actionable skos-history Start with one SKOS file per version. 1) Create the deltas - insertions](https://reader033.vdocuments.us/reader033/viewer/2022042007/5e70ae76fbd42672be50f7c4/html5/thumbnails/32.jpg)
skos-history at the National Library of Finland
see separate slides at http://tinyurl.com/skos-history-nlf
Page 32
![Page 33: Change Tracking in Knowledge Organization Systems with ......Three basic steps to an actionable skos-history Start with one SKOS file per version. 1) Create the deltas - insertions](https://reader033.vdocuments.us/reader033/viewer/2022042007/5e70ae76fbd42672be50f7c4/html5/thumbnails/33.jpg)
Agenda
User questions and requirements Getting a grip on changes: Overview Creating a version store Generic queries Dataset-specific adaption of queries
skos-history in use Application at the National Library of Finland Application for STW Thesaurus for Economics
Outlook: Future work and the skos-history project
Page 33
![Page 34: Change Tracking in Knowledge Organization Systems with ......Three basic steps to an actionable skos-history Start with one SKOS file per version. 1) Create the deltas - insertions](https://reader033.vdocuments.us/reader033/viewer/2022042007/5e70ae76fbd42672be50f7c4/html5/thumbnails/34.jpg)
STW Thesaurus for Economics
created in the 1990s on the web and available as SKOS since 2009 bilingual (German/English) about 6000 descriptors, 500 subject categories overhaul during the last five years (five consecutive versions)
Page 34
![Page 35: Change Tracking in Knowledge Organization Systems with ......Three basic steps to an actionable skos-history Start with one SKOS file per version. 1) Create the deltas - insertions](https://reader033.vdocuments.us/reader033/viewer/2022042007/5e70ae76fbd42672be50f7c4/html5/thumbnails/35.jpg)
STW change reports (precompiled query results)
Page 35
![Page 36: Change Tracking in Knowledge Organization Systems with ......Three basic steps to an actionable skos-history Start with one SKOS file per version. 1) Create the deltas - insertions](https://reader033.vdocuments.us/reader033/viewer/2022042007/5e70ae76fbd42672be50f7c4/html5/thumbnails/36.jpg)
Visualizing change with aggregated data
Page 36
![Page 37: Change Tracking in Knowledge Organization Systems with ......Three basic steps to an actionable skos-history Start with one SKOS file per version. 1) Create the deltas - insertions](https://reader033.vdocuments.us/reader033/viewer/2022042007/5e70ae76fbd42672be50f7c4/html5/thumbnails/37.jpg)
Page 37
![Page 38: Change Tracking in Knowledge Organization Systems with ......Three basic steps to an actionable skos-history Start with one SKOS file per version. 1) Create the deltas - insertions](https://reader033.vdocuments.us/reader033/viewer/2022042007/5e70ae76fbd42672be50f7c4/html5/thumbnails/38.jpg)
Drill down from chart to change report
Page 38
![Page 39: Change Tracking in Knowledge Organization Systems with ......Three basic steps to an actionable skos-history Start with one SKOS file per version. 1) Create the deltas - insertions](https://reader033.vdocuments.us/reader033/viewer/2022042007/5e70ae76fbd42672be50f7c4/html5/thumbnails/39.jpg)
Future work and the skos-history project
Apply to differing concept schemes Distill general properties useful for human-readable change
reports as well as machine-actionable data Get a grip on clusters of interrelated changes
Please consider joining – particularly if
you are in charge of a KOS and want to publish its change history you are using one or several KOS in an application, or
intellectually, and want to trace and re-apply upstream changes just feel challenged by the task
Page 39
![Page 40: Change Tracking in Knowledge Organization Systems with ......Three basic steps to an actionable skos-history Start with one SKOS file per version. 1) Create the deltas - insertions](https://reader033.vdocuments.us/reader033/viewer/2022042007/5e70ae76fbd42672be50f7c4/html5/thumbnails/40.jpg)
Page 40
Thanks for listening!
Joachim NeubertZBW – Leibniz Information Centre for [email protected]
Osma SuominenThe National Library of [email protected]
Project repository: https://github.com/jneubert/skos-history