accurator: acquisition of qualitative annotations from niche communities
DESCRIPTION
MCN2014 Panel presentation on Accurator: Acquisition of Qualitative Annotations from Niche Communities.TRANSCRIPT
![Page 1: Accurator: Acquisition of Qualitative Annotations from Niche Communities](https://reader033.vdocuments.us/reader033/viewer/2022042816/559b32111a28abe5568b459b/html5/thumbnails/1.jpg)
Henrike Hövelmann
AccuratorAcquisition of Qualitative Annotations
from Niche Communities
Chris Dijkshoorn
![Page 2: Accurator: Acquisition of Qualitative Annotations from Niche Communities](https://reader033.vdocuments.us/reader033/viewer/2022042816/559b32111a28abe5568b459b/html5/thumbnails/2.jpg)
Collaboration between museum and research institutes
Nichesourcing of annotations
Accurator system sneak preview
Datasets used in Accurator
Evaluation
Future work
Accurator
![Page 3: Accurator: Acquisition of Qualitative Annotations from Niche Communities](https://reader033.vdocuments.us/reader033/viewer/2022042816/559b32111a28abe5568b459b/html5/thumbnails/3.jpg)
Collection
‣ Collection of ~1,000,000 objects ‣ Print collection of ~550.000 prints ‣ Artworks on display ~8.000
Online Collection
‣ Accessible through API ‣ Published as Linked Data ‣ 548,785 object records ‣ 424,098 prints ‣ 212,517 works are digitised
Rijksmuseum Amsterdam
![Page 4: Accurator: Acquisition of Qualitative Annotations from Niche Communities](https://reader033.vdocuments.us/reader033/viewer/2022042816/559b32111a28abe5568b459b/html5/thumbnails/4.jpg)
Make artworks accessible online
Professional catalogers and photographers
Process
‣ Register artwork ‣ Provide annotations ‣ Digitise artwork ‣ Publish online
~40,000 new object records a year
Print Room Online Project
![Page 5: Accurator: Acquisition of Qualitative Annotations from Niche Communities](https://reader033.vdocuments.us/reader033/viewer/2022042816/559b32111a28abe5568b459b/html5/thumbnails/5.jpg)
Focus on art-historical information
Occasional lack of expertise regarding subject matter annotations
This print is described as:
‣ “Bird with blue head” ‣ “Branch with red leaves”
Annotating Artworks
![Page 6: Accurator: Acquisition of Qualitative Annotations from Niche Communities](https://reader033.vdocuments.us/reader033/viewer/2022042816/559b32111a28abe5568b459b/html5/thumbnails/6.jpg)
Socially-Enriched Access to Linked Cultural Media Diverse collaboration
‣ 4 Research institutes ‣ 5 Cultural heritage institutions ‣ 5 Industry partners
4 PhDs are working on Accurator
‣ Crowdsourcing the annotation process
‣ Finding niche groups with required expertise
research project
![Page 7: Accurator: Acquisition of Qualitative Annotations from Niche Communities](https://reader033.vdocuments.us/reader033/viewer/2022042816/559b32111a28abe5568b459b/html5/thumbnails/7.jpg)
State of the art
‣ Steve.museum ‣ YourPaintings
Rijksmuseum allowed tagging on website
Tagging
![Page 8: Accurator: Acquisition of Qualitative Annotations from Niche Communities](https://reader033.vdocuments.us/reader033/viewer/2022042816/559b32111a28abe5568b459b/html5/thumbnails/8.jpg)
Use nichesourcing to gather annotations
‣ Find niche groups with required expertise
‣ Use the knowledge of expert laymen
‣ Cluster tasks based on topic
Beyond crowdsourcing
![Page 9: Accurator: Acquisition of Qualitative Annotations from Niche Communities](https://reader033.vdocuments.us/reader033/viewer/2022042816/559b32111a28abe5568b459b/html5/thumbnails/9.jpg)
TrustClusterSearch
User Modelling
Crowd
Cultural Heritage
Collection
Annotations
NicheExperts
NicheExperts
NicheExperts
Accurator System
![Page 10: Accurator: Acquisition of Qualitative Annotations from Niche Communities](https://reader033.vdocuments.us/reader033/viewer/2022042816/559b32111a28abe5568b459b/html5/thumbnails/10.jpg)
NicheExpert
Japanese
Redstart
Nichesourcing Annotations
![Page 11: Accurator: Acquisition of Qualitative Annotations from Niche Communities](https://reader033.vdocuments.us/reader033/viewer/2022042816/559b32111a28abe5568b459b/html5/thumbnails/11.jpg)
Accurator screencast
![Page 12: Accurator: Acquisition of Qualitative Annotations from Niche Communities](https://reader033.vdocuments.us/reader033/viewer/2022042816/559b32111a28abe5568b459b/html5/thumbnails/12.jpg)
Base choice on gaps in collection
Started with two topics
‣ Castles ‣ Flowers
Contacted multiple groups, but low response
Change the focus
Which niche group to address?
![Page 13: Accurator: Acquisition of Qualitative Annotations from Niche Communities](https://reader033.vdocuments.us/reader033/viewer/2022042816/559b32111a28abe5568b459b/html5/thumbnails/13.jpg)
Focus on birds
![Page 14: Accurator: Acquisition of Qualitative Annotations from Niche Communities](https://reader033.vdocuments.us/reader033/viewer/2022042816/559b32111a28abe5568b459b/html5/thumbnails/14.jpg)
Specimen Collection
‣ Songbirds 3.210 specimen
Print Collection
‣ Koekkoek 406 prints
Thesaurus
‣ IOC World Bird List 34,197 concepts
Naturalis biodiversity data
![Page 15: Accurator: Acquisition of Qualitative Annotations from Niche Communities](https://reader033.vdocuments.us/reader033/viewer/2022042816/559b32111a28abe5568b459b/html5/thumbnails/15.jpg)
Open Annotation Data Model for storing information
Semantic support for annotators
‣ Autocompletion from structured vocabularies
‣ IOC world bird list in Accurator
Semantic Annotation
![Page 16: Accurator: Acquisition of Qualitative Annotations from Niche Communities](https://reader033.vdocuments.us/reader033/viewer/2022042816/559b32111a28abe5568b459b/html5/thumbnails/16.jpg)
Semantic Annotation - Data model
rdf:value
dc:title skos:prefLabel
hdl:504055 ico: 25F34
"Eagle owl in magnolia"@en
"owls"@en
edm:aggregated
CHOagg:COLLEC
T.504055
edm:isShownBy
dc:subject
ann:id_6111fedf002e9
oa:hasTarget
birds:species-bubo_bubo
oa:hasBody
genid:target_44478b9e
oa:hasTarget
oa:hasSource
#xywh=percent:19,40,39,27
ann:h"0.27"^^xsd:float
ann:w "0.38"^^xsd:float
ann:x "0.18"^^xsd:floatann:y
"0.40"^^xsd:float
![Page 17: Accurator: Acquisition of Qualitative Annotations from Niche Communities](https://reader033.vdocuments.us/reader033/viewer/2022042816/559b32111a28abe5568b459b/html5/thumbnails/17.jpg)
Compare provided annotations to correct answers
‣ Define correct answer ‣ Collect “ground truth”
Experts tend to disagree
Obtained a set of Naturalis bird prints with scientific names
Evaluation results
“The frog stands for the rejected lover”
“Sexy ladies”
“Women”
“The frog indicates the profession of the women”
![Page 18: Accurator: Acquisition of Qualitative Annotations from Niche Communities](https://reader033.vdocuments.us/reader033/viewer/2022042816/559b32111a28abe5568b459b/html5/thumbnails/18.jpg)
Run experiment with ground truth
Incorporate automatic trust assessment in Accurator
Show effects of annotating by leveraging it for collection integration
Future work
![Page 19: Accurator: Acquisition of Qualitative Annotations from Niche Communities](https://reader033.vdocuments.us/reader033/viewer/2022042816/559b32111a28abe5568b459b/html5/thumbnails/19.jpg)
Possibilities for integration - Rijksmuseum Website
![Page 20: Accurator: Acquisition of Qualitative Annotations from Niche Communities](https://reader033.vdocuments.us/reader033/viewer/2022042816/559b32111a28abe5568b459b/html5/thumbnails/20.jpg)
PrintKono Bairei
Possibilities for integration - Dutch Species
![Page 21: Accurator: Acquisition of Qualitative Annotations from Niche Communities](https://reader033.vdocuments.us/reader033/viewer/2022042816/559b32111a28abe5568b459b/html5/thumbnails/21.jpg)
Chris Dijkshoorn [email protected] http://chrisdijkshoorn.nl
Accurator Acquisition of Qualitative Annotations from Niche Communities
Henrike Hövelmann [email protected]