expand. learn. interact: enabling digital humanities
DESCRIPTION
Talk at the National eScience Symposium, The Netherlands http://www.esciencecenter.nl/symposiumTRANSCRIPT
Expand. Learn. Interact. Enabling (Digital) Humani9es
http://lora-aroyo.org ! http://slideshare.net/laroyo ! @laroyo
Lora Aroyo VU University Amsterdam
“The Gallery of Cornelis van der Geest” (Willem van Haecht)
http://lora-aroyo.org ! http://slideshare.net/laroyo ! @laroyo
http://lora-aroyo.org ! http://slideshare.net/laroyo ! @laroyo
is a rich digital space
to explore TOPICS, e.g hun9ng, dogs
Quen9n Metsys The Madonna and Child
to explore ARTISTS, e.g. their pain9ngs, biographies, links
to explore EVENTS, e.g types, par9cipants, links
http://lora-aroyo.org ! http://slideshare.net/laroyo ! @laroyo
Mul9-‐dimensional
“Jungle of scales” Mul9-‐granularity
Mul9-‐perspec9ve
Mul9tude of collec9ons
Digital Heritage Mission
Lora Aroyo
“We enable everyone to u/lize the collec/ons to learn, experience & create.” (mission Sound and Vision) “a place where a na/on nourishes its memory, and exerts its imagina/on -‐ where it connects with its past and invents its future” (Pierre Ryckmans, 1996) „places that foster curiosity, celebrate ideas and enable discovery.” (sxswLAM community, hMp://www.sxswlam.com/) Help people
finding their way
Informa(on: Museums & Archives as Inventories of the World
André Malraux, The Imaginary Museum of World Sculpture, 1953
Interpreta(on: Museums & Archives as a Place to Engage with the World
effec/ve & efficient in delivering change and tangible benefits strategic advantage through the innova/on inherent in this digital ac/vity economic benefit to community that demonstrate the value of endeavours in monetary terms ( the community has been changed by the resource in beneficial ways that can be clearly iden/fied
Source: Simon Tanner, Kings College London , hBp://www.kdcs.kcl.ac.uk/innovaGon/impact.html
http://lora-aroyo.org ! http://slideshare.net/laroyo ! @laroyo
New Era of Compu9ng
Lora Aroyo
• amount of data & scale of computa9on available have increased by a previously inconceivable amount
• CS & AI moved out of thought problems – entered empirical science
IBM Confiden/al
Open Domain Question-Answering Machine, that given – Rich Natural Language Questions – Over a Broad Domain of Knowledge
Won a 2-game Jeopardy match against the all-time winners – viewed by over 50,000,000
Expand. Learn. Interact.
Enabling (Digital) Humani9es
http://lora-aroyo.org ! http://slideshare.net/laroyo ! @laroyo
How to Do It?
Lora Aroyo
1. Expand
“EXPAND human cogni/on, makes the jobs we do easier, like a cogni9ve prosthesis, especially when dealing with processing massive data, or data that requires human interpreta9on”
• Support for Interpreta9on • Structured Seman9cs
… Digital Humanities researchers
http://lora-aroyo.org ! http://slideshare.net/laroyo ! @laroyo
… often don’t find what they were searching for
http://lora-aroyo.org ! http://slideshare.net/laroyo ! @laroyo
HOW DO WE FILL THE GAP?
http://lora-aroyo.org ! http://slideshare.net/laroyo ! @laroyo
Concep9ons of eHuman9es
INTERPRETATION
• a theory of interpretaGon of informaGon • bringing people and technology together to explore:
– how to model and represent informa/on – how to provide engaging interac/on – how to support interpreta/on
“Digital HermeneuGcs: Agora and the online understanding of cultural heritage” In proceedings of Web Science
Conference, (ACM: New York, 2011)
Digital Hermeneu9cs
Simple Event Model (SEM) OpenAnnota/on (OA) & SKOS
DIVE:MEDIA OBJECT
SEM:EVENT
SEM:PLACE
SEM:TIME
SEM:ACTOR
SKOS:CONCEPT
OA:ANNOTATION
• Links to EUROPEANA (mul/lingual) • Links to DBPedia, other vocabularies
Erp, M. van; Oomen, J.; Segers, R.; Akker, C. van de; Aroyo, L.; Jacobs, G.; Legêne, S; Meij, L. van der;O ssenbruggen, J.R. van; Schreiber, G. Automa/c Heritage Metadata Enrichment with Historic Events Museums and the Web 2011 hMp://www.museumsandtheweb.com/mw2011/papers/automa/c_heritage_metadata_enrichment_with_hi
engaging users
through event
narratives
hMp://sealinc.ops.few.vu.nl/invenit/
Exploring with Structured Seman9cs
CLUSTERING ! RANKING ! TRUST ! INTEREST
hMp://sealinc.ops.few.vu.nl/invenit/
Lora Aroyo
“LEARN as you use it – most machine errors are easy for a human to detect, and we can instrument usage of systems to beMer understand the system and the problem it solves”
2. Learn
• Teach machines • Deal with diversity of interpreta9on
Teaching Machines
http://lora-aroyo.org ! http://slideshare.net/laroyo ! @laroyo
Diversity, Opinions, Interpreta9ons
http://lora-aroyo.org ! http://slideshare.net/laroyo ! @laroyo
Human Disagreement IS Essential in Helping Machines with Semantic Interpretation!
http://lora-aroyo.org ! http://slideshare.net/laroyo ! @laroyo
“best collec9ve decisions are result of disagreement,
not consensus or compromise” James Surowiecki
Crowdsourcing Diversity of Interpreta9on
A novel approach to gather diversity of perspec9ves & opinions from the crowd, expand expert vocabularies with these and gather new type of gold standard for machines
L. Aroyo, C. Welty: Crowd Truth: Harnessing disagreement in crowdsourcing a rela/on extrac/on gold standard. ACM WebSci 2013.
CrowdTruth.org
http://lora-aroyo.org ! http://slideshare.net/laroyo ! @laroyo
Crowdsourcing for Video Analysis
http://lora-aroyo.org ! http://slideshare.net/laroyo ! @laroyo
CrowdTruth.org
Following the grandeur of Baroque, Rococo art is often dismissed as frivolous and unserious, but Waldemar Januszczak disagrees. […] The first episode is about travel in the 18th century and how it impacted greatly on some of the finest art ever made. The world was getting smaller and took on new influences shown in the glorious Bavarian pilgrimage architecture, Canaletto's romantic Venice and the blossoming of exotic designs and tastes all over Europe.
Rococo: Travel, pleasure, madness
A boarding school where boys from the Dutch East Indies receive a voca/onal educa/on has been set up by the ministry of Social Affairs near Batavia. Trumpeter sounds the reveille; -‐ the boys get out of bed; -‐ the muster is held in front of the building; akerwards the boys stand in line with a food bowl; they get rice pudding and eat this around long tables in the open air; -‐ the boys get instruc/ons and prac/ce a.o. drawing shapes, forging, metalworking, filing and woodworking;
News from Indonesia: youth care
hdp://vista-‐tv.eu
hdp://dive.beeldengeluid.nl
Crowdsourcing for Image Analysis
http://lora-aroyo.org ! http://slideshare.net/laroyo ! @laroyo
CrowdTruth.org
Crowdsourcing for Text Analysis
http://lora-aroyo.org ! http://slideshare.net/laroyo ! @laroyo
CrowdTruth.org
• One truth: data collec/on efforts assume one correct interpreta/on for every example
• All examples are created equal: ground truth treats all examples the same – either match the correct result or not
• Detailed guidelines help: if examples cause disagreement -‐ add instruc/ons to limit interpreta/ons
• Disagreement is bad: increase quality of annota/on data by reducing disagreement among the annotators
• One is enough: most of the annotated examples are evaluated by one person
• Experts are beder: annotators with domain knowledge provide beMer annota/ons
• Once done, forever valid: annota/ons are not updated; new data not aligned with previous
7 Myths of Human Annota9on
myths directly influence the prac/ce of collec/ng human annotated data; Need to be revisited in the context of new changing world & in the face of a new
theory of truth (CrowdTruth)
Lora Aroyo
Lora Aroyo
“INTERACT naturally. We need to bring machines closer to their users, we have adapted ourselves enough to them, they should understand natural language, spoken or wriMen, be able to process images and videos. These simple human problems are extremely complex for machines, but are hallmarks of a new compu9ng era.”
3. Interact
• Gathering Human Seman9cs • Engaging Users
Engage the Crowd in Cultural Heritage
hMp://2-‐dot-‐rma-‐accurator.appspot.com/
Engage the Niche in Cultural Heritage
Engage with Games in Cultural Heritage
Engage with Games in Sciences
http://lora-aroyo.org ! http://slideshare.net/laroyo ! @laroyo
hdp://game.crowdtruth.org/
http://lora-aroyo.org ! http://slideshare.net/laroyo ! @laroyo
hdp://waisda.nl/ hdp://spotvogel.vroegevogels.vara.nl/
Engage with Games in Entertainment
Engage with Novel Interfaces Event-‐based Browsing Of Linked Historical Media
dive.beeldengeluid.nl
3rd Price at the Seman9c Web Challenge 2014
OPENIMAGES.EU • 3000 videos • NL Ins/tute for Sound & Vision • mostly news broadcasts
DELPHER.NL • 1.5 Million Scans of • Radio bulle/ns • (hand annotated) • 1937 – 1984
Digital Submarine UI Infinity of Explora9on
Events Linking Objects
Crowd Bringing Human Perspec9ves
Linked (Open) Data
Expand. Learn. Interact.
Enabling (Digital) Humani9es
http://lora-aroyo.org ! http://slideshare.net/laroyo ! @laroyo
The Road Ahead
Lora Aroyo
• Quality measures • Capturing Trust • Dealing with Provenance • Complexity of Analy/cs • Workflows for Impact • Ac/ve Learning
Technology Challenges
Lora Aroyo
• Influence directly Heritage organiza/ons
• Interdisciplinary research with computer sciences
• Smart, connected, open heritage
• Visible, usable, durable heritage
• Shared knowledge infrastructure
Digital Humani9es Roadmap
Agora Team: Lora Aroyo, Guus Schreiber, Lourens van der Meij, Marieke van Erp, Chiel van den Akker, Susan Legêne, Geertje Jacobs,
Johan Oomen
DIVE Team: Victor de Boer, Oana Inel, Lora Aroyo, Johan Oomen, Elco Van Staveren, Werner Helmich & Dennis De Beurs
dive.beeldengeluid.nl
SealincMedia Team: Alessandro Bozzon, Geert-‐Jan Houben, Lora Aroyo, Lizzy Jongma, Guus Schreiber, Chris Dijkshoorn, Jasper Oosterman, Jacco van Ossenbruggen, Archana NoMamkandath,
Myriam Traub
http://sealinc.ops.few.vu.nl/invenit/
CrowdTruth Team: Lora Aroyo, Chris Welty Robert-‐Jan Sips, Carlos Mar9nez Or9z, Anca Dumitrache, Oana Inel, Benjamin Timmermans, Susanna van de Ven, Merel van Empel, Jelle v.d. Ploeg, Ta/ana Cristea, Khalid Khamkham, HarriëMe Smook, Rens van Honschooten, Arne Rutjes
CrowdTruth.org github.com/CrowdTruth
Links
http://lora-aroyo.org ! http://slideshare.net/laroyo ! @laroyo
On the Web • http://waida.nl • http://prestoprime.org • http://agora.cs.vu.nl • http://sealincmedia.wordpress.com • http://notube.tv • http://vista-tv.eu • http://dive.beeldengeluid.nl • http://crowdtruth.org • http://game.crowdtruth.org • http://wm.cs.vu.nl
On Twider @waisda @agora-‐project @sealincmedia @prestocenter @notubeproject @vistatv #CrowdTruth
lora-aroyo.org slideshare.com/laroyo
@laroyo