Download - Research and Development at Sound and Vision
![Page 1: Research and Development at Sound and Vision](https://reader035.vdocuments.us/reader035/viewer/2022070322/55910f971a28ab8b2f8b4607/html5/thumbnails/1.jpg)
Research and Development at Sound and Vision
Victor de Boer
[email protected] [email protected]
Met slides van Johan Oomen, Lotte Belice Baltussen, Maarten Brinkerink, Bouke Huurnink
![Page 2: Research and Development at Sound and Vision](https://reader035.vdocuments.us/reader035/viewer/2022070322/55910f971a28ab8b2f8b4607/html5/thumbnails/2.jpg)
![Page 3: Research and Development at Sound and Vision](https://reader035.vdocuments.us/reader035/viewer/2022070322/55910f971a28ab8b2f8b4607/html5/thumbnails/3.jpg)
Mediapark Hilversum
Ontstaan in 1997: 3 archieven (RVD,
AVAC/Omroepen, Stichting Film en Wetenschap)
1 museum (Omroepmuseum)
Grootste audiovisueel archief
![Page 4: Research and Development at Sound and Vision](https://reader035.vdocuments.us/reader035/viewer/2022070322/55910f971a28ab8b2f8b4607/html5/thumbnails/4.jpg)
De Opdracht1. “Beheren en bewaken van dé audiovisuele
schatkamer van Nederland”
Cultuurhistorisch instituut• Archief• Museum• Kenniscentrum
2. “Toegang tot deze schatkamer voor iedereen!”
• Archiefcollectie voor programmamakers, onderwijs
• Media Experience voor algemeen publiek
![Page 5: Research and Development at Sound and Vision](https://reader035.vdocuments.us/reader035/viewer/2022070322/55910f971a28ab8b2f8b4607/html5/thumbnails/5.jpg)
14 april 2023
Nederlands Instituut voor Beeld en Geluid
5
70% audiovisueel erfgoed > 800.000 uur
250 000 uur televisie150 000 uur radio
300 000 uur muziek50 000 uur docu, film, reclame, etc
![Page 7: Research and Development at Sound and Vision](https://reader035.vdocuments.us/reader035/viewer/2022070322/55910f971a28ab8b2f8b4607/html5/thumbnails/7.jpg)
![Page 8: Research and Development at Sound and Vision](https://reader035.vdocuments.us/reader035/viewer/2022070322/55910f971a28ab8b2f8b4607/html5/thumbnails/8.jpg)
![Page 9: Research and Development at Sound and Vision](https://reader035.vdocuments.us/reader035/viewer/2022070322/55910f971a28ab8b2f8b4607/html5/thumbnails/9.jpg)
Archieftaak in het digitale domein
Archieftaak ook in non-lineair medialandschap adequaat vervullen;
Internet en mobiele technologie hebben de manier waarop we communiceren volledig veranderd;
Audiovisueel materiaal: van zeldzaam tijdsdocument tot ‘preferred’ informatiedrager;
Archiveringsproces in digitaal tijdperk heel anders.
![Page 10: Research and Development at Sound and Vision](https://reader035.vdocuments.us/reader035/viewer/2022070322/55910f971a28ab8b2f8b4607/html5/thumbnails/10.jpg)
“The Archive as a Laboratory”
![Page 11: Research and Development at Sound and Vision](https://reader035.vdocuments.us/reader035/viewer/2022070322/55910f971a28ab8b2f8b4607/html5/thumbnails/11.jpg)
• Exchange– Interoperability between collections– Open Data– Linking collection to semantic web
• Access– Annotation– Retrieval– Contextualization
• Knowledge about end-users– Interviews– Experiments– Log studies
Research and Development
![Page 12: Research and Development at Sound and Vision](https://reader035.vdocuments.us/reader035/viewer/2022070322/55910f971a28ab8b2f8b4607/html5/thumbnails/12.jpg)
• Exchange– Interoperability between collections– Open Data– Linking collection to semantic web
• Access– Annotation– Retrieval– Contextualization
• Knowledge about end-users– Interviews– Experiments– Log studies
Research and Development
![Page 13: Research and Development at Sound and Vision](https://reader035.vdocuments.us/reader035/viewer/2022070322/55910f971a28ab8b2f8b4607/html5/thumbnails/13.jpg)
Thanks to the proliferation of channels and the portability of new computing and telecommunications technologies, we are entering an era where media will be everywhere. (Jenkins, 2006)
The core design principle underlying the Web’s usefulness and growth is openness and universality. (Oomen & Aroyo, 2011)
![Page 14: Research and Development at Sound and Vision](https://reader035.vdocuments.us/reader035/viewer/2022070322/55910f971a28ab8b2f8b4607/html5/thumbnails/14.jpg)
• A lot of cultural heritage is digisited or is being digitised.
• Consequently, it is brought online. (Or?)• The archival turn: “[...] material that was until
recently locked into archival vaults and mainly used by professionals has now become available and accessible to non-industry users.” -- De Leeuw, 2012
14-04-2023
INTRODUCTION
![Page 15: Research and Development at Sound and Vision](https://reader035.vdocuments.us/reader035/viewer/2022070322/55910f971a28ab8b2f8b4607/html5/thumbnails/15.jpg)
TYPES OF VALUE
1. Economic (income)2. Public reach and access3. Re-use4. Participation
Focus of most CH organisations is on 2, 3 and 4
14-04-2023
![Page 16: Research and Development at Sound and Vision](https://reader035.vdocuments.us/reader035/viewer/2022070322/55910f971a28ab8b2f8b4607/html5/thumbnails/16.jpg)
2. PUBLIC REACH AND ACCESS
• Number of (online) visitors• Number of app downloads• Examples: website visits, project website visits,
engagement on Twitter and Facebook, repins on Pinterest.
14-04-2023
![Page 17: Research and Development at Sound and Vision](https://reader035.vdocuments.us/reader035/viewer/2022070322/55910f971a28ab8b2f8b4607/html5/thumbnails/17.jpg)
3. RE-USE
• Re-use of collections by an organisation itself, and by the public.
• e.g. developments the last years between the CH world and open initiatives, Europeana API
14-04-2023
![Page 18: Research and Development at Sound and Vision](https://reader035.vdocuments.us/reader035/viewer/2022070322/55910f971a28ab8b2f8b4607/html5/thumbnails/18.jpg)
4. PARTICIPATION
14-04-2023
• Participatory culture• From engagement to creative re-use
![Page 19: Research and Development at Sound and Vision](https://reader035.vdocuments.us/reader035/viewer/2022070322/55910f971a28ab8b2f8b4607/html5/thumbnails/19.jpg)
PARTICIPATION
14-04-2023
Oomen & Aroyo (2011)
![Page 20: Research and Development at Sound and Vision](https://reader035.vdocuments.us/reader035/viewer/2022070322/55910f971a28ab8b2f8b4607/html5/thumbnails/20.jpg)
14-04-2023
BEING OPEN: CONTRIBUTING TO THE COMMONS
Many of the projects we and others in the CH field do that have the aim to increase access, re-use and participation are ‘open’:
“A piece of content or data is open if anyone is free to use, reuse, and redistribute it — subject only, at most, to the requirement to attribute and/or share-alike.” --http://opendefinition.org/
![Page 21: Research and Development at Sound and Vision](https://reader035.vdocuments.us/reader035/viewer/2022070322/55910f971a28ab8b2f8b4607/html5/thumbnails/21.jpg)
14-04-2023
BEING OPEN: A GENERAL TREND
• The open data movement => New EU directive on open data
• Global mass digitisation• Example: Comité des Sages report• Example: Europeana API
![Page 22: Research and Development at Sound and Vision](https://reader035.vdocuments.us/reader035/viewer/2022070322/55910f971a28ab8b2f8b4607/html5/thumbnails/22.jpg)
14-04-2023
BEING OPEN: CREATIVE COMMONS LICENSES
![Page 23: Research and Development at Sound and Vision](https://reader035.vdocuments.us/reader035/viewer/2022070322/55910f971a28ab8b2f8b4607/html5/thumbnails/23.jpg)
Open Images is an open media platform that offers online access to audiovisual archive material to stimulate creative reuse.
Built by Sound and Vision & Knowledgeland but designed for participation by others.
![Page 24: Research and Development at Sound and Vision](https://reader035.vdocuments.us/reader035/viewer/2022070322/55910f971a28ab8b2f8b4607/html5/thumbnails/24.jpg)
• Open source (MMBase, FFmpeg, LAMP)
• Open media formats (Ogg Theora, WebM)
• Open standards (Dublin Core, CC-REL, HTML 5)
• Open API (OAI-PMH, CC-0)
• Open content (CC-licenses, PD Mark)
OPEN, OPEN, OPEN!
14-04-2023
OPEN, OPEN, OPEN!
![Page 25: Research and Development at Sound and Vision](https://reader035.vdocuments.us/reader035/viewer/2022070322/55910f971a28ab8b2f8b4607/html5/thumbnails/25.jpg)
Openbeelden.nl / openimages.eu
• CC BY - SA as preferable license
• 3000 items• “Internet
Quality”
![Page 26: Research and Development at Sound and Vision](https://reader035.vdocuments.us/reader035/viewer/2022070322/55910f971a28ab8b2f8b4607/html5/thumbnails/26.jpg)
∼800,000h
∼110h
![Page 27: Research and Development at Sound and Vision](https://reader035.vdocuments.us/reader035/viewer/2022070322/55910f971a28ab8b2f8b4607/html5/thumbnails/27.jpg)
Objectives• Public outreach by embracing new
technologies and ‘participatory culture’• Contextualization by interlinking with
other platforms• Exploring new services and distribution
models• Supporting a National and European
Audiovisual Commons
![Page 28: Research and Development at Sound and Vision](https://reader035.vdocuments.us/reader035/viewer/2022070322/55910f971a28ab8b2f8b4607/html5/thumbnails/28.jpg)
MW2012
![Page 29: Research and Development at Sound and Vision](https://reader035.vdocuments.us/reader035/viewer/2022070322/55910f971a28ab8b2f8b4607/html5/thumbnails/29.jpg)
![Page 30: Research and Development at Sound and Vision](https://reader035.vdocuments.us/reader035/viewer/2022070322/55910f971a28ab8b2f8b4607/html5/thumbnails/30.jpg)
Hackathons
http://www.beeldvoorbeeld.nl/tv/
![Page 31: Research and Development at Sound and Vision](https://reader035.vdocuments.us/reader035/viewer/2022070322/55910f971a28ab8b2f8b4607/html5/thumbnails/31.jpg)
![Page 32: Research and Development at Sound and Vision](https://reader035.vdocuments.us/reader035/viewer/2022070322/55910f971a28ab8b2f8b4607/html5/thumbnails/32.jpg)
• Exchange– Interoperability between collections– Open Data– Linking collection to semantic web
• Access– Annotation– Retrieval– Contextualization
• Knowledge about end-users– Interviews– Experiments– Log studies
Research and Development
![Page 33: Research and Development at Sound and Vision](https://reader035.vdocuments.us/reader035/viewer/2022070322/55910f971a28ab8b2f8b4607/html5/thumbnails/33.jpg)
Annotation today
• INSERT PICTURE OF IMMIX– +-15 archivists, with changing role– Speech recognition and radio streaming in
![Page 35: Research and Development at Sound and Vision](https://reader035.vdocuments.us/reader035/viewer/2022070322/55910f971a28ab8b2f8b4607/html5/thumbnails/35.jpg)
From 2015 this should happen automatically…
Current situation: Video ingested from broadcasters, documentalists annotate
using thesaurus terms
![Page 36: Research and Development at Sound and Vision](https://reader035.vdocuments.us/reader035/viewer/2022070322/55910f971a28ab8b2f8b4607/html5/thumbnails/36.jpg)
….…..…….….. ? “Ezel”
How could you do this?
![Page 37: Research and Development at Sound and Vision](https://reader035.vdocuments.us/reader035/viewer/2022070322/55910f971a28ab8b2f8b4607/html5/thumbnails/37.jpg)
How?
Image recognitionSpeaker recognitionSpeech recognition
Broadcasting guidesCrowdsourcingTT888 subtitles
….…..…….….. ? “Ezel”
How could you do this?
![Page 38: Research and Development at Sound and Vision](https://reader035.vdocuments.us/reader035/viewer/2022070322/55910f971a28ab8b2f8b4607/html5/thumbnails/38.jpg)
Term extraction from TT888 subtitles
![Page 39: Research and Development at Sound and Vision](https://reader035.vdocuments.us/reader035/viewer/2022070322/55910f971a28ab8b2f8b4607/html5/thumbnails/39.jpg)
Algoritme
Wordf requencies Dutch
Named Entity Recognition
Thesaurus B&G
“Ezel”“Amsterdam”“Jos Brink”
![Page 40: Research and Development at Sound and Vision](https://reader035.vdocuments.us/reader035/viewer/2022070322/55910f971a28ab8b2f8b4607/html5/thumbnails/40.jpg)
• Compare to manual terms from documentalists• What (type of) terms are extracted (or not)• Are the automatically extracted ones better?• End user test
Evaluation: How well does the algorithm work?
![Page 41: Research and Development at Sound and Vision](https://reader035.vdocuments.us/reader035/viewer/2022070322/55910f971a28ab8b2f8b4607/html5/thumbnails/41.jpg)
Crowdsourcing
![Page 43: Research and Development at Sound and Vision](https://reader035.vdocuments.us/reader035/viewer/2022070322/55910f971a28ab8b2f8b4607/html5/thumbnails/43.jpg)
![Page 44: Research and Development at Sound and Vision](https://reader035.vdocuments.us/reader035/viewer/2022070322/55910f971a28ab8b2f8b4607/html5/thumbnails/44.jpg)
COGNITIVE SURPLUS
The so-called “cognitive surplus” that used to be spent on passive activities (notably watching television) can now be used in a profoundly different way, for new kinds of creativity and problem-solving. (Oomen & Aroyo, 2011, Clay Shirky 2010)
14-04-2023
![Page 45: Research and Development at Sound and Vision](https://reader035.vdocuments.us/reader035/viewer/2022070322/55910f971a28ab8b2f8b4607/html5/thumbnails/45.jpg)
WAISDA? (What’s That?)
- Game-With-a-Purpose (GWAP)- Allows internet users to annotate audiovisual archive
material in the form of a (serious) game- The goal of the game is consensus between players
(which also works as a filter)- Fun and competition as motivation
14-04-2023
![Page 46: Research and Development at Sound and Vision](https://reader035.vdocuments.us/reader035/viewer/2022070322/55910f971a28ab8b2f8b4607/html5/thumbnails/46.jpg)
GOALS AND ADDED VALUE
- Investigate the added value of social tagging- Experimenting with new forms of services for the
public (serious games)Which results in:- Time-related metadata- Social tagging (bridging the semantic gap)- Interaction between the archive/broadcaster and the
public
14-04-2023
![Page 47: Research and Development at Sound and Vision](https://reader035.vdocuments.us/reader035/viewer/2022070322/55910f971a28ab8b2f8b4607/html5/thumbnails/47.jpg)
![Page 49: Research and Development at Sound and Vision](https://reader035.vdocuments.us/reader035/viewer/2022070322/55910f971a28ab8b2f8b4607/html5/thumbnails/49.jpg)
GAME MECHANICS 2/5
![Page 50: Research and Development at Sound and Vision](https://reader035.vdocuments.us/reader035/viewer/2022070322/55910f971a28ab8b2f8b4607/html5/thumbnails/50.jpg)
GAME MECHANICS 4/5
![Page 51: Research and Development at Sound and Vision](https://reader035.vdocuments.us/reader035/viewer/2022070322/55910f971a28ab8b2f8b4607/html5/thumbnails/51.jpg)
GAME MECHANICS 5/5
![Page 52: Research and Development at Sound and Vision](https://reader035.vdocuments.us/reader035/viewer/2022070322/55910f971a28ab8b2f8b4607/html5/thumbnails/52.jpg)
RESULTS AND FINDINGS 1/2
- Three implementations resulted in over a million social tags (ongoing)
- On average 40-50% of the social tags consists of matched tags
- On average 10-20% of the social tags are unique- ‘Super taggers’ are responsible for the vast majority
of the social tags that are added
14-04-2023
![Page 53: Research and Development at Sound and Vision](https://reader035.vdocuments.us/reader035/viewer/2022070322/55910f971a28ab8b2f8b4607/html5/thumbnails/53.jpg)
RESULTS AND FINDINGS 2/2
- The extend to which expert cataloguers deem the social tags to be useful, heavily depends on the type of content
- The same is true for the balance between social tags the correspond with terms from a controlled-vocabulary and terms invented by users themselves
- First experiments suggest that the social tags enable high recall fragment retrieval.
14-04-2023
![Page 54: Research and Development at Sound and Vision](https://reader035.vdocuments.us/reader035/viewer/2022070322/55910f971a28ab8b2f8b4607/html5/thumbnails/54.jpg)
RETRIEVAL AND ACCESS
![Page 55: Research and Development at Sound and Vision](https://reader035.vdocuments.us/reader035/viewer/2022070322/55910f971a28ab8b2f8b4607/html5/thumbnails/55.jpg)
Retrieval RealityRetrieval reality
![Page 56: Research and Development at Sound and Vision](https://reader035.vdocuments.us/reader035/viewer/2022070322/55910f971a28ab8b2f8b4607/html5/thumbnails/56.jpg)
• Search with all sorts of annotations– Game, automatic– A/B tests
• Log analysis (Huurnink et al.)– Most queries are: [name of program] + [proper
name]• New forms of access
– LinkedTV– DIVE
Retrieval research
![Page 57: Research and Development at Sound and Vision](https://reader035.vdocuments.us/reader035/viewer/2022070322/55910f971a28ab8b2f8b4607/html5/thumbnails/57.jpg)
All information that can be used to interpret and understand the production, publication and reception of audiovisual
heritage from multiple standpoints.
metadata ≠ context source
CONTEXTUALISATION:AN AV-ARCHIVE DEFINITION
![Page 58: Research and Development at Sound and Vision](https://reader035.vdocuments.us/reader035/viewer/2022070322/55910f971a28ab8b2f8b4607/html5/thumbnails/58.jpg)
...TO CONTEXT: MUTUALLY CONNECTED COLLECTIONS...
14-04-2023
Connecting collections:topics, people, genres, etc
Catalogue Photos
B&G
Wiki
Programm
eguides
![Page 59: Research and Development at Sound and Vision](https://reader035.vdocuments.us/reader035/viewer/2022070322/55910f971a28ab8b2f8b4607/html5/thumbnails/59.jpg)
Video hyperlinking
Method to link a video to other multimedia sources Applications:
Search: connections between sources based on a query (clustering, storytelling)
Detail-on-demand: zooming in on specific elements through linked information sources (contextualisering)
12 februari 2013
![Page 60: Research and Development at Sound and Vision](https://reader035.vdocuments.us/reader035/viewer/2022070322/55910f971a28ab8b2f8b4607/html5/thumbnails/60.jpg)
Networked heritage
![Page 61: Research and Development at Sound and Vision](https://reader035.vdocuments.us/reader035/viewer/2022070322/55910f971a28ab8b2f8b4607/html5/thumbnails/61.jpg)
Concept: Jan Sluijters (schilder)DBpedia
Related items
Links
• Styles (Expressionism, Cubism, Fauvism)
• Period (contemporaries)
LinkedTV: Example of contextualization
![Page 62: Research and Development at Sound and Vision](https://reader035.vdocuments.us/reader035/viewer/2022070322/55910f971a28ab8b2f8b4607/html5/thumbnails/62.jpg)
LinkedTV – SmartTV
12 februari 2013
Cultureel erfgoed scenario, Tussen Kunst & KitschMet dank aan overeenkomst met AVRO!
![Page 63: Research and Development at Sound and Vision](https://reader035.vdocuments.us/reader035/viewer/2022070322/55910f971a28ab8b2f8b4607/html5/thumbnails/63.jpg)
DIVE• Linked Data
– B&G videos– Koninklijke
bibliotheek
• Event based– Context, Narratives
• New paradigms of explorative search and browsing
DIVE
![Page 64: Research and Development at Sound and Vision](https://reader035.vdocuments.us/reader035/viewer/2022070322/55910f971a28ab8b2f8b4607/html5/thumbnails/64.jpg)
![Page 66: Research and Development at Sound and Vision](https://reader035.vdocuments.us/reader035/viewer/2022070322/55910f971a28ab8b2f8b4607/html5/thumbnails/66.jpg)
• How well can we integrate these heterogeneous collections – How many (correct) links?
• How well can we extract knowledge from the descriptions with ensemble methods?
• Hoe werkbaar is dit type interface– Vinden mensen andere, nieuwe mediaobjecten?]
• Professionals• Amateurs
– Zijn ze hier tevreden over?– Krijgen ze een beter beeld over gebeurtenissen?
Evaluation
![Page 67: Research and Development at Sound and Vision](https://reader035.vdocuments.us/reader035/viewer/2022070322/55910f971a28ab8b2f8b4607/html5/thumbnails/67.jpg)
• Exchange– Interoperability between collections– Open Data– Linking collection to semantic web
• Access– Annotation– Retrieval– Contextualization
• Knowledge about end-users– Interviews– Experiments– Log studies
Research and Development
![Page 68: Research and Development at Sound and Vision](https://reader035.vdocuments.us/reader035/viewer/2022070322/55910f971a28ab8b2f8b4607/html5/thumbnails/68.jpg)
[email protected] | [email protected]
beeldengeluid.nlhttp://www.beeldengeluid.nl/blogs/Research-and-Development