r&d at sound and vision
TRANSCRIPT
![Page 1: R&D at Sound and Vision](https://reader034.vdocuments.us/reader034/viewer/2022052700/55a183ad1a28abe05a8b45b0/html5/thumbnails/1.jpg)
![Page 2: R&D at Sound and Vision](https://reader034.vdocuments.us/reader034/viewer/2022052700/55a183ad1a28abe05a8b45b0/html5/thumbnails/2.jpg)
R&D AT THE
NETHERLANDS INSTITUTE
FOR SOUND AND VISION
Bouke Huurnink
Lotte Belice Baltussen
5 April 2013
![Page 3: R&D at Sound and Vision](https://reader034.vdocuments.us/reader034/viewer/2022052700/55a183ad1a28abe05a8b45b0/html5/thumbnails/3.jpg)
THE ARCHIVE
![Page 4: R&D at Sound and Vision](https://reader034.vdocuments.us/reader034/viewer/2022052700/55a183ad1a28abe05a8b45b0/html5/thumbnails/4.jpg)
![Page 5: R&D at Sound and Vision](https://reader034.vdocuments.us/reader034/viewer/2022052700/55a183ad1a28abe05a8b45b0/html5/thumbnails/5.jpg)
Nederlands Instituut voor Beeld en Geluid
• Foto loopbrug / canyon
• Foto restaurant
• Detail foto’s
• Opening bea
![Page 6: R&D at Sound and Vision](https://reader034.vdocuments.us/reader034/viewer/2022052700/55a183ad1a28abe05a8b45b0/html5/thumbnails/6.jpg)
HISTORY BEELD EN GELUID
• Founded in 1996
• Archives from 4 organisations:
• Public Broadcasting Archive
• Netherlands Government Information Service
• Foundation for Film and Science
• Broadcaster Museum
![Page 7: R&D at Sound and Vision](https://reader034.vdocuments.us/reader034/viewer/2022052700/55a183ad1a28abe05a8b45b0/html5/thumbnails/7.jpg)
MISSION STATEMENT
• The best audiovisual archive in the digital domain
• 5 strategic priorities– enriching, connecting, and giving meaning to its
collections
– bringing people into contact with audiovisual heritage
– collecting material from the past, present, and future
– digital storage and access, with manual and automatic annotation
– actively contributing to durability and accessibility of the NL Audiovisual Collection
![Page 8: R&D at Sound and Vision](https://reader034.vdocuments.us/reader034/viewer/2022052700/55a183ad1a28abe05a8b45b0/html5/thumbnails/8.jpg)
USER GROUPS
• Education and research
• Creative industry
• Heritage institutes
• Consumers
maart 2012
![Page 9: R&D at Sound and Vision](https://reader034.vdocuments.us/reader034/viewer/2022052700/55a183ad1a28abe05a8b45b0/html5/thumbnails/9.jpg)
• 70% of Dutch AV heritage
• 800 000 hrs, including
• 250 000 hrs television
• 150 000 hrs radio
• 300 000 hrs music
• 50 000 hrs documentary, film, commercials etc
• 2 million pictures
• 20 000 objects
maart 2012
SIZE OF THE COLLECTION
![Page 10: R&D at Sound and Vision](https://reader034.vdocuments.us/reader034/viewer/2022052700/55a183ad1a28abe05a8b45b0/html5/thumbnails/10.jpg)
maart 2012
• film as of 1898
• commercials as of 1920
• polygoon (cinema journals)1922 – 1980
• radio as of 1934
• television as of 1951
• news and sports programs
• Royal house
• national music collection (cd)
• amateur film
• corporate films
EXAMPLE COLLECTION CONTENT
![Page 11: R&D at Sound and Vision](https://reader034.vdocuments.us/reader034/viewer/2022052700/55a183ad1a28abe05a8b45b0/html5/thumbnails/11.jpg)
RESEARCH &
DEVELOPMENT
![Page 12: R&D at Sound and Vision](https://reader034.vdocuments.us/reader034/viewer/2022052700/55a183ad1a28abe05a8b45b0/html5/thumbnails/12.jpg)
FOCUS AREAS
• Access– Annotation
– Retrieval
– Contextualization
• Exchange– Interoperability between collections
– Linking collection to semantic web
• Knowledge about end-users– Interviews
– Experiments
– Log studies
![Page 13: R&D at Sound and Vision](https://reader034.vdocuments.us/reader034/viewer/2022052700/55a183ad1a28abe05a8b45b0/html5/thumbnails/13.jpg)
FOCUS AREAS
• Access– Annotation
– Retrieval
– Contextualization
• Exchange– Interoperability between collections
– Linking collection to semantic web
• Knowledge about end-users– Interviews
– Experiments
– Log studies
TODAY
![Page 14: R&D at Sound and Vision](https://reader034.vdocuments.us/reader034/viewer/2022052700/55a183ad1a28abe05a8b45b0/html5/thumbnails/14.jpg)
ANNOTATION
![Page 15: R&D at Sound and Vision](https://reader034.vdocuments.us/reader034/viewer/2022052700/55a183ad1a28abe05a8b45b0/html5/thumbnails/15.jpg)
ANNOTATION TODAY
• INSERT PICTURE OF IMMIX
– +-15 archivists, with changing role
– Speech recognition and radio streaming in
![Page 16: R&D at Sound and Vision](https://reader034.vdocuments.us/reader034/viewer/2022052700/55a183ad1a28abe05a8b45b0/html5/thumbnails/16.jpg)
ANNOTATION RESEARCH
• Automatic annotation– Computer vision
– Audio analysis
• Help from the crowds– Crowd-sourcing
– Interactive games
• Secondary sources of descriptions– Subtitles
• Contextual material
![Page 17: R&D at Sound and Vision](https://reader034.vdocuments.us/reader034/viewer/2022052700/55a183ad1a28abe05a8b45b0/html5/thumbnails/17.jpg)
ANNOTATION RESEARCH
• Automatic annotation– Computer vision
– Audio analysis
• Help from the crowds– Crowd-sourcing
– Interactive games
• Secondary sources of descriptions– Subtitles
• Contextual material
![Page 18: R&D at Sound and Vision](https://reader034.vdocuments.us/reader034/viewer/2022052700/55a183ad1a28abe05a8b45b0/html5/thumbnails/18.jpg)
![Page 19: R&D at Sound and Vision](https://reader034.vdocuments.us/reader034/viewer/2022052700/55a183ad1a28abe05a8b45b0/html5/thumbnails/19.jpg)
RETRIEVAL AND
ACCESS
![Page 20: R&D at Sound and Vision](https://reader034.vdocuments.us/reader034/viewer/2022052700/55a183ad1a28abe05a8b45b0/html5/thumbnails/20.jpg)
RETRIEVAL REALITY
![Page 21: R&D at Sound and Vision](https://reader034.vdocuments.us/reader034/viewer/2022052700/55a183ad1a28abe05a8b45b0/html5/thumbnails/21.jpg)
RETRIEVAL RESEARCH
• Search with all sorts of annotations
– Small scale, diverse
– Hard to integrate with production
• New forms of access
– AVResearcher (MeRDES)
– AXES
– TROVE
– SocialZap
– LinkedTV
– …
![Page 23: R&D at Sound and Vision](https://reader034.vdocuments.us/reader034/viewer/2022052700/55a183ad1a28abe05a8b45b0/html5/thumbnails/23.jpg)
CONTEXT
![Page 24: R&D at Sound and Vision](https://reader034.vdocuments.us/reader034/viewer/2022052700/55a183ad1a28abe05a8b45b0/html5/thumbnails/24.jpg)
CONTEXT IS...
![Page 25: R&D at Sound and Vision](https://reader034.vdocuments.us/reader034/viewer/2022052700/55a183ad1a28abe05a8b45b0/html5/thumbnails/25.jpg)
CONTEXT IS....EVERYTHING?
![Page 26: R&D at Sound and Vision](https://reader034.vdocuments.us/reader034/viewer/2022052700/55a183ad1a28abe05a8b45b0/html5/thumbnails/26.jpg)
All information that can be used to interpret and understand
the production, publication and reception of audiovisual
heritage from multiple standpoints.
metadata ≠ context source
CONTEXTUALISATION:
AN AV-ARCHIVE DEFINITION
![Page 27: R&D at Sound and Vision](https://reader034.vdocuments.us/reader034/viewer/2022052700/55a183ad1a28abe05a8b45b0/html5/thumbnails/27.jpg)
5-4-2013
• What do you want to contextualise?
• What can be used as context?
• Which categories of context do you have?
• Which target users need which type(s) of context?
• How can you connect and represent context sources?
CONTEXTUALISATION:
IMPORTANT QUESTIONS
![Page 28: R&D at Sound and Vision](https://reader034.vdocuments.us/reader034/viewer/2022052700/55a183ad1a28abe05a8b45b0/html5/thumbnails/28.jpg)
FROM SEPARATE SILOS...
5-4-2013
![Page 29: R&D at Sound and Vision](https://reader034.vdocuments.us/reader034/viewer/2022052700/55a183ad1a28abe05a8b45b0/html5/thumbnails/29.jpg)
...TO CONTEXT: MUTUALLY
CONNECTED COLLECTIONS...
5-4-2013
Connecting collections:
topics, people, genres, etc
Ca
talo
gu
e
Ph
oto
s
B&
G W
iki
Pro
gra
mm
e
gu
ide
s
![Page 30: R&D at Sound and Vision](https://reader034.vdocuments.us/reader034/viewer/2022052700/55a183ad1a28abe05a8b45b0/html5/thumbnails/30.jpg)
...AND NEW POSSIBLITIIES FOR
INTERPRETING AV-HERITAGE
NL-HaNA, ANEFO / neg. stroken, 1945-1989, 2.24.01.05, bestandeelnummer 929.8771
CC BY-SA
![Page 31: R&D at Sound and Vision](https://reader034.vdocuments.us/reader034/viewer/2022052700/55a183ad1a28abe05a8b45b0/html5/thumbnails/31.jpg)
EXAMPLE OF WHAT DO WE WANT TO
DO AND PREVENT
![Page 32: R&D at Sound and Vision](https://reader034.vdocuments.us/reader034/viewer/2022052700/55a183ad1a28abe05a8b45b0/html5/thumbnails/32.jpg)
![Page 33: R&D at Sound and Vision](https://reader034.vdocuments.us/reader034/viewer/2022052700/55a183ad1a28abe05a8b45b0/html5/thumbnails/33.jpg)
![Page 34: R&D at Sound and Vision](https://reader034.vdocuments.us/reader034/viewer/2022052700/55a183ad1a28abe05a8b45b0/html5/thumbnails/34.jpg)
EXAMPLE: COMERDA
5-4-2013
• BRIDGE project: ISLA, Centre for Television in Transition (UU), S&V
• Goal: linking diverse, relevant collections to aid tv-researchers
![Page 35: R&D at Sound and Vision](https://reader034.vdocuments.us/reader034/viewer/2022052700/55a183ad1a28abe05a8b45b0/html5/thumbnails/35.jpg)
VIDEO HYPERLINKING
Method to link a video to other multimedia sources
Applications:
Search: connections between sources based on a query
(clustering, storytelling)
Detail-on-demand: zooming in on specific elements
through linked information sources (contextualisering)
12 februari 2013
![Page 36: R&D at Sound and Vision](https://reader034.vdocuments.us/reader034/viewer/2022052700/55a183ad1a28abe05a8b45b0/html5/thumbnails/36.jpg)
VIDEO HYPERLINKING:
AXES VS LINKEDTV
12 februari 2013
![Page 37: R&D at Sound and Vision](https://reader034.vdocuments.us/reader034/viewer/2022052700/55a183ad1a28abe05a8b45b0/html5/thumbnails/37.jpg)
12 februari 2013
VIDEO HYPERLINKING:
AXES VS LINKEDTV
![Page 38: R&D at Sound and Vision](https://reader034.vdocuments.us/reader034/viewer/2022052700/55a183ad1a28abe05a8b45b0/html5/thumbnails/38.jpg)
Concept: Jan Sluijters (Painter)
In DBpedia & VIAF
Related items
Further links
• Styles
(Expressionism, Cu
bism, Fauvism)
• Period
(contemporaries)
Concept relations:
VIDEO HYPERLINKING:
LINKEDTV ENRICHMENT PROCESS
![Page 39: R&D at Sound and Vision](https://reader034.vdocuments.us/reader034/viewer/2022052700/55a183ad1a28abe05a8b45b0/html5/thumbnails/39.jpg)
THANKS!
@lottebelice | [email protected]
Special thanks
• Marc Bron, Anders Bouwer, Maarten de Rijke (ILPS, UvA)
• Jasmijn van Gorp, Sonja de Leeuw (UU)
• Johan Oomen, Roeland Ordelman, Jaap Blom (B&G)
![Page 40: R&D at Sound and Vision](https://reader034.vdocuments.us/reader034/viewer/2022052700/55a183ad1a28abe05a8b45b0/html5/thumbnails/40.jpg)
WORKING TOGETHER
![Page 41: R&D at Sound and Vision](https://reader034.vdocuments.us/reader034/viewer/2022052700/55a183ad1a28abe05a8b45b0/html5/thumbnails/41.jpg)
ACCESS
• Sound and Vision Labs
– Place to put prototypes and
software
– Public/private access
– Data exchange
– Under construction
• We can put you in touch with
users and broadcasters
![Page 42: R&D at Sound and Vision](https://reader034.vdocuments.us/reader034/viewer/2022052700/55a183ad1a28abe05a8b45b0/html5/thumbnails/42.jpg)
DATA
• What do we have available?
– Catalog collection (complete)
– Keyframes
– MPG1
– Query logs (searches, clicks, purchases)
– Other collections
![Page 43: R&D at Sound and Vision](https://reader034.vdocuments.us/reader034/viewer/2022052700/55a183ad1a28abe05a8b45b0/html5/thumbnails/43.jpg)
EXAMPLE RESEARCH IDEAS
• Effective analysis of tweets about TV
• Large-scale video analysis
• Search over heterogeneous noisy metadata
• Online learning
• Incorporating thesaurus in search (suggestion)
• Binary concept detection (tagging)
![Page 44: R&D at Sound and Vision](https://reader034.vdocuments.us/reader034/viewer/2022052700/55a183ad1a28abe05a8b45b0/html5/thumbnails/44.jpg)
WANT TO KNOW MORE?
• Running research
• New ideas
• Data access
• Potential collaboration
• …?
Get in touch!
Find this presentation @ http://www.slideshare.net/boukehuurnink1
NL-HaNA, ANEFO / neg. stroken, 1945-
1989, 2.24.01.05, bestandeelnummer 933-2365 CC BY-SA