accessing large av collections using visual analysis in digital humanities

ACCESSING LARGEAUDIOVISUAL COLLECTIONS

USING VISUAL ANALYSISAV IN DH WORKSHOP @ DH2014 LAUSANNE

ROELAND ORDELMAN

NETHERLANDS INSTITUTE FOR SOUND AND VISION

BUSINESS ARCHIVE DUTCH PUBLIC BROADCASTERS

LARGE DIGITIZATION PROGRAMS

CLARIAH PRESENTATIE 11 September 2013

6

+800.000 hours of audiovisual content

‘POTENTIAL’

Find what you were (not) looking for

Browse video to find what you were looking for

X

We need labels!

Labels connect(content, context)

Labeling

CLARIAH PRESENTATIE 11 September 2013

13

BIG DATA!

CLARIAH PRESENTATIE 11 September 2013

14

INNOVATIVE PLATFORMS

We need USEFUL labels

16

USEFUL?

Developer/ICT researcher

DH Researcher

17

FEEDBACK

Research & Education

Broadcast Professionals

Hergebruik

Media Archivists (documentalisten)

Beschrijven

Journalists Research

Academic researchers Investigate

Education Illustrate

19

Use Scenarios&

System Requirements

Interview & elicitation sessions

Mock-up creation & evaluation

Prototype evaluation

System evaluation

Surveys & log analysis

Qualitative

Qualitative

QualitativeQuantitative

Quantitative/Qualitative

BUILDING PROTOTYPES

2012

2013

-PRO

-RES

Onderzoekers

Media Professionals

<nisv@axes> ls –lTotal 10-r--r--r--. 1 nisv axes 301 Jun 26 2011 METADATA-r--r--r--. 1 nisv axes 301 Jun 26 2011 SUBTITLES-r--r--r--. 1 nisv axes 301 Jun 26 2011 SPEECH RECOGNITION-r--r--r--. 1 nisv axes 301 Jun 26 2011 FACE RECOGNITION-r--r--r--. 1 nisv axes 301 Jun 26 2011 VISUAL CONCEPT DETECT-r--r--r--. 1 nisv axes 301 Jun 26 2011 EVENT DETECTION-r--r--r--. 1 nisv axes 301 Jun 26 2011 LOCATION DETECTION-r--r--r--. 1 nisv axes 301 Jun 26 2011 QUERY BY EXAMPLE-r--r--r--. 1 nisv axes 301 Jun 26 2011 SEARCH-r--r--r--. 1 nisv axes 301 Jun 26 2011 RECOMMENDATION-r--r--r--. 1 nisv axes 301 Jun 26 2011 USER INTERFACE<nisv@axes> |

Face Recognition

Query by example

DETECTION REQUIRES TRAINING(EXAMPLES)

2nd EC review meeting – Hilversum – Mar 19th 2013

2nd EC review meeting – Hilversum – Mar 19th 2013

EXPECTATION MANAGEMENT

2nd EC review meeting – Hilversum – Mar 19th 2013

Expectation Management

• Expectation management:– Training examples versus result list– Google images search versus visual search in AV

• Understanding visual search:– why something is hard to detect

• visual characteristics, training examples

– Noise is not bad per definition

DH perspective

• First explorations in various projects– Requirements studies– Demonstrations– Prototypes

• Technology is ready to start exploring its use in real use scenarios (e.g., query by example)

• Feed DH ideas into ICT research community

Technology exists that could helpTechnology does not solve all

problemsDiscuss with ICT experts

Technology has a price, what is the RoI?

AWARENESS

How does technology fit inHow do limitations fit in

‘Technology Critique’ (Historian 2.0?)ICT and curriculum

METHODOLOGY/TRAINING

What can it do?How does it work?

How does it perform?How can it be improved?

MICRO MACRO

How can we use it?What do we need?How does it scale?

Who could benefit as well?

www.axes-project.euroelandordelman.nl

Questions?

<nisv@axes>

<nisv@axes>