exploring the news | always multi- source, multimodal and personalized

Exploring the news | Always multi-source, multimodal and personalized

News Rover

The News Today

The News Today

The News Today

The News Today

The News Today

The News Today

Trends and StatisticsNews consumption is moving online

Advantages:• Access on any device• Available any time, any

place• Find related content

Where did you get your news yesterday?

Source: PEW RESEARCH CENTER 2012 News Consumption Survey

sportspoliticstechsports economicspolitics

Heterogeneous Large-scale Often Isolated

NewsRover Live Recording System

100 TV channels recorded continuously

Linked to Online News and Twitter topics

Per day:• 110 hours video recorded

• 1,380 video stories indexed

• 460 Google News topics crawled

• 550 Twitter topics crawled

• Total size:– 28,000 hours video

recorded

– 464,000 video stories

– 24,500 News topics

– 80,000 Twitter topics

Navigating the News with StructureW

hen

?W

ho

?W

hat

?W

her

e?

‘Same-sex Marriage’ ‘France/Mali’ conflict

‘Debt Limit’

Topical Organization

‘Cong. Budget Office’ ‘Consumer Debt’

Coverage Trendline

Event Geo-visualization

Name Extraction & Normalization

Multimodal Topic Extraction & Linking

Who Said What? in VideoFeatu

re

Mod

es

Audiotrack:

Segmentation:

Speech Segmentation

Visual Speaker Detection

Speaker Gender Classification

Face Tracking & Clustering

Segmented Audiotrack:

Text

Vis

ua

lA

ud

io

Anchor Detection

Name Extraction from Aligned CC-ASR

Name Extraction from OCR

Name Gender Classification

Speaker Diarization

Ed

it D

ista

nce

A

lign

men

t

Male GMM

Female GMM

MFC

C

Cla

ssifi

er

Gen

der

New york times -> search

Shih-Fu Chang, 11/2011

Read, Tag and Search