video information retrieval mark ruzomberka ist 497 11/07/02
TRANSCRIPT
![Page 1: Video Information Retrieval Mark Ruzomberka IST 497 11/07/02](https://reader035.vdocuments.us/reader035/viewer/2022062409/5697bfc21a28abf838ca5078/html5/thumbnails/1.jpg)
Video Information Retrieval
Mark Ruzomberka
IST 497
11/07/02
![Page 2: Video Information Retrieval Mark Ruzomberka IST 497 11/07/02](https://reader035.vdocuments.us/reader035/viewer/2022062409/5697bfc21a28abf838ca5078/html5/thumbnails/2.jpg)
Joke
![Page 3: Video Information Retrieval Mark Ruzomberka IST 497 11/07/02](https://reader035.vdocuments.us/reader035/viewer/2022062409/5697bfc21a28abf838ca5078/html5/thumbnails/3.jpg)
Outline
What is Video Information Retrieval (VIR) ?Reasons VIR is necessaryTheoreticalWhere we are todayExamplesProblemsFuture Work Conclusion
![Page 4: Video Information Retrieval Mark Ruzomberka IST 497 11/07/02](https://reader035.vdocuments.us/reader035/viewer/2022062409/5697bfc21a28abf838ca5078/html5/thumbnails/4.jpg)
What is Video Information Retrieval (VIR) ?
Recognition technologies Image Voice Text transcripts
Document retrieval technologies Topic segmentation Topic matching Text summarization
Presentation Technologies Combine Recognition and retrieval technologies
Result is an integrated application
![Page 5: Video Information Retrieval Mark Ruzomberka IST 497 11/07/02](https://reader035.vdocuments.us/reader035/viewer/2022062409/5697bfc21a28abf838ca5078/html5/thumbnails/5.jpg)
VIR-Need, or Why do I care?
Consider the task of trying to find a five minute video clip of interest in a library of 1000 hour long tapes.
Consider the “go to the part where” problem
![Page 6: Video Information Retrieval Mark Ruzomberka IST 497 11/07/02](https://reader035.vdocuments.us/reader035/viewer/2022062409/5697bfc21a28abf838ca5078/html5/thumbnails/6.jpg)
What do people want from IR
D-Lib Magazine’s asks:
“What do People want from Information Retrieval?”
# 8 Multimedia
![Page 7: Video Information Retrieval Mark Ruzomberka IST 497 11/07/02](https://reader035.vdocuments.us/reader035/viewer/2022062409/5697bfc21a28abf838ca5078/html5/thumbnails/7.jpg)
Specificly, Reasons for Video IR
Reading is slow compared to your potential for understanding information
Humans think in pictures not words Reading is particularly slow on a computer screen Example: Daydreaming while some one is talking Reading a page in a book and not remembering what it was about
![Page 8: Video Information Retrieval Mark Ruzomberka IST 497 11/07/02](https://reader035.vdocuments.us/reader035/viewer/2022062409/5697bfc21a28abf838ca5078/html5/thumbnails/8.jpg)
VIR makes for quicker human understanding.
Palm/Grafitti 25 Hand Writing 35-40 Typing 50-70 Speaking 135-175 Reading 200 Listening 400 - 500 Thinking 500+
•Video IR allows for faster access to information
![Page 9: Video Information Retrieval Mark Ruzomberka IST 497 11/07/02](https://reader035.vdocuments.us/reader035/viewer/2022062409/5697bfc21a28abf838ca5078/html5/thumbnails/9.jpg)
Theoretical:
Think of the “Jetsons mail system”
You “talk” to the computer, Computer intelligently “talks” back to you
![Page 10: Video Information Retrieval Mark Ruzomberka IST 497 11/07/02](https://reader035.vdocuments.us/reader035/viewer/2022062409/5697bfc21a28abf838ca5078/html5/thumbnails/10.jpg)
Where we are today
Two of Video Information Retrieval System are currently available:
Type One- keyword/text basedType Two- Content based
![Page 11: Video Information Retrieval Mark Ruzomberka IST 497 11/07/02](https://reader035.vdocuments.us/reader035/viewer/2022062409/5697bfc21a28abf838ca5078/html5/thumbnails/11.jpg)
Type One- keyword/text based
•DVR- basic expansion of image IR, •not as interesting
![Page 12: Video Information Retrieval Mark Ruzomberka IST 497 11/07/02](https://reader035.vdocuments.us/reader035/viewer/2022062409/5697bfc21a28abf838ca5078/html5/thumbnails/12.jpg)
Type Two- Content based
Video Mail Informedia MSR Video
Skimmer
![Page 13: Video Information Retrieval Mark Ruzomberka IST 497 11/07/02](https://reader035.vdocuments.us/reader035/viewer/2022062409/5697bfc21a28abf838ca5078/html5/thumbnails/13.jpg)
Example: Video Mail
University of Cambridge 1994-1996
AT&T 1999
2000-project ended
![Page 14: Video Information Retrieval Mark Ruzomberka IST 497 11/07/02](https://reader035.vdocuments.us/reader035/viewer/2022062409/5697bfc21a28abf838ca5078/html5/thumbnails/14.jpg)
Video Mail: Medusa network
Medusa multimedia environment at Olivetti Research Ltd. In Cambridge
It takes a modular approach unlike that of a pc or workstation Unified by a common interface to ATM network Devices plug directly into network and include:
Cameras Audio devices Networked frame buffers Processor farms Disk drives
![Page 15: Video Information Retrieval Mark Ruzomberka IST 497 11/07/02](https://reader035.vdocuments.us/reader035/viewer/2022062409/5697bfc21a28abf838ca5078/html5/thumbnails/15.jpg)
Video Mail: Medusa Network
“The network is the computer” metaphor is used Solves storage and network speed problems Complicates expense problem
![Page 16: Video Information Retrieval Mark Ruzomberka IST 497 11/07/02](https://reader035.vdocuments.us/reader035/viewer/2022062409/5697bfc21a28abf838ca5078/html5/thumbnails/16.jpg)
How it works-Overview
![Page 17: Video Information Retrieval Mark Ruzomberka IST 497 11/07/02](https://reader035.vdocuments.us/reader035/viewer/2022062409/5697bfc21a28abf838ca5078/html5/thumbnails/17.jpg)
The Integrated Application
“narrow” by sender,date, time
![Page 18: Video Information Retrieval Mark Ruzomberka IST 497 11/07/02](https://reader035.vdocuments.us/reader035/viewer/2022062409/5697bfc21a28abf838ca5078/html5/thumbnails/18.jpg)
Video Mail: Video Browser
Content is now being viewedKeywords are flagged
![Page 19: Video Information Retrieval Mark Ruzomberka IST 497 11/07/02](https://reader035.vdocuments.us/reader035/viewer/2022062409/5697bfc21a28abf838ca5078/html5/thumbnails/19.jpg)
Video Mail: Video Browser
In the latest version “thumb-nailed” pictures of key frames replace color coded line of the search keyword
![Page 20: Video Information Retrieval Mark Ruzomberka IST 497 11/07/02](https://reader035.vdocuments.us/reader035/viewer/2022062409/5697bfc21a28abf838ca5078/html5/thumbnails/20.jpg)
Informedia
The Informedia Digital Video Library Project automatically combines speech, image and natural language understanding to create a full-content searchable digital video library.
![Page 21: Video Information Retrieval Mark Ruzomberka IST 497 11/07/02](https://reader035.vdocuments.us/reader035/viewer/2022062409/5697bfc21a28abf838ca5078/html5/thumbnails/21.jpg)
Informedia
![Page 22: Video Information Retrieval Mark Ruzomberka IST 497 11/07/02](https://reader035.vdocuments.us/reader035/viewer/2022062409/5697bfc21a28abf838ca5078/html5/thumbnails/22.jpg)
Informedia: human factor issues
Interaction MotivationEffective usage modes
Commercial compressionVHS quality playback. Terabyte (1,000 gigabytes) of storage 1000 hours of video.
![Page 23: Video Information Retrieval Mark Ruzomberka IST 497 11/07/02](https://reader035.vdocuments.us/reader035/viewer/2022062409/5697bfc21a28abf838ca5078/html5/thumbnails/23.jpg)
Problems
1.Human understanding
2.Spoken document retrieval
3.Poor video browsers
4.Expensive
5.Slow access to data
6.Large amounts of data
![Page 24: Video Information Retrieval Mark Ruzomberka IST 497 11/07/02](https://reader035.vdocuments.us/reader035/viewer/2022062409/5697bfc21a28abf838ca5078/html5/thumbnails/24.jpg)
Microsoft Research (MSR) Video Skimmer
![Page 25: Video Information Retrieval Mark Ruzomberka IST 497 11/07/02](https://reader035.vdocuments.us/reader035/viewer/2022062409/5697bfc21a28abf838ca5078/html5/thumbnails/25.jpg)
Microsoft Research (MSR) Video Skimmer
Enhanced Browser Controls: Time Compression Pause Removal Textual Indices:
TOC, Notes
Visual IndicesShot Boundary FramesTimeline Markers
Jump Control (Back/Next)
![Page 26: Video Information Retrieval Mark Ruzomberka IST 497 11/07/02](https://reader035.vdocuments.us/reader035/viewer/2022062409/5697bfc21a28abf838ca5078/html5/thumbnails/26.jpg)
Problem: Poor Content Based Video Browsers
Current VCR model allows for poor navigation “go the the part where they say” problem
![Page 27: Video Information Retrieval Mark Ruzomberka IST 497 11/07/02](https://reader035.vdocuments.us/reader035/viewer/2022062409/5697bfc21a28abf838ca5078/html5/thumbnails/27.jpg)
Problem: Expensive
Hard drive space expensive Video adds to problem
High bandwidth needs are also expensive
Year Drive Size Drive Cost Per MB/Cost
1956 5 megabytes 50,000.00 10,000.00
1980 26 megabytes 5,000.00 193.00
1985 10 megabytes 710.00 71.00
1989 40 megabytes 1,199.00 36.00
1995 1.2 gigabytes 680.00 68.60
2000 30.0 gigabytes 249.99 0.96
•http://www.littletechshoppe.com/ns1625/winchest.html
![Page 28: Video Information Retrieval Mark Ruzomberka IST 497 11/07/02](https://reader035.vdocuments.us/reader035/viewer/2022062409/5697bfc21a28abf838ca5078/html5/thumbnails/28.jpg)
Problem: Slow Access to Data
Broadband still not available everywhereAvailability doesn’t mean acceptanceEspecially after dot com crash 2000
![Page 29: Video Information Retrieval Mark Ruzomberka IST 497 11/07/02](https://reader035.vdocuments.us/reader035/viewer/2022062409/5697bfc21a28abf838ca5078/html5/thumbnails/29.jpg)
Problem: Large Amounts of Data
Current Systems use MPEG2Newer compression technologies
MPEG 4-DIVX -DVD QualityVideo consumes orders of magnitude
more storage than textMPEG 7 is on horizon
![Page 30: Video Information Retrieval Mark Ruzomberka IST 497 11/07/02](https://reader035.vdocuments.us/reader035/viewer/2022062409/5697bfc21a28abf838ca5078/html5/thumbnails/30.jpg)
Future Work ?
Sky the limit ?Sci-Fi the limit ?
Hard Drive Space, Bandwidth are current limitations.
![Page 31: Video Information Retrieval Mark Ruzomberka IST 497 11/07/02](https://reader035.vdocuments.us/reader035/viewer/2022062409/5697bfc21a28abf838ca5078/html5/thumbnails/31.jpg)
Conclusion
Not yet ready for prime timeStorage and Network Costs decreasingSuccess is in day to day usageSlowly Becoming Mainstream E.x.TivoProblems of “real world tests”
Idiot proof ATM and Medusa aren’t mainstream
![Page 32: Video Information Retrieval Mark Ruzomberka IST 497 11/07/02](https://reader035.vdocuments.us/reader035/viewer/2022062409/5697bfc21a28abf838ca5078/html5/thumbnails/32.jpg)
Papers
Video Mail Retrieval Using Voice: Report on Keyword.. - Jones, Foote, Jones.. (1994) What do people want from Information Retrieval?. Croft, Bruce W. D-Lib Magazine. (1995) Video Skimming for Quick Browsing based on Audio and Image.. - Smith, Kanade (1995) The VISION digital video library (context) - Gauch, Li et al. – (1997) Informedia: News-on-Demand Multimedia Information.. - Hauptmann, Witbrock (1997) M.G. Christel and D.J. Martin, "Information Visualization within a Digital Video Library", J.
Intelligent Info. Systems 11(3), (1998), pp. 235-257 Browsing Digital Video. Li, Gupta, Sanocki et. Al.
![Page 33: Video Information Retrieval Mark Ruzomberka IST 497 11/07/02](https://reader035.vdocuments.us/reader035/viewer/2022062409/5697bfc21a28abf838ca5078/html5/thumbnails/33.jpg)
Questions?
![Page 34: Video Information Retrieval Mark Ruzomberka IST 497 11/07/02](https://reader035.vdocuments.us/reader035/viewer/2022062409/5697bfc21a28abf838ca5078/html5/thumbnails/34.jpg)
Joke?
"There are 10 types of people in the world...
those who understand binary and those who don't."