multimedia search and retrieval: new concepts, system implementation, and application qian huang,...
Post on 19-Dec-2015
216 Views
Preview:
TRANSCRIPT
Multimedia Search and Retrieval: New Concepts, System Implementation, and Application
Qian Huang, Atul Puri, Zhu Liu
IEEE TRANSACTION ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, AUGUEST 2000
Outline Introduction Media Types and Their Properties Multimedia Data Abstraction and Retrieval System Architecture System Implementation An Application: Multimedia Broadcast
News Summary
Introduction MPEG-7 focuses on the standardization of a
common interface for describing multimedia materials,supporting pull and push information.
Our proposed multimedia information retrieval system cover system architecture, data storage and representation, database and its remote access, and human-machine interfaces.
The concept of physical and logical layers and events.
Media Types and Their Properties Audio
Audio events can be defined as continuous audio segments that have some distinct acoustic characteristics
Visual Text
Multimedia Data Abstraction and Retrieval
A physical layer addresses the media events that are time continuous and better described using the specific characteristics of a media signal.
A logical layer addresses multimedia events that present integrated behavior of other events across media.
Multimedia Data Abstraction and Retrieval(cont.) Physical Program and Physical Event
Physical program a continuous stream of multimedia data that
begins and ends due to the physical scene edits and cuts decided by human operators
Physical event A continuous segment of multimedia data,
representing a coherent and meaningful event within a physical program
Multimedia Data Abstraction and Retrieval(cont.)
Logical Event A virtual view of the multimedia data constr
ucted from a set of physical programs or events, or recursively from other physical or logical events.
Application dependent Semantic Content Structure
Conceptual and virtual Provide a seamless interface to user Table of Content (ToC)
Relationship among different event layers
System Architecture
System Implementation Server Implementation
Content Hierarchy Extraction Layered Content Storage
Store lower level information in a file system and higher level information in standard database systems
When raw data is stored in a file system, different events in the data stream can be accessed using time based indexing
Database Content Access Employ JDBC to provide connectivity to our
database for query and update
Multimedia data storage in combined file system and database
Example of event indexing from database to a file
Database access
System Implementation(cont.)
Client Browser Implementation Java applet-based interface
Platform independence support for a variety of coded media
Remote Database Query RMI (Remote Method Invocation)
Browsing Interface Table of Content: a conceptual content
structure TimeLine: a content structure in time Event Presentation: presentation and playback
Content Playback with a JMF Player
Remote database access
Browser interface for a news program with both ToC and timeline representations of the semantic structure within the program
An Application: Multimedia Broadcast News Server End
Content Hierarchy Extraction Broadcast news data is well structured. Lower levels – extract primitive physical
events in each individual media stream. Higher semantic level – merge different
events into semantically meaningful logical events. ex: Headline stories and news summaries.
Content hierarchy of broadcast news programs
Content Storage and Access In the file system, for each physical
broadcast news program, store the timetables for audio, video,and text,individually.
The tables in the database system will index into these timetables for the actual retrieval and data access.
Physical events in each media are recorded in separate database tables at the physical event layer.
Relationship between logical event and physical event objects
Database design in broadcast news application
Client End Remote Database Query
The physical program ID is uniquely determined by the channel and the date of the program.
Content Presentation Content Structure Presentation
ToC and TimeLine Static(single click) and playback(double click)
Story Presentation News Summary Presentation
Streaming Playback
Browser interface for a news program with both ToC and timeline representations of the semantic structure within the program
Visual representation for story “El Nino”
Representation for news summary of the day
Playback interface with synchronized audio/video/text
Summary We have introduced a new layered
approach to multimedia content representation and storage for search and retrieval.
The concepts are implemented in a client/server multimedia information retrieval system designed for broadcast news search and retrieval.
top related