bit 3193 multimedia database chapter 1 : introduction to multimedia database
Post on 26-Dec-2015
257 Views
Preview:
TRANSCRIPT
BIT 3193 MULTIMEDIA DATABASE
CHAPTER 1 : INTRODUCTION TO MULTIMEDIA
DATABASE
What is Multimedia Database ?
The ability to manage, store and retrieve the different types of media
(multimedia data) (Lynne Dunckley, 2003)
What is Multimedia Database ?
• support multimedia data types in addition to providing facilities for traditional DBMS functions :– database creation, – data modeling, – data retrieval, data access and organization, and – data independence.
(Kosch, H. and Döller, M., 2005)
Multimedia Data
Static
Dynamic
•do not have a time dimension•their contents and meanings do not depend on the presentation time• example : graphics/still images, alphanumeric data
• have time dimensions
• their meanings and correctness depend on the rate at which they are presented
• example : animation, video and audio
TEXT
consists of plain alphanumeric character : ASCII
Storage space requirement in bytes, being equal to the numbers of characters (including spaces) in the document
Example :A typical book with 300 pages, each of has 3,000 characters
Storage = 900 Kb
GRAPHICS
• graphics elements are represented in mathematical formulas• Example : rectangle element identifier of rectangles + coordinates of 2 opposite corners are stored –> to change, only modify these parameters• storage requirements are very low
• the graphics is divided into small picture elements called pixels• each pixel corresponds to a dot on the screen• the intensity @ color of pixels are stored in a pixel-based graphics file
Vector-based
Pixel-based
ANIMATION
Produced by sequential rendering a number of frames of graphics
If the graphics are pixel-based, the animation is the same as video
If the graphics are vector-based, indexingand retrieval can be carried out in similarway for vector graphics but have extra temporal dimension
AUDIO
Caused by a disturbance in air pressurethat reaches the human eardrum
Parameter used is amplitude (dB)
A sound wave is continuous in both time and amplitude
Amplitude
Time
Figure A : Example sound wave
Sound wave is a analog signal
For computers to process and communicate an audio signal, it must beconverted into a digital signal : ADC
ADC involved 3 stages:• sampling, • quantization• coding
AUDIO
Consists of a number of frames or imagesthat have to be played at a fixed rate
Parameter used is fps (frames per second)
2 major characteristics of video:• has a time dimension • takes a huge amount of data to represent
VIDEO
2 common frame rates :• 25 fps : PAL systems• 30 fps : NTSC systems
Example : 10-minutes video withimage size 512 pixels by512 lines, pixel depth of24 bits/pixel and frame rate 30fps
Storage :600*30*512*512*3=13.8Gb
There are three challenges that arise from multimedia data that do not occur with other data types.
Size
Semantic NatureTime
• To get an idea of the size of media data objects :
• single colored image could require 6 Mb
• video object (30 fps, 5 minutes video clip) would require 54 Gb.
• audio will occupy 8 Kb for each second
• Data size will affect the storage, retrieval and transmission of multimedia.
• Therefore techniques that reduce the size of multimedia data without impacting on the information are crucial.
• Video and audio must run in the correct sequence and at an acceptable rate otherwise it becomes meaningless.
• have significance for the way the media objects are stored, retrieved, transmitted and synchronized together.
• example : video clip of an interview audio + image data (must synchronized together).
• more complex than traditional data types.
• difficult to identify components within the media that could be used for retrieval or transaction processing.
• interpretations may need based on certain features of multimedia data and stored as METADATA.
• any data
• that is required to interpret other data
• as meaningful information
• it is used for retrieving and manipulating the data
Multimedia Database Applications
Entertainment System Video on Demand
Medical Information System
Public Protection
The registered user of the system can request a video from the catalog. The videos may be available according to a previously advertised fixed schedule or available at any time, subject to a small delay. The user can select a video based on textual information of the cast, production team and synopsis of the plot. Production information such as storyboards, screenplay and production notes can be included. Users can view the video contiguously or play randomly selected scenes. The video can be paused and resumed play as requested within constraint.
• This is a single media application.
• The user is not involved in capturing, editing or manipulating the media.
• Communication of the media is unidirectional.
• Delivery may be simplified by scheduling requests and combining the delivery to several users at the same time.
• There is high data volume that requires high performance storage and networking system.
• There is a large number of users may accept some loss of quality.
Exercise
Identify the system requirements of the
following multimedia applications case
study.
In a number of countries police use visual information to identify people or to record the scenes of crime for evidence. These photographic records are a valuable archive. In the UK everyone arrested is photographed and their images are sampled and stored with their fingerprints. It is also planned to store sampled DNA profiles of suspects. Until a subject is convicted, access to photographic information is restricted. Interrogation of the database may be on the basis of automatic fingerprint recognition, DNA matching and face recognition. Video surveillance also needs to be linked to the facial recognition system.
• The user is involved in capturing, editing, or manipulating the media.
• bi-directional data flow
• complex content modeling for complex correlated queries.
• diverse media – maps, images, audio, video.
• interactivity with media through simple matching queries.
The medical and related health professionals use and store visual information in the form of x-ray, ultrasound and other scanned images for diagnosis and monitoring purposes. There are strict rules on confidentiality of such information. The images are kept with patients’ records stored by unique identifier (e.g national insurance number). Visual information, provided that it is rendered anonymous, may also be used for research purposes. Effective image processing such as edge detection and feature extraction can be important in assisting expert diagnosis of lesions, tumors and tracking their growth. Images may be the result of a single instrumental approach, e.g x-ray, or the result of a combination of data from several resources.
• The user is involved in capturing, editing, or manipulating the media.
• highest quality media data with little toleration of data loss.
•Confidential security required.
•bi-directional data flow
• complex content modeling for complex correlated queries.
• diverse media – maps, images, audio, video.
• The physical storage describes how multimedia objects are stored in a file system.
• Multimedia objects are typically huge need different techniques for their storage and retrieval.
• This view deals with the issue of:
• providing fast access to stored data index mechanisms
• can be stored in different systems
• can be accessed over computer networks
• can query multimedia data in different ways depending on the type of information
• provide a filtered view of the mm databases to retrieve only the required data
• the object retrieved have to be appropriately presented
Application Interface Application Interface Application Interface
W1
W2Wn
Query 1
COMMUNICATION NETWORK
Data Models(OOP Metadata)
Text
W1
W2Wn
W1
W2Wn
Query 3Query 2
Data Access(Indexing)
Temporal Models(Petri Nets)
Image Video Audio
User’s view
Filtered view
Distributed view
Conceptual DataView
PhysicalStorage
View
Figure B : Components Involved in MMDBMS
PHYSICALSTORAGE
VIEWThe main issue is size.
Size of objects influences:• storage capacity requirements• retrieval bandwidth (bps) requirements
PHYSICALSTORAGE
VIEW
Deal with raw digitized data
Media Representation Data Size Disk Bandwidth
Text ASCII 200 KB / 100 Pages
Presentation Dependent
Image GIF, TIFF, JPEG
3.2MB/image0.4 MB/image
-do-
Video UncompressedHDTVMPEG
20 MB/sec110 MB/sec0.2-1.5 Mbits/sec
20 MB/sec110 MB/sec0.2-1.5 Mbits/sec
Audio UncompressedCD Quality
64 Kbits/sec1.4 Mbits/sec
64 Kbits/sec1.4 Mbits/sec
Table 1 : Media Types, Representation, Size and Bandwidth Requirements
This table describes the size and the retrieval disk bandwidth requirements for differentmedia, based on their format of presentation
Disk Bandwidth Requirements
Static Media (text and images)• depends on multimedia database application• because do not have any inherent temporal requirements
Dynamic Media (Audio and Video)• have inherent temporal requirements • proportional to their temporal requirements• can be accessed by multiple users simultaneously demand new capabilities from the file system and the OS
PHYSICALSTORAGE
VIEW
File System Requirements
Should have the following capabilities:• handling huge files• supporting simultaneously access to multiple files by multiple users• supporting the required disk bandwidth• can provide new application programming interfaces (play, fast forward, reverse for dynamic media)
PHYSICALSTORAGE
VIEW
Operating System Requirements
Should have the capabilities for handling real time characteristics for: scheduling of application process
• OS reserve the resources required for an application process
• depending on the availability of resources application process may or may not be admitted for execution
• communication between application process and the OS kernel
• reduced overhead can affect the performance of applications
PHYSICALSTORAGE
VIEW
These objects are acquired (from devices)and created (digitized, compressed and stored) independent of its contents
For using these objects as meaningful data, one needs to identify their content
CONCEPTUALDATA VIEW
Objects are in binary form
It dependent of the:• media type• the role of an application
Example : video clip of a movie• the sequence of frames contains actors, actresses, the background, action going on, etc.
CONCEPTUALDATA VIEW
The description of the object’s content,called metadata is subjective
1 302013
A2 : Criminal Takes Out GunA3 : Criminal Points Gun at Actress
A4 : Hero Shoot Criminal
A1 : Hero Fights Criminal
Frames
Figure C : Example Description of a Video Clip
These abstractions form a data modelfor a particular application domain
For fast accesses, indexing mechanism is needed to sort the data according to thefeatures that are modeled.
CONCEPTUALDATA VIEW
The conceptual data view of a raw datahelps in building a set of abstraction
Presentation to the user has to be properly synchronized
These synchronization characteristics aredescribed by temporal models
CONCEPTUALDATA VIEW
MM database may be composed ofmultiple media objects
• metadata • indexing mechanism• temporal models• spatial models• data models
CONCEPTUALDATA VIEW
Hence the conceptual data view ofmultimedia data consists of the followingcomponents:
Metadata
• Deals with the:• content• structures• semantics of media objects
• The creation of metadata depends on:• media type• type of information
• Available techniques for automatic (or semi-auto) generation of metadata is important
CONCEPTUALDATA VIEW
Metadata
• Example :
• Video media:• The techniques should identify camera shots, characters in a shot, background of a shot, etc
• Image/text/audio data:• The techniques should extract and describe the features of interest – recognition techniques
CONCEPTUALDATA VIEW
Indexing Mechanisms
• To provide fast access
• should be able to handle different features of objects such as color or texture
CONCEPTUALDATA VIEW
Temporal Models
• Describes the time and duration of presentation of each media objects • Example :
Video object v1 has to be presented at time t1for a duration t3-t1 and has to be synchronizedWith the presentation of audio object a1
CONCEPTUALDATA VIEW
v1 v2
a1 a2
Video
Audio
timet1 t2 t3 t5 t6
Spatial Models
• Represent the way media objects are presented• by specifying the layout of windows on a monitor• Example:CONCEPTUAL
DATA VIEW Text
Window
ImageWindow
VideoWindow
TextStream
ImageStream
VideoStream
AudioStream
Speaker
Data Models
• Object oriented approach is normally used
• to represent:• the characteristics of objects• metadata associated with them• their temporal• spatial requirement
CONCEPTUALDATA VIEW
Huge sizes of media objects require largebandwidth or throughput (bps)
Real time nature of objects need guarantees on:• end-to-end delay• delay jitter
DISTRIBUTED VIEW
Multimedia data can be distributedover computer networks
End-to-end delay
•specifies maximum that can be suffered (delay) by data during communicationDISTRIBUTED
VIEW
Delay jitter
• the variations in the end-to-end delay suffered by the dataDISTRIBUTED
VIEW
Existing communication protocols addressdo not have any real time requirements
Hence, based on the temporal relation-ships, the buffer required and the availablenetwork bandwidth, the client needs toidentify a retrieval schedule for requestingobjects from the server.
DISTRIBUTED VIEW
Guarantees are required for smoothpresentation of continuous media
User’s query can be of the following types:• query on the content of media objects• query by example• time indexed queries• spatial queries• application specific queries
FILTEREDVIEW
Is provided by user’s query to getrequirement information
Content Based Query
• Considering VOD server application, user can make queries by example such as:
• get me the movie in which this scene (an image) appears• get the movie where this video clip occurs• show me the movie which contains this song
FILTERED VIEW
Content Based Query
•This refers to the multimedia object that is used as an example.• The MMDBMS has to process the example data (this object) and find one that matches• The requirement for similarity (between the queried object and database object) can be on different characteristics:
• exact• partial
FILTERED VIEW
Content Based Query
• Partial matching:•We need to know the degree of mismatch that can be allowed between the example objects and the one in the database
FILTERED VIEW
Time Indexed Queries
• For continuous media, users can give queries in the temporal dimensions.
• Example:• Show me the movie 30 minutes after its start
FILTERED VIEW
Spatial Queries
• Media objects such as images and video have spatial characteristics
• Example: Show me the image where Mawi is seen on the left of Siti Nurhaliza
FILTERED VIEW
Application Specific Queries
• Multimedia database are highly application specific
• Example: medical or GIS Show me the video where the tissue evolves into a cancerous one
FILTERED VIEW
USER’S VIEW
User’s view of a MMDBMS is characterizedby the following requirements:
• user query interface• presentation of multimedia data• user interaction during presentation
User Query Interface
• Query interface should allow users to query by content, example, time, space or a combination of these possibilities.
• For queries by example, the user query interface has to obtain the example objects from appropriate devices e.g image from scanner
• In case of partial matching of the resolved queries, the query interface can suggest ways to modify the query to get exact matches.
USER’S VIEW
Presentation of Multimedia Data
• Media object can be on different format. Example: image tiff, gif• In some cases, might be necessity to convert data from one format to another format before presentation.• The presentation of multimedia object may have:
• temporal constraints• spatial constraints
• The constraints describe the layout of windows on the user’s screen
USER’S VIEW
User Interaction During Presentation
• User can interact during the presentation of multimedia objects.• The interaction is complex since multiple media objects are involved• Example:
• Devices such as microphone and video camera can be used for speech and gesture recognition
• Hence, simultaneous control of different devices and handling user input is required.
USER’S VIEW
User Interaction During Presentation
• The input from the user can be of the following types:
• modify the quality of the presentation, reduction or magnification of the image• direct the presentation, skip, restart etc
USER’S VIEW
top related