Download - MMBD - Multimedia Databases
![Page 1: MMBD - Multimedia Databases](https://reader033.vdocuments.us/reader033/viewer/2022052900/555e9fb7d8b42a6d068b54c9/html5/thumbnails/1.jpg)
MULTIMEDIA DATABASESAND MPEG7
Rahmi Volkan Başar
Department of Computer Engineering
METU
May, 2013
![Page 2: MMBD - Multimedia Databases](https://reader033.vdocuments.us/reader033/viewer/2022052900/555e9fb7d8b42a6d068b54c9/html5/thumbnails/2.jpg)
Multimedia Databases
• Introduction
• Capabilities of DB Types
• Search on MMDB
• Multimedia Content Description
• Research Fields
![Page 3: MMBD - Multimedia Databases](https://reader033.vdocuments.us/reader033/viewer/2022052900/555e9fb7d8b42a6d068b54c9/html5/thumbnails/3.jpg)
Multimedia Data
• Text: using a standard language (SGML, HTML)
• Graphics: encoded in CGM, postscript
• Images: bitmap, JPEG, MPEG
• Video: sequenced image data at specified rates
• Audio: recordings in a string of bits in digitized form
![Page 4: MMBD - Multimedia Databases](https://reader033.vdocuments.us/reader033/viewer/2022052900/555e9fb7d8b42a6d068b54c9/html5/thumbnails/4.jpg)
Database vs Multimedia Database
• Databases– well structured data organization
– efficient storage of large amounts of data
– querying
– transactional support for concurrent users
– numbers, strings
• Multimedia Databases– large content
– different structures
– not easily searched/queried
![Page 5: MMBD - Multimedia Databases](https://reader033.vdocuments.us/reader033/viewer/2022052900/555e9fb7d8b42a6d068b54c9/html5/thumbnails/5.jpg)
Use Cases
• Repositories: central location for data maintained by DBMS, organized in storage levels
• Presentations: delivery of audio and video data, temporarily stored, ‘VCR-like functionality’
• Collaborative: complex design, analyzing data
![Page 6: MMBD - Multimedia Databases](https://reader033.vdocuments.us/reader033/viewer/2022052900/555e9fb7d8b42a6d068b54c9/html5/thumbnails/6.jpg)
Capabilities
• Relational Databases
– Atomic / Tables
– Data relation – Common Foreign Keys
– Record: Content – No meta information
– A predefined set of domains for columns
• Hard to extend
• BLOB data type exist
![Page 7: MMBD - Multimedia Databases](https://reader033.vdocuments.us/reader033/viewer/2022052900/555e9fb7d8b42a6d068b54c9/html5/thumbnails/7.jpg)
Capabilities
• Object Oriented Databases
– Schema is “Class”
– All data is “Object”
– References
– New data types
• Easy. New class is a new data type.– Appropriate for multimedia data
![Page 8: MMBD - Multimedia Databases](https://reader033.vdocuments.us/reader033/viewer/2022052900/555e9fb7d8b42a6d068b54c9/html5/thumbnails/8.jpg)
Capabilities
• Object Relational Databases
– In addition to RDBMS
• Object references
• New types– Multimedia
– MMDBMS
• Extensible ORDBMSs
![Page 9: MMBD - Multimedia Databases](https://reader033.vdocuments.us/reader033/viewer/2022052900/555e9fb7d8b42a6d068b54c9/html5/thumbnails/9.jpg)
Search
• Collection of data. How to search?
– Any standards?
– Workarounds?
• Search: Retrieve similar images…
– Fast, Correct
• Content-based
– New techniques?
![Page 10: MMBD - Multimedia Databases](https://reader033.vdocuments.us/reader033/viewer/2022052900/555e9fb7d8b42a6d068b54c9/html5/thumbnails/10.jpg)
Search
• Content Based Retrieval Facilities
– Supported by MMDBMS
• Organize and Manage accordingly
– Compare based on a number of features
• Shape/Color/Texture
• Meta-Data?
– Always.
![Page 11: MMBD - Multimedia Databases](https://reader033.vdocuments.us/reader033/viewer/2022052900/555e9fb7d8b42a6d068b54c9/html5/thumbnails/11.jpg)
Content Based Retrieval
• Accurate representation of the multimedia objects in the database
– For accuracy and efficiency
– Combination: Different features
• Similarity Search
– High-dimensional feature vectors
• Special multi-dimensional indexing structures
• Dimension reduction methods.
![Page 12: MMBD - Multimedia Databases](https://reader033.vdocuments.us/reader033/viewer/2022052900/555e9fb7d8b42a6d068b54c9/html5/thumbnails/12.jpg)
Multimedia Content Description Standard: MPEG-7
• Influential XML based multimedia meta-data standard• Description of the storage media:
– Format, Image Size, Audio Quality, Video Frames etc.
• Creation and production information:– Creation date and location, title, genre, etc.
• Content semantic description:– Events, concepts, objects, etc.
• Content structural description:– Shot and key frames with color, texture and motion
features, etc.
• Metadata about the description:– Author, version, creation date, etc.
![Page 13: MMBD - Multimedia Databases](https://reader033.vdocuments.us/reader033/viewer/2022052900/555e9fb7d8b42a6d068b54c9/html5/thumbnails/13.jpg)
MPEG-7
• Expression of multimedia data
• Missing: Search for Implicit Data
– The meaning of the structure: Not expressed
– Ex. A video: length, format, name, dates etc.
• Gender: Documentary, Interview, Movie
• Theme: Science, Sports, Horror
• No consideration on search engines
![Page 14: MMBD - Multimedia Databases](https://reader033.vdocuments.us/reader033/viewer/2022052900/555e9fb7d8b42a6d068b54c9/html5/thumbnails/14.jpg)
MPEG-7
• Search:
– XPath, XQuery
– Semantic Views Query Language
![Page 15: MMBD - Multimedia Databases](https://reader033.vdocuments.us/reader033/viewer/2022052900/555e9fb7d8b42a6d068b54c9/html5/thumbnails/15.jpg)
Simple MPEG7 Example<Mpeg7>
<Description xsi:type="SemanticDescriptionType">
<Semantics>
<Label>
<Name> Car </Name>
</Label>
<Definition>
<FreeTextAnnotation>
Four wheel motorized vehicle
</FreeTextAnnotation>
</Definition>
<MediaOccurrence>
<MediaLocator>
<MediaUri> image.jpg </MediaUri>
</MediaLocator>
</MediaOccurrence>
</Semantics>
</Description>
</Mpeg7>
![Page 16: MMBD - Multimedia Databases](https://reader033.vdocuments.us/reader033/viewer/2022052900/555e9fb7d8b42a6d068b54c9/html5/thumbnails/16.jpg)
MPEG7 Details
• Standardizes 3 parts:
– Description tools
• Descriptors (D)
• Description Schemes (DS).
– Description Definition Language (DDL)
• To specify these schemes
– System tools
![Page 17: MMBD - Multimedia Databases](https://reader033.vdocuments.us/reader033/viewer/2022052900/555e9fb7d8b42a6d068b54c9/html5/thumbnails/17.jpg)
MPEG7 Details
• Descriptors (D)– Representation of a feature
• Syntactic and Semantic
– Low-level audio or visual features • Color, motion, texture etc
– Audiovisual content • Location, time etc
• Objects can be described– Several descriptors.
![Page 18: MMBD - Multimedia Databases](https://reader033.vdocuments.us/reader033/viewer/2022052900/555e9fb7d8b42a6d068b54c9/html5/thumbnails/18.jpg)
MPEG7 Details
• Description Schemes (DS) describe
– Specification of the relations
• Between Descriptors
• Between Description Schemes
– Relations can be structural and semantics
– High-level audiovisual (AV) features
• Regions, segments, events etc
![Page 19: MMBD - Multimedia Databases](https://reader033.vdocuments.us/reader033/viewer/2022052900/555e9fb7d8b42a6d068b54c9/html5/thumbnails/19.jpg)
MPEG7 Details
• Description Definition Language
– Based on XML
• Defines the structural relations between descriptors– Creation and modification of description schemes
– Creation of new descriptors.
![Page 20: MMBD - Multimedia Databases](https://reader033.vdocuments.us/reader033/viewer/2022052900/555e9fb7d8b42a6d068b54c9/html5/thumbnails/20.jpg)
MPEG7 Details
• System Tools
– Deal with Descriptor management
• Binarization
• Synchronization
• Transport
• Storage
![Page 21: MMBD - Multimedia Databases](https://reader033.vdocuments.us/reader033/viewer/2022052900/555e9fb7d8b42a6d068b54c9/html5/thumbnails/21.jpg)
MPEG7 Details - Overview
![Page 22: MMBD - Multimedia Databases](https://reader033.vdocuments.us/reader033/viewer/2022052900/555e9fb7d8b42a6d068b54c9/html5/thumbnails/22.jpg)
MPEG7 Details
• Next Slide
– Description of a Video Segment
![Page 23: MMBD - Multimedia Databases](https://reader033.vdocuments.us/reader033/viewer/2022052900/555e9fb7d8b42a6d068b54c9/html5/thumbnails/23.jpg)
![Page 24: MMBD - Multimedia Databases](https://reader033.vdocuments.us/reader033/viewer/2022052900/555e9fb7d8b42a6d068b54c9/html5/thumbnails/24.jpg)
MPEG7 Details
• How to extract semantics?
– i.e. Intelligent Information Retrieval
– Drawback of the standard
– Ontology help required:
• Domain Specific Ontology (Football, Location)
• Automatically extract information
• Use for a better search result
![Page 25: MMBD - Multimedia Databases](https://reader033.vdocuments.us/reader033/viewer/2022052900/555e9fb7d8b42a6d068b54c9/html5/thumbnails/25.jpg)
Research Fields
• Design: still in research
• Queries: techniques need to be modified
• Rest:
– Modeling: complex objects, wide range of types
– Storage: representation, compression, buffering during I/O, mapping
– Performance: physical limitations, parallel processing
![Page 26: MMBD - Multimedia Databases](https://reader033.vdocuments.us/reader033/viewer/2022052900/555e9fb7d8b42a6d068b54c9/html5/thumbnails/26.jpg)
• Thank you!
• Questions?
![Page 27: MMBD - Multimedia Databases](https://reader033.vdocuments.us/reader033/viewer/2022052900/555e9fb7d8b42a6d068b54c9/html5/thumbnails/27.jpg)
References
• Wikipedia: Various Pages• Computer Science and Engineering Department
Resources:– University of Notre Dame– Northumbria University– Carnegie Mellon University– Boston College– Simon Fraser University– Georgia Institute of Technology
• Interview with A. Anil Sinaci