![Page 1: Multimedia Database Management System Wei Tsang Ooi CS731](https://reader036.vdocuments.us/reader036/viewer/2022062407/56649d4e5503460f94a2e004/html5/thumbnails/1.jpg)
Multimedia Database Management System
Wei Tsang OoiCS731
![Page 2: Multimedia Database Management System Wei Tsang Ooi CS731](https://reader036.vdocuments.us/reader036/viewer/2022062407/56649d4e5503460f94a2e004/html5/thumbnails/2.jpg)
MMDBMS : Querying Interface,
Indexing and
Buffer Management
![Page 3: Multimedia Database Management System Wei Tsang Ooi CS731](https://reader036.vdocuments.us/reader036/viewer/2022062407/56649d4e5503460f94a2e004/html5/thumbnails/3.jpg)
Why MMDBMS ?
Concurrency ControlRecoveryPrivacyQuery SupportVersion Control
![Page 4: Multimedia Database Management System Wei Tsang Ooi CS731](https://reader036.vdocuments.us/reader036/viewer/2022062407/56649d4e5503460f94a2e004/html5/thumbnails/4.jpg)
Example of MMDBMS
Digital LibraryNews-On-DemandVideo-On-DemandMusic DatabaseTelemedicineGeographic Information
System
![Page 5: Multimedia Database Management System Wei Tsang Ooi CS731](https://reader036.vdocuments.us/reader036/viewer/2022062407/56649d4e5503460f94a2e004/html5/thumbnails/5.jpg)
No Intergration
size fps titleid filename
000001 530M 30 soam l6.mpg
000002 450M 30 tibor l7.mpg
000003 600M 30 parag l5.mpg
000004 510M 30 wei l4.mpg
![Page 6: Multimedia Database Management System Wei Tsang Ooi CS731](https://reader036.vdocuments.us/reader036/viewer/2022062407/56649d4e5503460f94a2e004/html5/thumbnails/6.jpg)
Semi-intergrated
000001b70ae9902...
BLOB
![Page 7: Multimedia Database Management System Wei Tsang Ooi CS731](https://reader036.vdocuments.us/reader036/viewer/2022062407/56649d4e5503460f94a2e004/html5/thumbnails/7.jpg)
Fully Intergrated
index
buffers
storage
![Page 8: Multimedia Database Management System Wei Tsang Ooi CS731](https://reader036.vdocuments.us/reader036/viewer/2022062407/56649d4e5503460f94a2e004/html5/thumbnails/8.jpg)
Nature of Multimedia Data
Large amount of dataTime sensitiveVague matching
![Page 9: Multimedia Database Management System Wei Tsang Ooi CS731](https://reader036.vdocuments.us/reader036/viewer/2022062407/56649d4e5503460f94a2e004/html5/thumbnails/9.jpg)
Database Components
Query Interface
Query Processing
indexbuffermanager
storagemanager
![Page 10: Multimedia Database Management System Wei Tsang Ooi CS731](https://reader036.vdocuments.us/reader036/viewer/2022062407/56649d4e5503460f94a2e004/html5/thumbnails/10.jpg)
Query Interface and
Processing
![Page 11: Multimedia Database Management System Wei Tsang Ooi CS731](https://reader036.vdocuments.us/reader036/viewer/2022062407/56649d4e5503460f94a2e004/html5/thumbnails/11.jpg)
Problems
Needs support for :• temporal and spatial relations• “natural” interface• fuzzy query
SQL is not suitable.
![Page 12: Multimedia Database Management System Wei Tsang Ooi CS731](https://reader036.vdocuments.us/reader036/viewer/2022062407/56649d4e5503460f94a2e004/html5/thumbnails/12.jpg)
13 Temporal Relations
Allen 83
before
meets
overlaps
finished by
contains
started by
equals
starts
finishes
during
overlapped by
met by
after
![Page 13: Multimedia Database Management System Wei Tsang Ooi CS731](https://reader036.vdocuments.us/reader036/viewer/2022062407/56649d4e5503460f94a2e004/html5/thumbnails/13.jpg)
Spatio Relations
Papadias, Theodoridis 96• Topological Relations• Directional Relations
![Page 14: Multimedia Database Management System Wei Tsang Ooi CS731](https://reader036.vdocuments.us/reader036/viewer/2022062407/56649d4e5503460f94a2e004/html5/thumbnails/14.jpg)
Topological Relations
disjoint
meet
overlap
covered by
inside
equal
covers
contains
![Page 15: Multimedia Database Management System Wei Tsang Ooi CS731](https://reader036.vdocuments.us/reader036/viewer/2022062407/56649d4e5503460f94a2e004/html5/thumbnails/15.jpg)
Directional Relations
North
South
West East
NENW
SESW
![Page 16: Multimedia Database Management System Wei Tsang Ooi CS731](https://reader036.vdocuments.us/reader036/viewer/2022062407/56649d4e5503460f94a2e004/html5/thumbnails/16.jpg)
Spatio-temporal Relations
overlap-aftermeet-during...
![Page 17: Multimedia Database Management System Wei Tsang Ooi CS731](https://reader036.vdocuments.us/reader036/viewer/2022062407/56649d4e5503460f94a2e004/html5/thumbnails/17.jpg)
Querying
ImageAudio
• Music• Sound• Speech
Video
![Page 18: Multimedia Database Management System Wei Tsang Ooi CS731](https://reader036.vdocuments.us/reader036/viewer/2022062407/56649d4e5503460f94a2e004/html5/thumbnails/18.jpg)
Querying Image
Common approach• allow query by sketches (color,
shape, texture) or examples.• perform matching by Feature
VectorsF = (v1, v2, ... vn)
• e.g. Color Histogram
![Page 19: Multimedia Database Management System Wei Tsang Ooi CS731](https://reader036.vdocuments.us/reader036/viewer/2022062407/56649d4e5503460f94a2e004/html5/thumbnails/19.jpg)
Querying Image
Exisiting Systems :• QBIC • VisualSEEK• PhotoBook• Virage• FourEyes
![Page 20: Multimedia Database Management System Wei Tsang Ooi CS731](https://reader036.vdocuments.us/reader036/viewer/2022062407/56649d4e5503460f94a2e004/html5/thumbnails/20.jpg)
Querying Music
Hawley 93• Input by MIDI Keyboard• Measure relative pitch (U, D, S)• Perform exact match with
existing database.
![Page 21: Multimedia Database Management System Wei Tsang Ooi CS731](https://reader036.vdocuments.us/reader036/viewer/2022062407/56649d4e5503460f94a2e004/html5/thumbnails/21.jpg)
Querying Music
Ghias, Logan, Chamberlin & Smith 95• Input by humming• Extract relative pitch• Perform approximate matching
![Page 22: Multimedia Database Management System Wei Tsang Ooi CS731](https://reader036.vdocuments.us/reader036/viewer/2022062407/56649d4e5503460f94a2e004/html5/thumbnails/22.jpg)
Querying Music
Chou, Chen, & Liu 96• Query by chord• Represents musics by chord
• C Am Em F C Am Em F ...
• Perform fuzzy match
![Page 23: Multimedia Database Management System Wei Tsang Ooi CS731](https://reader036.vdocuments.us/reader036/viewer/2022062407/56649d4e5503460f94a2e004/html5/thumbnails/23.jpg)
Querying Music
Chen & Chen 98• Query by “rhythm” (tempo ?)• Represents musics by rhythm
• | ¶¶— | ¶·¶·| ¶¶¶¶ | ...
• Perform fuzzy match
![Page 24: Multimedia Database Management System Wei Tsang Ooi CS731](https://reader036.vdocuments.us/reader036/viewer/2022062407/56649d4e5503460f94a2e004/html5/thumbnails/24.jpg)
Querying Sound
Wold, Blum, Keisar & Wheaton 96• Analyze audio to extract features
•loudness, pitch, brightness, bandwidth and harmonicity
• Segment the audio to pieces• Feature Vector Matchings
![Page 25: Multimedia Database Management System Wei Tsang Ooi CS731](https://reader036.vdocuments.us/reader036/viewer/2022062407/56649d4e5503460f94a2e004/html5/thumbnails/25.jpg)
Querying Speech
Hauptmann & Witbrock 97Informedia
• Use speech recorgnition to convert audio to text
• segment audio using silence detection
Query by speaking keywords
![Page 26: Multimedia Database Management System Wei Tsang Ooi CS731](https://reader036.vdocuments.us/reader036/viewer/2022062407/56649d4e5503460f94a2e004/html5/thumbnails/26.jpg)
Querying Video
Query by speech• Informedia
Visual approach• VideoQ
![Page 27: Multimedia Database Management System Wei Tsang Ooi CS731](https://reader036.vdocuments.us/reader036/viewer/2022062407/56649d4e5503460f94a2e004/html5/thumbnails/27.jpg)
VideoQ
Chang et. al 97• User can sketch objects• Specifies
• color• texture• shape• motion• duration• camera zoom and pan.
![Page 28: Multimedia Database Management System Wei Tsang Ooi CS731](https://reader036.vdocuments.us/reader036/viewer/2022062407/56649d4e5503460f94a2e004/html5/thumbnails/28.jpg)
VisualQ Example
Someone SkiingTitanic Sinking
![Page 29: Multimedia Database Management System Wei Tsang Ooi CS731](https://reader036.vdocuments.us/reader036/viewer/2022062407/56649d4e5503460f94a2e004/html5/thumbnails/29.jpg)
Indexing
![Page 30: Multimedia Database Management System Wei Tsang Ooi CS731](https://reader036.vdocuments.us/reader036/viewer/2022062407/56649d4e5503460f94a2e004/html5/thumbnails/30.jpg)
Indexing
Requirements• support spatio-temporal
operations• support fuzzy matches
![Page 31: Multimedia Database Management System Wei Tsang Ooi CS731](https://reader036.vdocuments.us/reader036/viewer/2022062407/56649d4e5503460f94a2e004/html5/thumbnails/31.jpg)
Indexing Images
N-dimentional indices for feature vector
Well studied in DB/CG community
Two examples :• VP-tree• R-tree
![Page 32: Multimedia Database Management System Wei Tsang Ooi CS731](https://reader036.vdocuments.us/reader036/viewer/2022062407/56649d4e5503460f94a2e004/html5/thumbnails/32.jpg)
VP-tree
PQ
R
ST
U
V
W
R
PQS VWTU
![Page 33: Multimedia Database Management System Wei Tsang Ooi CS731](https://reader036.vdocuments.us/reader036/viewer/2022062407/56649d4e5503460f94a2e004/html5/thumbnails/33.jpg)
VP-tree
PQ
R
ST
U
V
W
R
P
S Q
U
T VW
![Page 34: Multimedia Database Management System Wei Tsang Ooi CS731](https://reader036.vdocuments.us/reader036/viewer/2022062407/56649d4e5503460f94a2e004/html5/thumbnails/34.jpg)
R-tree
![Page 35: Multimedia Database Management System Wei Tsang Ooi CS731](https://reader036.vdocuments.us/reader036/viewer/2022062407/56649d4e5503460f94a2e004/html5/thumbnails/35.jpg)
Indexing Audio
Audio are modeled as stringsInexact match is neededCommon indices for string
search can be used
Example• PAT-tree
![Page 36: Multimedia Database Management System Wei Tsang Ooi CS731](https://reader036.vdocuments.us/reader036/viewer/2022062407/56649d4e5503460f94a2e004/html5/thumbnails/36.jpg)
PAT-tree
ab b c
abc c abc c
ababc abc babc bc c
![Page 37: Multimedia Database Management System Wei Tsang Ooi CS731](https://reader036.vdocuments.us/reader036/viewer/2022062407/56649d4e5503460f94a2e004/html5/thumbnails/37.jpg)
Indexing Video
Treat time as third dimensionWe can use any
multidimension indexing structures
![Page 38: Multimedia Database Management System Wei Tsang Ooi CS731](https://reader036.vdocuments.us/reader036/viewer/2022062407/56649d4e5503460f94a2e004/html5/thumbnails/38.jpg)
Buffer Management
![Page 39: Multimedia Database Management System Wei Tsang Ooi CS731](https://reader036.vdocuments.us/reader036/viewer/2022062407/56649d4e5503460f94a2e004/html5/thumbnails/39.jpg)
Buffer Management
Minimizing response timeEnsuring continuity and
synchronization
Prefetching & Replacement
![Page 40: Multimedia Database Management System Wei Tsang Ooi CS731](https://reader036.vdocuments.us/reader036/viewer/2022062407/56649d4e5503460f94a2e004/html5/thumbnails/40.jpg)
Glossaries
PresentationMedia StreamMedia Object
![Page 41: Multimedia Database Management System Wei Tsang Ooi CS731](https://reader036.vdocuments.us/reader036/viewer/2022062407/56649d4e5503460f94a2e004/html5/thumbnails/41.jpg)
Buffer Management
Relevance BasedDistance BasedBuffer Consumption
![Page 42: Multimedia Database Management System Wei Tsang Ooi CS731](https://reader036.vdocuments.us/reader036/viewer/2022062407/56649d4e5503460f94a2e004/html5/thumbnails/42.jpg)
Relevance Based
Moser, Kraiß & Klas 95 : L/MRPDefinition:
• State = (curr obj, skip)• Relevance : State Real
![Page 43: Multimedia Database Management System Wei Tsang Ooi CS731](https://reader036.vdocuments.us/reader036/viewer/2022062407/56649d4e5503460f94a2e004/html5/thumbnails/43.jpg)
Relevance Based
Prefetch :• future objects with highest
relevanceReplace :
• old objects with lowest relevance
![Page 44: Multimedia Database Management System Wei Tsang Ooi CS731](https://reader036.vdocuments.us/reader036/viewer/2022062407/56649d4e5503460f94a2e004/html5/thumbnails/44.jpg)
Example (5,+2)
History
ReferencedSkip
Time Scale
Relevance
![Page 45: Multimedia Database Management System Wei Tsang Ooi CS731](https://reader036.vdocuments.us/reader036/viewer/2022062407/56649d4e5503460f94a2e004/html5/thumbnails/45.jpg)
Distance Based
Özden, Rastogi & Silberschatz 96
Definition• distance of a client C is the
offset between C and the immediate following client.
![Page 46: Multimedia Database Management System Wei Tsang Ooi CS731](https://reader036.vdocuments.us/reader036/viewer/2022062407/56649d4e5503460f94a2e004/html5/thumbnails/46.jpg)
Distance Based
Replace :• blocks consumed by clients with
largest distance
![Page 47: Multimedia Database Management System Wei Tsang Ooi CS731](https://reader036.vdocuments.us/reader036/viewer/2022062407/56649d4e5503460f94a2e004/html5/thumbnails/47.jpg)
Example
C4C3
C2
C1
![Page 48: Multimedia Database Management System Wei Tsang Ooi CS731](https://reader036.vdocuments.us/reader036/viewer/2022062407/56649d4e5503460f94a2e004/html5/thumbnails/48.jpg)
Example
C4C3
C2
C1
![Page 49: Multimedia Database Management System Wei Tsang Ooi CS731](https://reader036.vdocuments.us/reader036/viewer/2022062407/56649d4e5503460f94a2e004/html5/thumbnails/49.jpg)
Example
C4C3
C2
C1
![Page 50: Multimedia Database Management System Wei Tsang Ooi CS731](https://reader036.vdocuments.us/reader036/viewer/2022062407/56649d4e5503460f94a2e004/html5/thumbnails/50.jpg)
Buffer Consumption
Wu & Yu 97Definition :
• Buffer consumption = amount of buffer used x total time
![Page 51: Multimedia Database Management System Wei Tsang Ooi CS731](https://reader036.vdocuments.us/reader036/viewer/2022062407/56649d4e5503460f94a2e004/html5/thumbnails/51.jpg)
Buffer Consumption
Result :• Minimizing buffer consumption
increase system throughput• Increase retrieval rate of current
stream is better then prefetchingnext waiting stream
![Page 52: Multimedia Database Management System Wei Tsang Ooi CS731](https://reader036.vdocuments.us/reader036/viewer/2022062407/56649d4e5503460f94a2e004/html5/thumbnails/52.jpg)
Open Problems
![Page 53: Multimedia Database Management System Wei Tsang Ooi CS731](https://reader036.vdocuments.us/reader036/viewer/2022062407/56649d4e5503460f94a2e004/html5/thumbnails/53.jpg)
Querying
Query video by action ?Query music by
• relative pitch + chord + rhythm ?
Continuing quest to improve accuracy
![Page 54: Multimedia Database Management System Wei Tsang Ooi CS731](https://reader036.vdocuments.us/reader036/viewer/2022062407/56649d4e5503460f94a2e004/html5/thumbnails/54.jpg)
Indexing
Well studied problemIndexing for approximate
string search ?
![Page 55: Multimedia Database Management System Wei Tsang Ooi CS731](https://reader036.vdocuments.us/reader036/viewer/2022062407/56649d4e5503460f94a2e004/html5/thumbnails/55.jpg)
Buffer Management
Should take into consideration:• object dependencies (MPEG)