8/12/2019 04MPEG4

1/71

MULTIMEDIA SYSTEMSIREK DEFEE

MPEG-4, NETWORKED

MULTIMEDIA

STANDARD

8/12/2019 04MPEG4

2/71

MULTIMEDIA DATA REPRESENTATION

MULTIMEDIA SYSTEMS CAN BE MADE IF

THERE IS REPRESENTATION OF

MULTIMEDIA DATA. SUCH REPRESENTATION SHOULD BE

STANDARDIZED SO EVERYBODY CAN

USE IT

THERE ARE MANY DIFFERENT SUCH

STANDARDS, WE WILL GIVE EXAMPLE

OF MOST A ADVANCED ONE

MULTIMEDIA SYSTEMSIREK DEFEE

8/12/2019 04MPEG4

3/71

MULTIMEDIA SYSTEMSIREK DEFEE

MPEG-4 IS A STANDARD WHICH ENABLESEFFICIENT REPRESENTATION OF

COMPLEX MULTIMEDIA DATA FOR

FULLY INTERACTIVE APPLICATIONS

THE MPEG-4 STANDARD IS OBJECT-BASED

8/12/2019 04MPEG4

4/71

MULTIMEDIA SYSTEMSIREK DEFEE

8/12/2019 04MPEG4

5/71

MULTIMEDIA SYSTEMSIREK DEFEE

8/12/2019 04MPEG4

6/71

MULTIMEDIA SYSTEMSIREK DEFEE

8/12/2019 04MPEG4

7/71

8/12/2019 04MPEG4

8/71

MULTIMEDIA SYSTEMSIREK DEFEE

8/12/2019 04MPEG4

9/71

MULTIMEDIA SYSTEMSIREK DEFEE

8/12/2019 04MPEG4

10/71

MULTIMEDIA SYSTEMSIREK DEFEE

8/12/2019 04MPEG4

11/71

MULTIMEDIA SYSTEMSIREK DEFEE

8/12/2019 04MPEG4

12/71

MULTIMEDIA SYSTEMSIREK DEFEE

8/12/2019 04MPEG4

13/71

MULTIMEDIA SYSTEMSIREK DEFEE

8/12/2019 04MPEG4

14/71

MULTIMEDIA SYSTEMSIREK DEFEE

8/12/2019 04MPEG4

15/71

MULTIMEDIA SYSTEMSIREK DEFEE

8/12/2019 04MPEG4

16/71

MULTIMEDIA SYSTEMSIREK DEFEE

8/12/2019 04MPEG4

17/71

MULTIMEDIA SYSTEMSIREK DEFEE

8/12/2019 04MPEG4

18/71

MULTIMEDIA SYSTEMSIREK DEFEE

8/12/2019 04MPEG4

19/71

MULTIMEDIA SYSTEMSIREK DEFEE

8/12/2019 04MPEG4

20/71

MULTIMEDIA SYSTEMSIREK DEFEE

VIDEO OBJECT-BASED TREATMENT AND

COMPRESSION MAKE THE

MPEG-4 STANDARD VERY FLEXIBLE MODERN

AND EFFICIENT

BASIC VIDEO AND AUDIO PARTS OF THIS

STANDARD ARE USED IN DIGITAL TELEVISION

AND BLUE RAY DISCS

BUT MPEG-4 INVOLVES ALL DATA TYPES AND

ALSO SPECIAL TOOLS FOR PARTICULAR

APPLICATIONS AND WE GIVE EXAMPLES OF

THEM NEXT

8/12/2019 04MPEG4

21/71

MULTIMEDIA SYSTEMSIREK DEFEE

Sy!"#!$% V$&'() T**)& $ MPEG-4

8/12/2019 04MPEG4

22/71

MULTIMEDIA SYSTEMSIREK DEFEE

M*!$+(!$*

A # !y# * /(!( $& (#(0$1 $ 2')!$2#/$(()$%(!$*&3 Sy!"#!$%

B*!" &y!"#!$% (/ (!'0() /(!( %(

%*-#$&! $ !*/(y5& ()$%(!$*&

T"$& # /(!( ##/& !* 6# %*20#&/ (/

&!0#(2#/ $ 2*&! ()$%(!$*&

N# !#%"*)*1$#& (0# ##/#/ *03

C*20#&&$* (/ &!0#(2$1 * &y!"#!$% /(!(

8/12/2019 04MPEG4

23/71

MULTIMEDIA SYSTEMSIREK DEFEE

S*2# #(2)#& * ()$%(!$*&S*2# #(2)#& * ()$%(!$*&

7D +$/#*

A'12#!#/ 0#()$!y

T#)#0#%# S%$#!$$% +$&'()$8(!$*

V$0!'() 0#()$!y

9

8/12/2019 04MPEG4

24/71

MULTIMEDIA SYSTEMSIREK DEFEE

8/12/2019 04MPEG4

25/71

MULTIMEDIA SYSTEMSIREK DEFEE

A)$%(!$* #(2)#3 T#)#0#%#

The room is generated in computer

8/12/2019 04MPEG4

26/71

MULTIMEDIA SYSTEMSIREK DEFEE

A)$%(!$* #(2)#3 S%$#!$$% +$&'()$8(!$*

8/12/2019 04MPEG4

27/71

MULTIMEDIA SYSTEMSIREK DEFEE

A)$%(!$* #(2)#3 V$0!'() 7D *0)/

8/12/2019 04MPEG4

28/71

MULTIMEDIA SYSTEMSIREK DEFEE

Sy!"#!$% V$&'() T**)& $ MPEG-4

V#0&$* :

F(%# ($2(!$*

;D /y(2$% 2#&"S%()(6)# %*/$1 * &y!"#!$% !#!'0#

V$# /##/#! &%()(6)# %*/$1 * !#!'0#

V#0&$* ;

B*/y ($2(!$*

7D 2*/#) %*20#&&$*

8/12/2019 04MPEG4

29/71

8/12/2019 04MPEG4

30/71

MULTIMEDIA SYSTEMSIREK DEFEE

E(2)# * ()$%(!$*&

V$0!'() 2##!$1, !#)#-0#%#, +$/#*-

%*#0#%$1, ...

V$0!'() &!*0y !#))#0, +$0!'() (%!*0, '

$!#0(%#, ...

G(2#&, (+(!(0&, ...

8/12/2019 04MPEG4

31/71

MULTIMEDIA SYSTEMSIREK DEFEE

F(%# ($2(!$* F(%#: ( *6

8/12/2019 04MPEG4

32/71

MULTIMEDIA SYSTEMSIREK DEFEE

F(%# ($2(!$*

S"(#, !#!'0# (/ #0#&&$*&3

%$$#/ (0(2#!#0& $ !"#

$%*2$1 6$!&!0#(2

0#2*!# (& #)) (& )*%() %*!0*)

* !"# (0(2#!#0&

8/12/2019 04MPEG4

33/71

MULTIMEDIA SYSTEMSIREK DEFEE

I$!$() (%# *6

8/12/2019 04MPEG4

34/71

MULTIMEDIA SYSTEMSIREK DEFEE

F(%# ($2(!$* (0(2#!#0&

T"0## !& * (0(2#!#0& '/ !* /#&%0$6#

( (%# (/ $!& ($2(!$* %"(0(%!#0&!$%&3

Facial Definition Parameters (FDPs)

Facial Animation Parameters (FAPs)

Facial Interpolation Transform (FIT)

8/12/2019 04MPEG4

35/71

MULTIMEDIA SYSTEMSIREK DEFEE

F(%$() D#$$!$* P(0(2#!#0& - FDP&

D#$#& ( %$$% (%# +$(

7D #(!'0# *$!&

7D 2#&"@&%## 10("

F(%# T#!'0#

F(%# A$2(!$* T(6)# FAT

11.511.5

8/12/2019 04MPEG4

36/71

MULTIMEDIA SYSTEMSIREK DEFEE !

F(%# F#(!'0# P*$!&

N*02()$8# ($2(!$*

(0(2#!#0&

F$/ #(!'0# %*00#&*/#%#

$ /$#0#! (%#& R*'1")y /#$# &"(#

xy

z

11.4

11.2

10.2

10.4

10.10

10.8

10.6

2.14

7.1

11.6 4.6

4.4

4.2

5.2

5.4

2.10

2.122.1

11.1

Tongue

6.26.4 6.3

6.1Mouth

8.18.9 8.10

8.58.3

8.7

8.2

8.8

8.48.6

2.2

2.3

2.6

2.82.9

2.72.5 2.4

2.12.12 2.11

2.14

2.10

2.13

10.6 10.8

10.4

10.2

10.10

5.4

5.2

5.3

5.1

10.1

10.910.3

10.510.7

4.1 4.34.54.6

4.4 4.2

11.111.2 11.3

11.4

x

y

z

Nose

9.6 9.7

9.14 9.13

9.12

9.2

9.4 9.15 9.5

9.3

9.1

Teeth

9.109.11

9.8

9.9

Feature points ae!te" #y F$%s

&ther eature points

'ight eye (et eye

3.13

3.7

3.9

3.5

3.1

3.3

3.11

3.14

3.10

3.12 3.6

3.4

3.2

3.8

8/12/2019 04MPEG4

37/71

MULTIMEDIA SYSTEMSIREK DEFEE

7D 2#&" (/ #(!'0# *$!&

8/12/2019 04MPEG4

38/71

MULTIMEDIA SYSTEMSIREK DEFEE

T#!'0#

8/12/2019 04MPEG4

39/71

MULTIMEDIA SYSTEMSIREK DEFEE

FDP&

F(%# D#&%0$!$* P(0(2#!#0&

T* 2*/#&3 T* %'&!*2$8# !"# (%# 2*/#) (! !"# 0#%#$+#0 !*

( (0!$%')(0 (%# T* /*)*(/ ( (%# 2*/#) ()*1 $!" $!&

($2(!$* $*02(!$*

G##0())y, ! *%# #0 &&$* *0 %()$60(!$* (/@*0 /*)*(/

%*')/ 6# ! 2*0# *!# *0 =%$() ##%!&>

8/12/2019 04MPEG4

40/71

MULTIMEDIA SYSTEMSIREK DEFEE

F(%# A$2(!$* P(0(2#!#0& - FAP&

R#0#! ( %*2)#!# ! * (%$() (%!$*&

())* 0#0#!(!$* * 2*&! * !"#

(!'0() (%$() #0#&&$*&

A)) FAP& $+*)+$1 !0(&)(!$*()2*+#2#!3 $ !#02& * F(%$() A$2(!$*

P(0(2#!#0 U$!& FAPU&

A))*& %*&$&!#! $!#00#!(!$* * FAP&* (y (%$() 2*/#).

8/12/2019 04MPEG4

41/71

MULTIMEDIA SYSTEMSIREK DEFEE

FA"Us

ES

ENS

MNS

MW

IRISD

IRISD# Iris diameter $%& de'inition it is

e(ua) to the distance %et*een

upper ad )o*er e&e)id+ in neutra)

'ace

IRISD ,

IRISD# - .#/0

ES# E&e separation ES , ES# - .#/0E1S# E&e 2 nose separation E1S , E1S# - .#/0

M1S# Mouth 2 nose separation M1S , M1S# - .#/0

M3# Mouth *idth M3# - .#/0

AU Ang)e Unit .#E24 rad

8/12/2019 04MPEG4

42/71

MULTIMEDIA SYSTEMSIREK DEFEE

FAP&

F(%$() A%!$* P(0(2#!#0&

; "$1"-)#+#) FAPS3 Viseme+$&'() %*00#)(!# * "*#2#

Expression

8/12/2019 04MPEG4

43/71

MULTIMEDIA SYSTEMSIREK DEFEE

W"(! $& *! &!(/(0/$8#/

T"# (y !* #!0(%! !"# (0(2#!#0&

M(0#0&

S##%" /0$+# I2(1# (()y&$& (/

#(!'0# #!0(%!$*

T"# %"*$%# * "$%" (0(2#!#0& !* %*/# (/

$!" "$%" 0#%$&$* '(!$8(!$*

R(!# %*!0*)

8/12/2019 04MPEG4

44/71

MULTIMEDIA SYSTEMSIREK DEFEE

FIT&

F(%$() I!#0*)(!$* R')#&

S#%$$%(!$* * $!#0*)(!$* 0')#& *0

&*2#@()) FAP& 6y !"# /#0.

S#/#0 %$$#&FAP Interpolation Graph(FIG)(/ ! * $!#0*)(!$* '%!$*&

A))*& "$1"#0 /#10## * %*!0*) *+#0 !"#($2(!$* 0#&')!&.

8/12/2019 04MPEG4

45/71

MULTIMEDIA SYSTEMSIREK DEFEE

W"(! $& &!(/(0/$8#/

FDP& BIFS Sy!( (/ S#2(!$%&

R')#& *0 /#%*/$1 (/ (/(!(!$*

FAP& B$!&!0#(2 &y!( (/ (!$%&

R')#& *0 /#%*/$1 (/ ($2(!$*

FIT& Sy!( (/ (!$%&

D#%*/$1 0')#&

8/12/2019 04MPEG4

46/71

8/12/2019 04MPEG4

47/71

MULTIMEDIA SYSTEMS

IREK DEFEE

M#/$( O6

8/12/2019 04MPEG4

48/71

MULTIMEDIA SYSTEMS

IREK DEFEE

A/+(!(1#& * O6

8/12/2019 04MPEG4

49/71

MULTIMEDIA SYSTEMS

IREK DEFEE

Sy&!#2 O+#0+$#

(istener

*e+ux %resent,yn!h *e!o"e -o+posentera!tion

$u"io

,!ene

/raph

1

2

*es!rip

tion

%'Mg+t

nter.

*e! 1

*e! 2

-hanne

-o+poun"

e!t

%'

-ontro

O6

8/12/2019 04MPEG4

50/71

MULTIMEDIA SYSTEMS

IREK DEFEE

A'/$* O6

8/12/2019 04MPEG4

51/71

MULTIMEDIA SYSTEMS

IREK DEFEE

S%()(6$)$!y /##/$1 * !"# 6$! 0(!#

Sate))ite 5e))u)ar phone Internet ISD1

Secure com6

/ 0 ! 7 .# ./ .0 .! /0 / 07 !0

0 89: T&pica) Audio %and*idth7 89: /# 89:

%it2rate $8%ps+

Sca)a%)e 5oder

Speech coding

TTS

;enera) audio coding

8/12/2019 04MPEG4

52/71

MULTIMEDIA SYSTEMS

IREK DEFEE

A)$%(!$* D*2($&P0*$)#&

S##%"

)* 0(!# #%" %*/#0& (/ TTS T#!-!*-S##%"

S%()(6)#

#%" %*/#0&

1##0() ('/$* %*/#0&

()) %*/#0& $ &%()(6)# %*$1'0(!$*

Sy!"#!$%

(+#!(6)# &y!"#&$&

&%*0# /0$+# &y!"#&$&

TTS

8/12/2019 04MPEG4

53/71

MULTIMEDIA SYSTEMS

IREK DEFEE

MPEG-4 S##%" C*/$13 O+#0+$#

E%#))#! %*20#&&$* 6y '&$1 &*'0%# 2*/#)

L$#(0 P0#/$%!$+# C*/$1 LPC

P$!%" *0 *$ #%$!(!$* B#!!#0 %*20#&&$* !"( =1##0() ('/$*>

%*/#0&

*)y *0 =%)#( #%"> 0*2 &$1)# !()#0

8/12/2019 04MPEG4

54/71

MULTIMEDIA SYSTEMS

IREK DEFEE

S##%" C*/#0&

H(02*$% V#%!*0 E%$!(!$* C*/#0 HVXC

C*/# E%$!(!$* L$#(0 P0#/$%!$* CELP

W$/#6(/ CELP

,igna12an"3i"th4

56z

-hanne itrate 5#s

HVXC

0 10 20

WB CELP

CELP

1.2 4 6 24

16 24

3.6

7.2

8/12/2019 04MPEG4

55/71

MULTIMEDIA SYSTEMS

IREK DEFEE

C*22'$%(!$* C"(0(%!#0$&!$%& * C*/#0&

L* 6$! 0(!#

HVXC :.; 6@& !* :. 6@& +(0. 0(!#

;. 6@& !* 4. 6@& %*&!. 0(!#

CELP 4. 6@& !* ;4 6@& %*&!. 0(!#

L* *#-(y /#)(y

HVXC 77. 2& !* 2&

CELP : 2& !* 4 2&

N*! %*20*2$/ *0 2*/#2 &$1()&

8/12/2019 04MPEG4

56/71

MULTIMEDIA SYSTEMS

IREK DEFEE

B$! R(!# S%()(6$)$!y

P(0(2#!#0& %*/#/ '&$1 2')!$-&!(1# V#%!*0

'(!$8(!$*

6( )'& #"(%#2#! )(y#0 E"(%#2#! )(y#0& %( 6# &!0$#/ $

+#0

%"(#)/#%*/#0

8/12/2019 04MPEG4

57/71

MULTIMEDIA SYSTEMS

IREK DEFEE

P(0(2#!#0 U/(!# S%()(6$)$!y

L$#(0 P0#/$%!$* M*/#)

'/(!#/ #+#0y 0(2#

$!#0*)(!#/ #+#0y &'6-0(2#

E%$!(!$*

1($ '/(!#/ #+#0y &'60(2#

Fra+es

,u#8Fra+es

8/12/2019 04MPEG4

58/71

MULTIMEDIA SYSTEMS

IREK DEFEE

MPEG-4 BIFS

MPEG-4 DATA INCLUDE VARIOUS MEDIA TYPES

WHICH CAN BE USED AT THE SAME TIME

THIS REUIRES MECHANISM FOR THEIR

ORGANIZATION IN TIME AND SPACE

THIS MECHANISM IN THE MPEG-4 STANDARD IS

CALLED

BIFS BINARY FORMAT FOR SCENES

8/12/2019 04MPEG4

59/71

MULTIMEDIA SYSTEMS

IREK DEFEE

BIFS3 WHY

MPEG-4 $& ( *6

8/12/2019 04MPEG4

60/71

MULTIMEDIA SYSTEMS

IREK DEFEE

E(2)# * ( MPEG-4 A'/$*+$&'() S%##

:

;D A'/$*-+$&'()

A'/$* (/ V$/#* S!$)) I2(1#&

;D A'/$*-+$&'() &%##

A'/$* (/ V$/#* S%0*))$1 T#!

(/ S!$)) I2(1#&

8/12/2019 04MPEG4

61/71

MULTIMEDIA SYSTEMS

IREK DEFEE

E(2)# * ( MPEG-4 A'/$*+$&'() S%##

;

7D A'/$*-+$&'() &%##

7D W*0)/ (06$!0(0y &"(#/

+$/#* &!$)) $2(1#& 7D O6

8/12/2019 04MPEG4

62/71

MULTIMEDIA SYSTEMS

IREK DEFEE

BIFS S%## F#(!'0#& +;

B*/y A$2(!$*

A/+(%#/ A'/$* P#0%#!'() (0*(%" !* 2*/$y (!'0() &*'0%# A%*'&!$% 0*#0!$#& *0 "y&$%() 6(/ ('/$* 0#/#0$1

S!0#(2 (/ +#0 %*!0*)

VCR %*!0*)& (/ A)$%(!$* %$$% 2#&&(1$1

E!#&$6$)$!y P0*!*!y#& D#$$!$* * # BIFS $!#0(%#&

H$#0(0%"$%() 7D *6

8/12/2019 04MPEG4

63/71

MULTIMEDIA SYSTEMS

IREK DEFEE

...

Scene Description Stream

Object Descriptor Stream

Visual Stream

Visual Stream

Visual Stream

Audio Stream

Interactive Scene Description

MPEG-4 Sy&!#2& P0$%$)#

W# "(+#

/(!( &!0#(2&

(0(!# *0

#(%" *6

8/12/2019 04MPEG4

64/71

MULTIMEDIA SYSTEMS

IREK DEFEE

BIFS %*!#! $ MPEG-4 &y&!#2

9'M( No"es9'M( No"es

M%:/84 No"esM%:/84 No"es

M%:/84 ,trea+sM%:/84 ,trea+s

*:(9:';*:(9:';

BIFS-Update ESBIFS-Update ES

BIFS Anim ESBIFS Anim ES

%':,:NT$T&N%':,:NT$T&N

CC

OO

MM

PPOO

SS

II

TTII

OO

NN

RR

EE

NNDD

EE

RR

II

NN

GG

*:-&*N/*:-&*N/

DD

EE

CCOO

DD

II

NNGG

,-:N: /'$%6,-:N: /'$%6

M$N$/:M:NTM$N$/:M:NT

FBAFBA

SN S!"ndSN S!"nd

Inte#a$ti!nInte#a$ti!n

%D&'D N!de(%D&'D N!de(

'D N!de('D N!de(

A"di! N!de(A"di! N!de(

%D N!de(%D N!de(

MPEG 4 A $ ! ! / M )!$ /$ S !

8/12/2019 04MPEG4

65/71

MULTIMEDIA SYSTEMS

IREK DEFEE

...

De$!din)

M%:/84 ntera!ti

8/12/2019 04MPEG4

66/71

MULTIMEDIA SYSTEMS

IREK DEFEE

081

,3it!h

2F,8-o++an"

:,RS. CV

8/12/2019 04MPEG4

67/71

MULTIMEDIA SYSTEMS

IREK DEFEE

IPP P...

2F,8$ni+ :,

8/12/2019 04MPEG4

68/71

MULTIMEDIA SYSTEMS

IREK DEFEE

BIFS S%## C*20#&&$*

BIFS S%##

(%!*0 :-; * &%## !#! $)#&

C*!#! /##/#%y

H$#0(0%"$%(), )$#(0 '(!$8(!$* * &%## /(!(

D$#0#!$() 2')!$)# $#)/& %*/$1 (/ 2#&" %*/$1$!#10(!$* +;

BIFS-A$2

(%!*0 :-7 %*20#&&$* * ($2(!$*

L$#(0 '(!$8(!$*

P0#/$%!$+# %*/$1 $%)'/$1 0*!(!$* (/ *02()&

A/(!$+# (0$!"2#!$% #%*/$1

BIFS S%## F#(!'0#&

8/12/2019 04MPEG4

69/71

MULTIMEDIA SYSTEMS

IREK DEFEE

BIFS S%## F#(!'0#&

A'/$* +$/#* *6

8/12/2019 04MPEG4

70/71

MULTIMEDIA SYSTEMS

IREK DEFEE

MPEG-4 BIFS 6(/ +$%#&

2roa"!ast2roa"!ast

-o++uni!ation-o++uni!ation

-ient

L$+# S*'0%# @ U

2F,2F,

S!*0#/ C*!#!

8/12/2019 04MPEG4

71/71

C*%)'&$*

BIFS 0*+$/#& ( 0$%" !**)$! *0 %*2*&$!$* *MPEG-4 2#/$( $ +#0y )#&$6)# (/ 1##0() (y

BIFS %( 6# 0*$)#/ !* $! 6#&! !"# ()$%(!$*

(0#(

P0*+$/#& ( 1**/ 2$ *

F'%!$*()$!y

C*2)#$!y

C*20#&&$*