thesis reddy

54
7/23/2019 Thesis Reddy http://slidepdf.com/reader/full/thesis-reddy 1/54   A Non-obtrusive Head Mounted  A Non-obtrusive Head Mounted Face Capture System Face Capture System Thesis Committee: Thesis Committee: Dr. George C. Stockman (Main Dr. George C. Stockman (Main  Advisor)  Advisor) Dr. Frank Biocca (Co-Advisor) Dr. Frank Biocca (Co-Advisor) Dr. Charles Owen Dr. Charles Owen Dr. annick !olland ("#ternal Dr. annick !olland ("#ternal Fac$lt Fac$lt Chandan K. Reddy Chandan K. Reddy Masters Thesis !e"ense Masters Thesis !e"ense

Upload: julian1972mx

Post on 19-Feb-2018

227 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Thesis Reddy

7/23/2019 Thesis Reddy

http://slidepdf.com/reader/full/thesis-reddy 1/54

 

 A Non-obtrusive Head Mounted A Non-obtrusive Head Mounted

Face Capture SystemFace Capture System

Thesis Committee:Thesis Committee:Dr. George C. Stockman (MainDr. George C. Stockman (Main

 Advisor) Advisor)

Dr. Frank Biocca (Co-Advisor)Dr. Frank Biocca (Co-Advisor)

Dr. Charles OwenDr. Charles Owen

Dr. annick !olland ("#ternalDr. annick !olland ("#ternalFac$ltFac$lt

Chandan K. Reddy Chandan K. Reddy 

Masters Thesis !e"enseMasters Thesis !e"ense

Page 2: Thesis Reddy

7/23/2019 Thesis Reddy

http://slidepdf.com/reader/full/thesis-reddy 2/54

 

Modes o" CommunicationModes o" Communication

&e#t onl% - e.g. Mail' "lectronic Mail&e#t onl% - e.g. Mail' "lectronic Mail

 oice onl% e.g. &ele*hone oice onl% e.g. &ele*hone

+C camera ,ased conerencing e.g.+C camera ,ased conerencing e.g.

e, came, cam

M$lti-$ser &eleconerencingM$lti-$ser &eleconerencing

&eleconerencing thro$gh irt$al&eleconerencing thro$gh irt$al

"nvironments"nvironments

 A$gmented !ealit% Based A$gmented !ealit% Based

&eleconerencing&eleconerencing

Page 3: Thesis Reddy

7/23/2019 Thesis Reddy

http://slidepdf.com/reader/full/thesis-reddy 3/54

 

#rob$em !e%nition#rob$em !e%nition

Face Ca*t$re S%stem ( FCS )

 irt$al iew S%nthesis

De*th "#traction and /D FaceModeling

0ead Mo$nted +ro1ection Dis*la%s

/D &ele-immersive "nvironments

0igh Bandwidth 2etwork Connections

Page 4: Thesis Reddy

7/23/2019 Thesis Reddy

http://slidepdf.com/reader/full/thesis-reddy 4/54

 

Thesis ContributionsThesis Contributions

Com*lete hardware set$* or the FCS.Com*lete hardware set$* or the FCS.

Camera-mirror *arameter estimation or theCamera-mirror *arameter estimation or the

o*timal con3g$ration o the FCS.o*timal con3g$ration o the FCS.

Generation o 4$alit% rontal videos rom two sideGeneration o 4$alit% rontal videos rom two side

 videos videos

!econstr$ction o te#t$re ma**ed /D ace model!econstr$ction o te#t$re ma**ed /D ace model

rom two side viewsrom two side views "val$ation mechanisms or the generated rontal"val$ation mechanisms or the generated rontal

 views. views.

C

Page 5: Thesis Reddy

7/23/2019 Thesis Reddy

http://slidepdf.com/reader/full/thesis-reddy 5/54

 

&'istin( Face Capture&' st n( Face CaptureSystemsSystems

 Advantages 5 Advantages 5 Freedom or 0ead MovementsFreedom or 0ead Movements

Draw,acks 5Draw,acks 5 O,str$ction o the $ser6s Field o viewO,str$ction o the $ser6s Field o view

Main A**lications 5Main A**lications 5 Character Animation and Mo,ileCharacter Animation and Mo,ile

environmentsenvironments

CourteCourte

sy :sy :

FaceCap)d - aFaceCap)d - aproductproduct

"rom Standard"rom Standard!eviation!eviation

*ptica$ Face Trac+er , a*ptica$ Face Trac+er , aproduct "rom Adaptiveproduct "rom Adaptive

*ptics*ptics

Page 6: Thesis Reddy

7/23/2019 Thesis Reddy

http://slidepdf.com/reader/full/thesis-reddy 6/54

 

&'istin( Face Capture&'istin( Face CaptureSystemsSystems

 Advantages 5 Advantages 5 2o ,$rden or the $ser2o ,$rden or the $ser

Draw,acks 5Draw,acks 5 0ighl% e4$i**ed environments and0ighl% e4$i**ed environments and

restricted head motionrestricted head motion

Main A**lications 5Main A**lications 5 &eleconerencing and&eleconerencing and

CourteCourte

sy:sy:

Nationa$ te$e-immersionNationa$ te$e-immersion

nitiativenitiative

Sea o" CamerasSea o" Cameras/NC Chappe$ Hi$$0/NC Chappe$ Hi$$0

Page 7: Thesis Reddy

7/23/2019 Thesis Reddy

http://slidepdf.com/reader/full/thesis-reddy 7/54 

#roposed Face Capture#roposed Face CaptureSystemSystem

2ovel Face Ca*t$re S%stem that is ,eing2ovel Face Ca*t$re S%stem that is ,eing

develo*ed.develo*ed.

&wo Cameras ca*t$re the corres*onding&wo Cameras ca*t$re the corres*onding

(F. Biocca and . +. !olland' 7&ele*ortal ace-to-ace s%stem8' 

+atent Filed' 9:::.)

Page 8: Thesis Reddy

7/23/2019 Thesis Reddy

http://slidepdf.com/reader/full/thesis-reddy 8/54 

 Advanta(es Advanta(es

;ser6s 3eld o view is $no,str$cted;ser6s 3eld o view is $no,str$cted +orta,le and eas% to $se+orta,le and eas% to $se Gives ver% acc$rate and 4$alit% aceGives ver% acc$rate and 4$alit% ace

imagesimages Can *rocess in real-timeCan *rocess in real-time Sim*le and $ser-riendl% s%stemSim*le and $ser-riendl% s%stem

Static with res*ect to h$man headStatic with res*ect to h$man head Fli**ing the mirror cameras view theFli**ing the mirror cameras view the

$ser6s view*oint$ser6s view*oint

Page 9: Thesis Reddy

7/23/2019 Thesis Reddy

http://slidepdf.com/reader/full/thesis-reddy 9/54 

 App$ications App$ications

Mo,ile "nvironmentsMo,ile "nvironments

Colla,orative ork Colla,orative ork 

M$lti-$ser &eleconerencingM$lti-$ser &eleconerencing Medical AreasMedical Areas

Distance <earningDistance <earning

Gaming and "ntertainment ind$str%Gaming and "ntertainment ind$str% OthersOthers

Page 10: Thesis Reddy

7/23/2019 Thesis Reddy

http://slidepdf.com/reader/full/thesis-reddy 10/54 

System !esi(nSystem !esi(n

Page 11: Thesis Reddy

7/23/2019 Thesis Reddy

http://slidepdf.com/reader/full/thesis-reddy 11/54 

&1uipment Re1uired&1uipment Re1uired

0ardware0ardware 9 li*stick cameras9 li*stick cameras

9 lenses with ocal length9 lenses with ocal length

=9mm=9mm9 mirrors with =.> inch9 mirrors with =.> inchdiameterdiameter

9 Matro# Meteor ??9 Matro# Meteor ??standard cardsstandard cards

 <ighting e4$i*ment<ighting e4$i*ment

 GA to 2&SC Converter GA to 2&SC Converter

 A +ro1ector A +ro1ector A Micro*hone A Micro*hone

SotwareSotware 2etwork 2etwork M?< <?&" @.:M?< <?&" @.:

 is$al St$dio .: is$al St$dio .:

 Ado,e +remiere .: Ado,e +remiere .:

So$nd !ecorderSo$nd !ecorder

  ?nternet 9?nternet 9

2AC /::: M+"G "ncoder2AC /::: M+"G "ncoder

2AC ::: M"G Decoder2AC ::: M"G Decoder

Page 12: Thesis Reddy

7/23/2019 Thesis Reddy

http://slidepdf.com/reader/full/thesis-reddy 12/54 

*ptica$ 2ayout*ptica$ 2ayout

&hree Com*onents to ,e considered&hree Com*onents to ,e considered CameraCamera

MirrorMirror

0$man Face0$man Face

Page 13: Thesis Reddy

7/23/2019 Thesis Reddy

http://slidepdf.com/reader/full/thesis-reddy 13/54 

Speci%cation #arametersSpeci%cation #arameters

CameraCamera Sensing area5 /.9 mm 9. mm (8).Sensing area5 /.9 mm 9. mm (8). +i#el Dimensions5 ?mage sensed is o+i#el Dimensions5 ?mage sensed is o

dimensions @E *i#els. Digitied imagedimensions @E *i#els. Digitied image

sie is /9: 9: d$e to restrictions o thesie is /9: 9: d$e to restrictions o the!AM sie.!AM sie.

Focal <ength(Fc)5 =9 mm (C< =9;M).Focal <ength(Fc)5 =9 mm (C< =9;M). Field o iew (FO)5 =>.9Field o iew (FO)5 =>.9 ::  ==. ==. ::..

Diameter (Dc)5 =9mmDiameter (Dc)5 =9mm Fn$m,er (2c)5 = -achieve ma#im$m lightness.Fn$m,er (2c)5 = -achieve ma#im$m lightness. Minim$m orking Distance (MD)- 9:: mm.Minim$m orking Distance (MD)- 9:: mm. De*th o Field (DOF)5 to ,e estimatedDe*th o Field (DOF)5 to ,e estimated

Page 14: Thesis Reddy

7/23/2019 Thesis Reddy

http://slidepdf.com/reader/full/thesis-reddy 14/54 

Speci%cation #arameters Contd.0Speci%cation #arameters Contd.0

MirrorMirror  Diameter (Dm) I Fn$m,er (2m)Diameter (Dm) I Fn$m,er (2m)  Focal <ength (m)Focal <ength (m)  Magni3cation actor (Mm)Magni3cation actor (Mm)

 !adi$s o c$rvat$re (!m)!adi$s o c$rvat$re (!m) 0$man Face0$man Face

0eight o the ace to ,e ca*t$red (0J 9>:mm)0eight o the ace to ,e ca*t$red (0J 9>:mm) idth o the ace to ,e ca*t$red (J =@> mm)idth o the ace to ,e ca*t$red (J =@> mm)

DistancesDistances Distance ,etween the camera and the mirror.Distance ,etween the camera and the mirror.

(D(DcmcmJ=>:mm)J=>:mm)

Distance ,etween the mirror and the ace. (DDistance ,etween the mirror and the ace. (Dmf mf  

J9::mm)J9::mm)

Page 15: Thesis Reddy

7/23/2019 Thesis Reddy

http://slidepdf.com/reader/full/thesis-reddy 15/54 

Customi3ation o" Cameras andCustomi3ation o" Cameras andMirrorsMirrors

OK-the-shel camerasOK-the-shel cameras C$stomiing camera lens is a tedio$s task C$stomiing camera lens is a tedio$s task  &rade-oK has to ,e made ,etween the 3eld o view&rade-oK has to ,e made ,etween the 3eld o view

and the de*th o 3eldand the de*th o 3eld

Son% DC <S= with =9mm lens is s$ita,le or o$rSon% DC <S= with =9mm lens is s$ita,le or o$ra**licationa**lication

C$stom designed mirrorsC$stom designed mirrors  A *lano-conve# lens with :mm diameter is coated A *lano-conve# lens with :mm diameter is coated

with ,lack on the *lanar side.with ,lack on the *lanar side. &he radi$s o c$rvat$re o the conve# s$race is&he radi$s o c$rvat$re o the conve# s$race is

=>>.: mm.=>>.: mm. &he thickness at the center o the lens is > mm.&he thickness at the center o the lens is > mm. &he thickness at the edge is /.@ mm.&he thickness at the edge is /.@ mm.

Page 16: Thesis Reddy

7/23/2019 Thesis Reddy

http://slidepdf.com/reader/full/thesis-reddy 16/54 

4$oc+ dia(ram o" the system4$oc+ dia(ram o" the system

Page 17: Thesis Reddy

7/23/2019 Thesis Reddy

http://slidepdf.com/reader/full/thesis-reddy 17/54 

&'perimenta$ setup&'perimenta$ setup

Page 18: Thesis Reddy

7/23/2019 Thesis Reddy

http://slidepdf.com/reader/full/thesis-reddy 18/54 

 5irtua$ 5ideo Synthesis 5irtua$ 5ideo Synthesis

Page 19: Thesis Reddy

7/23/2019 Thesis Reddy

http://slidepdf.com/reader/full/thesis-reddy 19/54 

#rob$em Statement#rob$em Statement

6eneratin( virtua$ "ronta$ vie7 "rom t7o side6eneratin( virtua$ "ronta$ vie7 "rom t7o side

 vie7s vie7s

Page 20: Thesis Reddy

7/23/2019 Thesis Reddy

http://slidepdf.com/reader/full/thesis-reddy 20/54 

!ata processin(!ata processin(

&wo s%nchronied videos are ca*t$red in real-&wo s%nchronied videos are ca*t$red in real-time (/: ramesIsec) sim$ltaneo$sl%.time (/: ramesIsec) sim$ltaneo$sl%.

For eKective ca*t$ring and *rocessing' theFor eKective ca*t$ring and *rocessing' thedata is stored in $ncom*ressed ormat.data is stored in $ncom*ressed ormat.

Machine S*eci3cations (<orelei LMachine S*eci3cations (<orelei Lmetla,.cse.ms$.ed$)5metla,.cse.ms$.ed$)5 +enti$m ??? *rocessor+enti$m ??? *rocessor +rocessor s*eed5 @ M0+rocessor s*eed5 @ M0 !AM Sie5 /E MB!AM Sie5 /E MB 0ard Disk write S*eed (*ractical)5 MBIs0ard Disk write S*eed (*ractical)5 MBIs

M?<-<?&" is con3g$red to $se =>: MB o !AMM?<-<?&" is con3g$red to $se =>: MB o !AM

Page 21: Thesis Reddy

7/23/2019 Thesis Reddy

http://slidepdf.com/reader/full/thesis-reddy 21/54

 

!ata processin( Contd.0!ata processin( Contd.0

Sie o = second video /: N /9: N 9: N/Sie o = second video /: N /9: N 9: N/

.> MB .> MB ;sing =>: MB !AM' onl% =: seconds video;sing =>: MB !AM' onl% =: seconds video

rom two cameras can ,e ca*t$redrom two cameras can ,e ca*t$red h% does the *rocessing have to ,e oinePh% does the *rocessing have to ,e oineP

Cali,ration *roced$re is not a$tomaticCali,ration *roced$re is not a$tomatic Disk writing s*eed m$st ,e at least = MBIS.Disk writing s*eed m$st ,e at least = MBIS.

&o ca*t$re 9 videos o : N E: resol$tion' the&o ca*t$re 9 videos o : N E: resol$tion' theDisk writing s*eed m$st ,e at least > MBIS PPPDisk writing s*eed m$st ,e at least > MBIS PPP

Page 22: Thesis Reddy

7/23/2019 Thesis Reddy

http://slidepdf.com/reader/full/thesis-reddy 22/54

 

Structured 2i(htStructured 2i(httechni1uetechni1ue

 A s4$are grid in the A s4$are grid in the

rontal view a**ears as arontal view a**ears as a

4$adrilateral (with c$rved4$adrilateral (with c$rved

edges) in the real sideedges) in the real side

+ro1ecting a grid on+ro1ecting a grid on

the rontal view othe rontal view o

the acethe ace

Page 23: Thesis Reddy

7/23/2019 Thesis Reddy

http://slidepdf.com/reader/full/thesis-reddy 23/54

 

Co$or 4a$ancin(Co$or 4a$ancin(

Hardware based approach White balancing of the cameras

Why this is more robust ? – why not softwarebased ? There is no change in the input camera

Better handling of varying lighting conditions

No pre - knowledge of the skin color is reuired

No additional overhead

!ts enough if both cameras are color balanced relativel

Page 24: Thesis Reddy

7/23/2019 Thesis Reddy

http://slidepdf.com/reader/full/thesis-reddy 24/54

 

 Left Calibration Left Calibration

 Face Image Face Image

Transformation TablesTransformation Tables

 Projector  Projector 

 Right Calibration Right Calibration

 Face Image Face Image

*8-$ine Ca$ibration Sta(e*8-$ine Ca$ibration Sta(e

Page 25: Thesis Reddy

7/23/2019 Thesis Reddy

http://slidepdf.com/reader/full/thesis-reddy 25/54

 

TransformationTransformation

TablesTables

 Right Right

 Face Image Face Image

 Left Left

 Face Image Face Image

Right Warped Right Warped 

Face ImageFace Image

Left Warped Left Warped 

Face ImageFace Image

Mosaiced Face ImageMosaiced Face Image

*perationa$ Sta(e*perationa$ Sta(e

Page 26: Thesis Reddy

7/23/2019 Thesis Reddy

http://slidepdf.com/reader/full/thesis-reddy 26/54

 

 5irtua$ video synthesis 5irtua$ video synthesisCa$ibration phase0Ca$ibration phase0

5irtua$ video synthesis5irtua$ video synthesis

Page 27: Thesis Reddy

7/23/2019 Thesis Reddy

http://slidepdf.com/reader/full/thesis-reddy 27/54

 

 5irtua$ video synthesis 5irtua$ video synthesiscontd.0contd.0

Page 28: Thesis Reddy

7/23/2019 Thesis Reddy

http://slidepdf.com/reader/full/thesis-reddy 28/54

 

 5irtua$ Fronta$ 5ideo 5irtua$ Fronta$ 5ideo

Page 29: Thesis Reddy

7/23/2019 Thesis Reddy

http://slidepdf.com/reader/full/thesis-reddy 29/54

 

Comparison o" the Fronta$ 5ie7sComparison o" the Fronta$ 5ie7s

First ro7 , 5irtua$ "ronta$ vie7sFirst ro7 , 5irtua$ "ronta$ vie7s

Second ro7 , *ri(ina$ "ronta$ vie7sSecond ro7 , *ri(ina$ "ronta$ vie7s

id h i i

Page 30: Thesis Reddy

7/23/2019 Thesis Reddy

http://slidepdf.com/reader/full/thesis-reddy 30/54

 

 5ideo Synchroni3ation &ye 5ideo Synchroni3ation &yeb$in+in(0b$in+in(0

First ro7 , 5irtua$ "ronta$ vie7sFirst ro7 , 5irtua$ "ronta$ vie7s

Second ro7 , *ri(ina$ "ronta$ vie7sSecond ro7 , *ri(ina$ "ronta$ vie7s

h h d

Page 31: Thesis Reddy

7/23/2019 Thesis Reddy

http://slidepdf.com/reader/full/thesis-reddy 31/54

 

Face !ata throu(h HeadFace !ata throu(h HeadMounted SystemMounted System

Page 32: Thesis Reddy

7/23/2019 Thesis Reddy

http://slidepdf.com/reader/full/thesis-reddy 32/54

 

)! Face Mode$)! Face Mode$

Page 33: Thesis Reddy

7/23/2019 Thesis Reddy

http://slidepdf.com/reader/full/thesis-reddy 33/54

 

Coordinate SystemsCoordinate Systems

 There are "ve coordinate systems in our application

World #oordinate $ystem %W#$&

'ace #oordinate $ystem %'#$& (eft #amera #oordinate system %(##$&

)ight #amera #oordinate system %)##$&

*ro+ector #oordinate $ystem %*#$&

Page 34: Thesis Reddy

7/23/2019 Thesis Reddy

http://slidepdf.com/reader/full/thesis-reddy 34/54

 

Camera Ca$ibrationCamera Ca$ibration

Conversion rom /D world coordinates to 9DConversion rom /D world coordinates to 9Dcamera coordinates - +ers*ective &ransormationcamera coordinates - +ers*ective &ransormation

ModelModel

ss << ++rr

ss << ++cc

ss

CC====  CC=9=9  CC=/=/ 

CC== 

CC9=9=  CC9999  CC9/9/ 

CC99 

CC/=/=  CC/9/9  CC////  == 

        9        9

ss ++ % %

ss ++

==

ss ++##

"liminating the scale actor"liminating the scale actor

  $$ 1 1  (c (c====  c c/=/= $$ 1 1) #) # 1 1 Q (cQ (c=9=9  c c/9/9 $$ 1 1) %) % 1 1 Q (cQ (c=/=/  c c////

$$ 1 1) )  1 1 Q cQ c==

  v v 1 1  (c (c9=9=  c c/=/= v1) #1 Q (c v1) #1 Q (c9999  c c/9/9 v1) %1 Q (c v1) %1 Q (c9/9/ 

Page 35: Thesis Reddy

7/23/2019 Thesis Reddy

http://slidepdf.com/reader/full/thesis-reddy 35/54

 

Ca$ibration sphereCa$ibration sphere

 A s*here can ,e $sed or Cali,ration A s*here can ,e $sed or Cali,ration

Cali,ration *oints on the s*here areCali,ration *oints on the s*here are

chosen in s$ch a wa% that thechosen in s$ch a wa% that the

 Aim$thal angle is varied in ste*s o Aim$thal angle is varied in ste*s o

>>oo

  +olar angle is varied in ste*s o /:+olar angle is varied in ste*s o /:oo

&he location o these cali,ration&he location o these cali,ration

*oints is known in the /D coordinate*oints is known in the /D coordinate

S%stem with res*ect to the origin oS%stem with res*ect to the origin o

the s*herethe s*here &he origin o the s*here de3nes the&he origin o the s*here de3nes the

origin o the orld Coordinateorigin o the orld Coordinate

S%stemS%stem

Page 36: Thesis Reddy

7/23/2019 Thesis Reddy

http://slidepdf.com/reader/full/thesis-reddy 36/54

 

#roector Ca$ibration#roector Ca$ibration

Similar to Camera Cali,rationSimilar to Camera Cali,ration 9D image coordinates can not ,e o,tained9D image coordinates can not ,e o,tained

directl% rom a 9D image.directl% rom a 9D image.

 A 7Blank ?mage8 is *ro1ected onto the s*here A 7Blank ?mage8 is *ro1ected onto the s*here

&he 9D coordinates o the cali,ration *oints&he 9D coordinates o the cali,ration *ointson the *ro1ected image are notedon the *ro1ected image are noted

More *oints can ,e seen rom the *ro1ector6sMore *oints can ,e seen rom the *ro1ector6s*oint o view some *oints are common to*oint o view some *oints are common to,oth camera views,oth camera views

!es$lts a**ear to have slightl% more errors!es$lts a**ear to have slightl% more errorswhen com*ared to the camera cali,rationwhen com*ared to the camera cali,ration

Page 37: Thesis Reddy

7/23/2019 Thesis Reddy

http://slidepdf.com/reader/full/thesis-reddy 37/54

 

)! Face Mode$ Construction)! Face Mode$ Construction

h%Ph%P &o o,tain diKerent views o the ace&o o,tain diKerent views o the ace

&o generate the stereo *air to view it in&o generate the stereo *air to view it in

the 0M+Dthe 0M+D

Ste*s re4$iredSte*s re4$ired

Com*$tation o /D <ocationsCom*$tation o /D <ocations C$stomiation o /D ModelC$stomiation o /D Model

&e#t$re Ma**ing&e#t$re Ma**ing

Page 38: Thesis Reddy

7/23/2019 Thesis Reddy

http://slidepdf.com/reader/full/thesis-reddy 38/54

 

Computation o" )!Computation o" )!

pointspoints /d *oint estimation $sing stereo/d *oint estimation $sing stereo

Stereo ,etween two cameras is not *ossi,leStereo ,etween two cameras is not *ossi,le,eca$se o the occl$sion ,% the acial,eca$se o the occl$sion ,% the acialeat$reseat$res

0ence two stereo *air com*$tations0ence two stereo *air com*$tations <et camera and *ro1ector<et camera and *ro1ector !ight camera and *ro1ector!ight camera and *ro1ector

;sing stereo' com*$te /D *oints o;sing stereo' com*$te /D *oints o*rominent acial eat$re *oints in FCS*rominent acial eat$re *oints in FCS

Page 39: Thesis Reddy

7/23/2019 Thesis Reddy

http://slidepdf.com/reader/full/thesis-reddy 39/54

 

)! 6eneric Face Mode$)! 6eneric Face Mode$

 A generic ace model with /> vertices and A generic ace model with /> vertices andE=E trianglesE=E triangles

<et5 ront view and !ight5 side view<et5 ront view and !ight5 side view

Page 40: Thesis Reddy

7/23/2019 Thesis Reddy

http://slidepdf.com/reader/full/thesis-reddy 40/54

 

Te'ture Mapped )! FaceTe'ture Mapped )! Face

Page 41: Thesis Reddy

7/23/2019 Thesis Reddy

http://slidepdf.com/reader/full/thesis-reddy 41/54

 

&va$uation&va$uation

Page 42: Thesis Reddy

7/23/2019 Thesis Reddy

http://slidepdf.com/reader/full/thesis-reddy 42/54

 

&va$uation Schemes&va$uation Schemes

"val$ation o acial e#*ressions and is not"val$ation o acial e#*ressions and is notst$died e#tensivel% in literat$rest$died e#tensivel% in literat$re

"val$ation can ,e done or acial alignment'"val$ation can ,e done or acial alignment'

ace recognition or static imagesace recognition or static images

<i* and e%e movements in a d%namic event<i* and e%e movements in a d%namic event

+erce*t$al 4$alit% 0ow are the moods+erce*t$al 4$alit% 0ow are the moods

conve%edPconve%edP

&wo t%*es o eval$ation&wo t%*es o eval$ation O,1ective eval$ationO,1ective eval$ation

S$,1ective eval$ationS$,1ective eval$ation

Page 43: Thesis Reddy

7/23/2019 Thesis Reddy

http://slidepdf.com/reader/full/thesis-reddy 43/54

 

*bective &va$uation*bective &va$uation

&heoretical "val$ation&heoretical "val$ation 2o h$man eed,ack re4$ired2o h$man eed,ack re4$ired &his eval$ation can give $s a meas$re&his eval$ation can give $s a meas$re

o o  Face recognitionFace recognition Face alignmentFace alignment Facial movementsFacial movements

Methods a**liedMethods a**lied 2ormalied cross correlation2ormalied cross correlation "$clidean distance meas$res"$clidean distance meas$res

& $ ti & $ ti

Page 44: Thesis Reddy

7/23/2019 Thesis Reddy

http://slidepdf.com/reader/full/thesis-reddy 44/54

 

&va$uation ma(es&va$uation ma(es

; "rames 7ere considered "or obective; "rames 7ere considered "or obectiveeva$uationeva$uation

First ro7 , virtua$ "ronta$ vie7sFirst ro7 , virtua$ "ronta$ vie7s

Second ro7 , ori(ina$ "ronta$ vie7sSecond ro7 , ori(ina$ "ronta$ vie7s

Page 45: Thesis Reddy

7/23/2019 Thesis Reddy

http://slidepdf.com/reader/full/thesis-reddy 45/54

 

Norma$i3ed Cross-Norma$i3ed Cross-

Corre$ationCorre$ation

!egions considered or normalied cross-!egions considered or normalied cross-correlationcorrelation

( <et5 !eal image !ight5 irt$al image)( <et5 !eal image !ight5 irt$al image)

Page 46: Thesis Reddy

7/23/2019 Thesis Reddy

http://slidepdf.com/reader/full/thesis-reddy 46/54

 

Norma$i3ed Cross-Norma$i3ed Cross-

Corre$ationCorre$ation

<et ,e the virt$al image and ! ,e the real image<et ,e the virt$al image and ! ,e the real image <et w ,e the width and h ,e the height o the images<et w ,e the width and h ,e the height o the images

&he 2ormalied Cross-correlation ,etween the two images and ! is given ,%&he 2ormalied Cross-correlation ,etween the two images and ! is given ,%

wherewhere

Page 47: Thesis Reddy

7/23/2019 Thesis Reddy

http://slidepdf.com/reader/full/thesis-reddy 47/54

 

Norma$i3ed Cross-Norma$i3ed Cross-

Corre$ationCorre$ation ideo ideo

FramFrameses

<et<ete%ee%e

!ight!ighte%ee%e

Mo$tMo$thh

"%es"%esQQ

Mo$tMo$t

hh

Com*leCom*lete acete ace

FramFrame=e=

:.EE:.EE :.E@:.E@ :./:./ :.E:.E :.E:.E

FramFrame9e9

:.:. :.@9:.@9 :.E>:.E> :.@E:.@E :.E>:.E>

FramFrame/e/

:.:. :.@:.@ :.9:.9 :.@E:.@E :.E:.E

& $id !i t& c$idean !istance

Page 48: Thesis Reddy

7/23/2019 Thesis Reddy

http://slidepdf.com/reader/full/thesis-reddy 48/54

 

&uc$idean !istance&uc$idean !istancemeasuresmeasures

"$clidean distance ,etween two *oints i and 1 is given ,%"$clidean distance ,etween two *oints i and 1 is given ,%

<et !i1 ,e the e$clidean distance ,etween two *oints i and 1 in<et !i1 ,e the e$clidean distance ,etween two *oints i and 1 in

the real imagethe real image

<et i1 ,e the e$clidean distance ,etween two *oints i and 1 in<et i1 ,e the e$clidean distance ,etween two *oints i and 1 in

the virt$al imagethe virt$al image

!i 9 < Ri - 5i <!i 9 < Ri - 5i <

Page 49: Thesis Reddy

7/23/2019 Thesis Reddy

http://slidepdf.com/reader/full/thesis-reddy 49/54

 

&uc$idean !istance&uc$idean !istance

measuresmeasuresramesrames DDa a  DD, ,  DDc c  DDcgcg DDdgdg DDegeg "rror"rror

FrameFrame==

9.:9.:::

:.E:.E::

.=.=>>

/./.

9.9.>>

/./.

9.E:9.E:

FrameFrame99

:.>:.>

/.:/.:::

:.@:.@

..==

:.:.//

:.E:.E::

=.@=.@

FrameFrame

//

=.E=.E

EE

/.E/.E

.9.9

././

9.9.

EE

=.E=.E

//

/.=/.=

FrameFrame

=.:=.:

9.9.@@

9.=9.=::

././//

/.:/.:==

.:.:EE

/.//./

FrameFrame =.=. 9.99.9 >.>>.> .. =.9=.9 =.=. 9.99.9

Page 50: Thesis Reddy

7/23/2019 Thesis Reddy

http://slidepdf.com/reader/full/thesis-reddy 50/54

 

Subective &va$uationSubective &va$uation

"val$ates the h$man *erce*tion"val$ates the h$man *erce*tion Meas$rement o 4$alit% o a talking aceMeas$rement o 4$alit% o a talking ace Factors that might aKectFactors that might aKect

R$alit% o the videoR$alit% o the video Facial movements and e#*ressionsFacial movements and e#*ressions S%nchroniation o the two halves o the aceS%nchroniation o the two halves o the ace Color and &e#t$re o the aceColor and &e#t$re o the ace R$alit% o a$dioR$alit% o a$dio S%nchroniation o a$dioS%nchroniation o a$dio

 A *reliminar% st$d% has ,een made to A *reliminar% st$d% has ,een made toassess the 4$alit% o the generated videosassess the 4$alit% o the generated videos

C $ iC $ i dd F tF t

Page 51: Thesis Reddy

7/23/2019 Thesis Reddy

http://slidepdf.com/reader/full/thesis-reddy 51/54

 

Conc$usionConc$usion andand FutureFuture =or+  =or+ 

 5irtua$

Fronta$ ma(e

Te'ture Mapped)! Face Mode$

 5irtua$

Fronta$ 5ideo

)! Facia$ Animation

Conc$usion Future =or+ 

Page 52: Thesis Reddy

7/23/2019 Thesis Reddy

http://slidepdf.com/reader/full/thesis-reddy 52/54

 

Summary Summary 

Design and im*lementation o a novelDesign and im*lementation o a novel

Face Ca*t$re S%stemFace Ca*t$re S%stem

Generation o virt$al rontal view romGeneration o virt$al rontal view rom

two side views in a video se4$encetwo side views in a video se4$ence "#traction o de*th inormation $sing"#traction o de*th inormation $sing

stereo methodstereo method

&e#t$re ma**ed /D ace model&e#t$re ma**ed /D ace modelgenerationgeneration

"val$ation o virt$al rontal videos"val$ation o virt$al rontal videos

Page 53: Thesis Reddy

7/23/2019 Thesis Reddy

http://slidepdf.com/reader/full/thesis-reddy 53/54

 

Future =or+ Future =or+ 

Online *rocessing in real-timeOnline *rocessing in real-time

 A$tomatic cali,ration A$tomatic cali,ration

/D acial animation/D acial animation S$,1ective "val$ation o the virt$al rontal videosS$,1ective "val$ation o the virt$al rontal videos

Data com*ression while *rocessing andData com*ression while *rocessing and

transmissiontransmission

C$stomiation o camera lensesC$stomiation o camera lenses ?ntegration with a 0ead Mo$nted +ro1ection?ntegration with a 0ead Mo$nted +ro1ection

Dis*la%Dis*la%

Page 54: Thesis Reddy

7/23/2019 Thesis Reddy

http://slidepdf.com/reader/full/thesis-reddy 54/54

Than+ >ouThan+ >ou

Do$,ts'Do$,ts'

R$eriesR$eries 

S$ggestionsS$ggestions