large-scale, real-world face recognition in movie trailers alan wright

Large-Scale, Real-World Face Recognition in Movie

Trailers

Alan Wright

Plan of Attack

• Extract Facial Tracks from videos• Extract the features from the facial tracks.• Build framework to load and test data.• Begin with baseline testing (Sparse, min,

mean, etc)• Algorithm development…

Dataset

• 2010 Movie Trailerso 305 videoso Avg. of 625 faces per trailero Avg. of 23 face trackers per trailer

• PubFigo 58,787 photoso 200 people o Avg. of 295 photos per person.

Dataset

Facetrack: Dictionary:

Face Tracks

• We have:

• Instead of:

Linear Combination Approach

+ x2 + x3

+ x4 + x5 + x6

+ x7 + x8 + x9

Test Image

= x1

Training Images

Linear Combination Approach

y

Testing

A=

=

Training

x

Coefficients

Linear Combination & Face Tracks

Linear Combination & Face Tracks

Linear Combination & Face Tracks

Linear Combination & Face Tracks

Linear Combination & Face Tracks

Linear Combination & Face Tracks

Linear Combination & Face tracks

Linear Combination & Face Tracks

Linear Combination & Face Tracks

Linear Combination & Face Tracks

Sylvester Stallone

Variations

• Blurry images within face tracks

• Unknowns

• Facial Expressions

Variations

• Pose

• Occlusions

Preliminary Testing

• Ran SRC facial recognition on extracted facial tracks from Date Night.

• Choose the name that has the most “votes” in the track.

• Looking strictly at Steve Carell and Tina Fey tracks…

Results

• Correctly assessed 15 out of 34 face tracks correctly (44.12% accuracy).

• 6 out of 18 Tina Fey face tracks (33.33% accuracy).

• 9 out of 16 Steve Carell face tracks (56.25% accuracy).

Correct Result

• Track 7- IDed CORRECTLY as: Tina Feyo Alyssa Milano had 2 voteso Drew Barrymore had 4 voteso Jennifer Love Hewitt had 1 voteso Salma Hayek had 3 voteso Tina Fey had 22 votes

• 32 frames in this track

• 68.75% identified as Tina Fey

Correct Result

Correct Result

• Good facial crop.

• Many frames.

• Good lighting

• Non blurry

• Generally straightforward pose

Incorrect Result (1)

• Track 9- IDed as: Gael Garcia Bernalo Clive Owen had 2 voteso Cristiano Ronaldo had 1 voteso Gael Garcia Bernal had 3 voteso Steve Carell had 2 votes

Incorrect Result (1)

Incorrect Result (1)

• Blurry

• Not a great crop

• Only 7 frames

Incorrect Result (2)

• Track 26 - IDed as: Nicole Kidmano David Duchovny had 2 voteso Lucy Liu had 1 voteso Michael Bloomberg had 1 voteso Nicole Kidman had 3 voteso Salma Hayek had 1 votes

Incorrect Result (2)

Incorrect Result (2)

• Profile view

• Poor crop

• Only 8 frames.

Incorrect Result (3)

• track 40 - IDed as: Keira Knightleyo Adriana Lima had 6 voteso Cindy Crawford had 8 voteso Eliot Spitzer had 1 voteso Eva Mendes had 1 voteso Gillian Anderson had 1 voteso Keira Knightley had 18 voteso Meg Ryan had 3 voteso Minnie Driver had 1 voteso Salma Hayek had 3 voteso Tina Fey had 15 votes

Incorrect Result (3)

Incorrect Result (3)

• Dark lighting

• Pose variation

• Motion blur toward the end

• 57 frames

What’s next?

• Continue to look at the coefficient vector.

• Look into confidence of each choice (SCI and residual error)

• Apply different facial recognition algorithms.

• Best way to choose a name for each track? (Mode, strongest confidence, etc.)