Large-scale, Real-world facial recognition in movie
trailersAlan Wright
Presentation 7
recap of last week
Cast selector3
Cast selector
• Retrieves cast list from Rotten Tomatoes using their API.
• Ignore tracks we don’t want.
• Type custom names.
• Allows two people to simultaneously label tracks and no labeling will be repeated.
4
Cast selector
• All 2400+ tracks have now been labeled with the correct faces.
• Faces not in PubFig were still labeled.
• Easily label more tracks if new trailers are added.
• If faces are added to PubFig, the labeling will not need to be redone.
5
Labeling results
• 635 Unknown tracks
• 712 PubFig tracks
• 1113 labeled tracks (faces not in PubFig)
• 4 ignored tracks.
Labeling results
PubFig Ids
# of
labels
Labeling results
• Katherine Heigl was labeled the most with 51 tracks.
• Each PubFig face (in the trailers) has an average of 12 tracks.
Labeling Results
• The most labeled face, not in PubFig, was Edward Norton with 53 tracks.
• 218 faces were labeled, but not in PubFig.
• Average of 5 tracks per face.
New pr curveAccurate with labeled faces
How can we add more faces?
• Look at the distribution of faces that aren’t in PubFig
• Pick a threshold that will give us faces that appear often, and extend PubFig.
• Note: We want a good threshold because the average number of tracks per person (not in PubFig) is 5.
Track distributionFaces not in PubFig
# of
labels
Face IDs
Track distributionFaces not in PubFig
# of
labels
Face IDs
Threshold of 20
New faces
• Choosing a threshold of 20 or more tracks gives us 9 new people:
1. Edward Norton - 53
2. Amanda Seyfried - 37
3. Jason Bateman - 34
4. Hilary Swank - 31
5. Paul Rudd - 30
6. Robert De Niro - 27 Leelee Sobiesk - 26 Dwayne Johnson - 24 Johnny Depp - 24
new faces
• Downloaded images for these 9 people and added them to PubFig. (eye aligned, extracted features, etc)
new labeling distribution
• 635 Unknown tracks
• 998 Extended PubFig tracks
• 827 labeled tracks (faces not in PubFig)
• 4 ignored tracks.
What’s next?• Run over new supplemented data (Server will be up this afternoon)
• Implement other voting methods:
1.Logarithmic pooling
2.Borda Count
• Look at other ways to create a single confidence score for non-avg SRC and SVM methods
• Experiment with different parameters: crop, pca dimensions, features, voting
