insight : recognizing humans without face recognition
DESCRIPTION
InSight : Recognizing Humans without Face Recognition. He Wang , Xuan Bao , Romit Roy Choudhury , Srihari Nelakuditi. Motivation – Application Scenarios. s hare a ride to airport. Elle. Bret. Bob. 2. Overview – Self-Fingerprints. Cloud. Self-Fingerprints. Bob. Bret. Elle. - PowerPoint PPT PresentationTRANSCRIPT
InSight: Recognizing Humans without Face Recognition
He Wang, Xuan Bao, Romit Roy Choudhury, Srihari Nelakuditi
2
Motivation – Application Scenarios
2
share a ride
to airpor
t
BobElle
Bret
3
Overview – Self-FingerprintsCloud
Bret Bob John Elle
Self-Fingerprints
4
Overview – Recognition Cloud
Glass
5
Overview – Recognition Cloud
Glass
6
Overview – Recognition Cloud
Glass
7
Challenges
• Perspectives are different
• Clothes have wrinkles
• Lighting conditions change
8
Extracting Fingerprints
Colors
Patterns
Spatiograms
Wavelets
Extracting Fingerprints – Colors
9
RGB HSV
Spatiograms
Color Conversion
Extracting Fingerprints – Colors
RGB HSV
10
color histogram
spatial distribution
Spatiograms
11
Extracting Fingerprints – PatternsWavelets
Wavelet sub-bands: vertical, horizontal and diagonal dimensions.
12
Fingerprint Matching Cloud
Glass
Matching Spatiograms
S = {n’, µ’, σ’}
S = {n, µ, σ}cloud
glass
Similarity = color histograms spatial distributions
13
Fingerprint Matching Cloud
Glass
Matching Wavelets
Bagged Decision Tree (BDT)
W = {f1, f2, f3,…}
W = {f1’, f2’, f3’,…}
cloud
glass
14
Evaluation Setting
• PivotHead glass captured users from the front.
• 15 users was dressed in their regular attires.
• Users actively used their smartphones.
• Phone opportunistically took “profile” pictures of the user.
Evaluation - Matching Color Spatiograms
front
Evaluation - Matching Wavelets of Patterns
front
17
Evaluation – Combining Colors and PatternsEvaluation –
front
18
Evaluation – Performance with Self-Fingerprints
• Matching front view
front
19
Self-fingerprints may not be Sufficient
• Clothes’ difference is not captured when clothes are similar?
• Clothes have different colors/patterns at the back?
Bret
PaulDan
Bret
Refining the Self-Fingerprint (Similar Clothes)
21
Cloud
Glass
DanBob John
refining self-fingerprint
Refining the Self-Fingerprint (Similar Clothes)
22
Dan Paul
Cloud
Glass
Can recognize after refinement!
Refining the Self-Fingerprint
23
Dan
Dan = {F1}
Dan = {F1, F2}
Dan = {F1, F2, F3}
F1 F2
F2 F3
24
Refining the Self-Fingerprint (Back View)
Glass
Cloud
refining back view fingerprint
Bret
25
Evaluation – Performance with Self-Fingerprints
• Matching back view
back
26
Evaluation – Performance with Refined Fingerprints
• Matching back view after refining fingerprints
back
27
Discussion
• Privacy of opportunistic pictureso User consent before uploading
• Overlapping users in viewo Fingerprint refinement helps
• Incremental deployment with some non-participantso More time and mobility help
• Cloud vs p2po Different trade-offs
28
Conclusion
• Colors and patterns on clothes help fingerprint humans.
• Preliminary evaluation with 15 people provides promising results.
• Other type of fingerprints exists such as motion.
29
29
Looking for interns
Anyone for beer after this talk?
New Primitive for Broadcasting to Visible Vicinity.
Questions, Comments?Thank You