contributions a people dataset of 8035 images. three layer attribute classification framework using...
TRANSCRIPT
![Page 1: Contributions A people dataset of 8035 images. Three layer attribute classification framework using poselets. 1 2](https://reader035.vdocuments.us/reader035/viewer/2022070307/551ae1715503466b6a8b5c1b/html5/thumbnails/1.jpg)
![Page 2: Contributions A people dataset of 8035 images. Three layer attribute classification framework using poselets. 1 2](https://reader035.vdocuments.us/reader035/viewer/2022070307/551ae1715503466b6a8b5c1b/html5/thumbnails/2.jpg)
Contributions
A people dataset of 8035 images. Three layer attribute classification framework using poselets.
1 2
![Page 3: Contributions A people dataset of 8035 images. Three layer attribute classification framework using poselets. 1 2](https://reader035.vdocuments.us/reader035/viewer/2022070307/551ae1715503466b6a8b5c1b/html5/thumbnails/3.jpg)
People Dataset
H3D (Humans in 3D) Pascal VOC 2010 (trn+val)
8035images
TRN: 2003 VAL: 2011 TEST: 4022
9 different attributes in total.
At least 2 attributes per image
Agreement of 4 out of 5
1
![Page 4: Contributions A people dataset of 8035 images. Three layer attribute classification framework using poselets. 1 2](https://reader035.vdocuments.us/reader035/viewer/2022070307/551ae1715503466b6a8b5c1b/html5/thumbnails/4.jpg)
Attribute classification using poselets2
![Page 5: Contributions A people dataset of 8035 images. Three layer attribute classification framework using poselets. 1 2](https://reader035.vdocuments.us/reader035/viewer/2022070307/551ae1715503466b6a8b5c1b/html5/thumbnails/5.jpg)
What is a What is a Poselet Poselet ??
Poselets capture part of the pose from a given viewpointPoselets capture part of the pose from a given viewpoint[Bourdev & Malik, ICCV09][Bourdev & Malik, ICCV09]
![Page 6: Contributions A people dataset of 8035 images. Three layer attribute classification framework using poselets. 1 2](https://reader035.vdocuments.us/reader035/viewer/2022070307/551ae1715503466b6a8b5c1b/html5/thumbnails/6.jpg)
PoseletsPoselets
Examples may differ visually but have common semanticsExamples may differ visually but have common semantics[Bourdev & Malik, ICCV09][Bourdev & Malik, ICCV09]
![Page 7: Contributions A people dataset of 8035 images. Three layer attribute classification framework using poselets. 1 2](https://reader035.vdocuments.us/reader035/viewer/2022070307/551ae1715503466b6a8b5c1b/html5/thumbnails/7.jpg)
PoseletsPoselets
But how are we going to create training examples of poselets?But how are we going to create training examples of poselets?
![Page 8: Contributions A people dataset of 8035 images. Three layer attribute classification framework using poselets. 1 2](https://reader035.vdocuments.us/reader035/viewer/2022070307/551ae1715503466b6a8b5c1b/html5/thumbnails/8.jpg)
How do we train a poselet for a given How do we train a poselet for a given pose configuration?pose configuration?
![Page 9: Contributions A people dataset of 8035 images. Three layer attribute classification framework using poselets. 1 2](https://reader035.vdocuments.us/reader035/viewer/2022070307/551ae1715503466b6a8b5c1b/html5/thumbnails/9.jpg)
Finding correspondences at training timeFinding correspondences at training time
Given part of a human poseGiven part of a human pose How do we find a similar pose How do we find a similar pose configuration in the training set?configuration in the training set?
![Page 10: Contributions A people dataset of 8035 images. Three layer attribute classification framework using poselets. 1 2](https://reader035.vdocuments.us/reader035/viewer/2022070307/551ae1715503466b6a8b5c1b/html5/thumbnails/10.jpg)
We use keypoints to annotate the joints, eyes, nose, etc. of peopleWe use keypoints to annotate the joints, eyes, nose, etc. of people
Left Hip
Left Shoulder
Finding correspondences at training timeFinding correspondences at training time
![Page 11: Contributions A people dataset of 8035 images. Three layer attribute classification framework using poselets. 1 2](https://reader035.vdocuments.us/reader035/viewer/2022070307/551ae1715503466b6a8b5c1b/html5/thumbnails/11.jpg)
Residual ErrorResidual Error
Finding correspondences at training timeFinding correspondences at training time
![Page 12: Contributions A people dataset of 8035 images. Three layer attribute classification framework using poselets. 1 2](https://reader035.vdocuments.us/reader035/viewer/2022070307/551ae1715503466b6a8b5c1b/html5/thumbnails/12.jpg)
Training poselet classifiersTraining poselet classifiers
ResiduResidual al Error:Error:
0.10.155
0.20.200
0.10.100
0.30.355
0.10.155
0.80.855
1.1. Given a seed patchGiven a seed patch2.2. Find the closest patch for every other personFind the closest patch for every other person3.3. Sort them by residual errorSort them by residual error4.4. Threshold themThreshold them
![Page 13: Contributions A people dataset of 8035 images. Three layer attribute classification framework using poselets. 1 2](https://reader035.vdocuments.us/reader035/viewer/2022070307/551ae1715503466b6a8b5c1b/html5/thumbnails/13.jpg)
Training poselet classifiersTraining poselet classifiers
1.1. Given a seed patchGiven a seed patch2.2. Find the closest patch for every other personFind the closest patch for every other person3.3. Sort them by residual errorSort them by residual error4.4. Threshold themThreshold them5.5. Use them as positive training examples to Use them as positive training examples to
train a linear SVM with HOG featurestrain a linear SVM with HOG features
![Page 14: Contributions A people dataset of 8035 images. Three layer attribute classification framework using poselets. 1 2](https://reader035.vdocuments.us/reader035/viewer/2022070307/551ae1715503466b6a8b5c1b/html5/thumbnails/14.jpg)
Which poselets should we train?Which poselets should we train?
• Choose thousands of random windows, generate poselet candidates, train linear SVMs
• Select a small set of poselets that are:– Individually effective– Complementary
![Page 15: Contributions A people dataset of 8035 images. Three layer attribute classification framework using poselets. 1 2](https://reader035.vdocuments.us/reader035/viewer/2022070307/551ae1715503466b6a8b5c1b/html5/thumbnails/15.jpg)
Some Poselets
![Page 16: Contributions A people dataset of 8035 images. Three layer attribute classification framework using poselets. 1 2](https://reader035.vdocuments.us/reader035/viewer/2022070307/551ae1715503466b6a8b5c1b/html5/thumbnails/16.jpg)
Attribute classification using poselets2
![Page 17: Contributions A people dataset of 8035 images. Three layer attribute classification framework using poselets. 1 2](https://reader035.vdocuments.us/reader035/viewer/2022070307/551ae1715503466b6a8b5c1b/html5/thumbnails/17.jpg)
Features• HOGs at two levels (~2K-4K features)
– 16 x 16– 32 x 32
• Color Histograms in H,S,B (30 features)– 10 bins for H, S and B
• Skin classifier output (3 features)– GMM with 5 components– Fraction of skin pixels– hands-skin, legs-skin, neck skin
![Page 18: Contributions A people dataset of 8035 images. Three layer attribute classification framework using poselets. 1 2](https://reader035.vdocuments.us/reader035/viewer/2022070307/551ae1715503466b6a8b5c1b/html5/thumbnails/18.jpg)
Poselet-level Attribute Classifiers
![Page 19: Contributions A people dataset of 8035 images. Three layer attribute classification framework using poselets. 1 2](https://reader035.vdocuments.us/reader035/viewer/2022070307/551ae1715503466b6a8b5c1b/html5/thumbnails/19.jpg)
Person-level Attribute Classifiers
![Page 20: Contributions A people dataset of 8035 images. Three layer attribute classification framework using poselets. 1 2](https://reader035.vdocuments.us/reader035/viewer/2022070307/551ae1715503466b6a8b5c1b/html5/thumbnails/20.jpg)
Context-level Attribute Classifiers
![Page 21: Contributions A people dataset of 8035 images. Three layer attribute classification framework using poselets. 1 2](https://reader035.vdocuments.us/reader035/viewer/2022070307/551ae1715503466b6a8b5c1b/html5/thumbnails/21.jpg)
Results
![Page 22: Contributions A people dataset of 8035 images. Three layer attribute classification framework using poselets. 1 2](https://reader035.vdocuments.us/reader035/viewer/2022070307/551ae1715503466b6a8b5c1b/html5/thumbnails/22.jpg)
Results
![Page 23: Contributions A people dataset of 8035 images. Three layer attribute classification framework using poselets. 1 2](https://reader035.vdocuments.us/reader035/viewer/2022070307/551ae1715503466b6a8b5c1b/html5/thumbnails/23.jpg)
Results
![Page 24: Contributions A people dataset of 8035 images. Three layer attribute classification framework using poselets. 1 2](https://reader035.vdocuments.us/reader035/viewer/2022070307/551ae1715503466b6a8b5c1b/html5/thumbnails/24.jpg)
Results
![Page 25: Contributions A people dataset of 8035 images. Three layer attribute classification framework using poselets. 1 2](https://reader035.vdocuments.us/reader035/viewer/2022070307/551ae1715503466b6a8b5c1b/html5/thumbnails/25.jpg)
Gender Classification Results
![Page 26: Contributions A people dataset of 8035 images. Three layer attribute classification framework using poselets. 1 2](https://reader035.vdocuments.us/reader035/viewer/2022070307/551ae1715503466b6a8b5c1b/html5/thumbnails/26.jpg)
Thanks