deeppose & convolutional pose...
TRANSCRIPT
![Page 1: DeepPose & Convolutional Pose Machinescseweb.ucsd.edu/.../Presentations/12_Human_Pose_Estimation_Rant… · DeepPose & Convolutional Pose Machines. Main Concepts 1. CNN with regressor](https://reader034.vdocuments.us/reader034/viewer/2022042915/5f538fa137cbdf6d2b262af1/html5/thumbnails/1.jpg)
DeepPose & Convolutional Pose Machines
![Page 2: DeepPose & Convolutional Pose Machinescseweb.ucsd.edu/.../Presentations/12_Human_Pose_Estimation_Rant… · DeepPose & Convolutional Pose Machines. Main Concepts 1. CNN with regressor](https://reader034.vdocuments.us/reader034/viewer/2022042915/5f538fa137cbdf6d2b262af1/html5/thumbnails/2.jpg)
Main Concepts1. CNN with regressor head.2. Object Localization.3. Bigger to smaller or Smaller to bigger view.4. Deep Supervised learning to prevent Vanishing gradient problem in deep nets
![Page 3: DeepPose & Convolutional Pose Machinescseweb.ucsd.edu/.../Presentations/12_Human_Pose_Estimation_Rant… · DeepPose & Convolutional Pose Machines. Main Concepts 1. CNN with regressor](https://reader034.vdocuments.us/reader034/viewer/2022042915/5f538fa137cbdf6d2b262af1/html5/thumbnails/3.jpg)
CNN with Regressor head slides: andrej et al
![Page 4: DeepPose & Convolutional Pose Machinescseweb.ucsd.edu/.../Presentations/12_Human_Pose_Estimation_Rant… · DeepPose & Convolutional Pose Machines. Main Concepts 1. CNN with regressor](https://reader034.vdocuments.us/reader034/viewer/2022042915/5f538fa137cbdf6d2b262af1/html5/thumbnails/4.jpg)
CNN with Regressor Head eg code in keras
conv = Convolution2D(64,10,10,)
flat = Flatten()(conv)
left_eye = Dense(1, activation='linear', name='A')(flat)
right_eye = Dense(1, activation='linear', name='B')(flat)
model = Model(input=frame_in, output=[left_eye,right_eye])
model.compile(loss='mse') #mse= L2 loss ,mean square error,
![Page 5: DeepPose & Convolutional Pose Machinescseweb.ucsd.edu/.../Presentations/12_Human_Pose_Estimation_Rant… · DeepPose & Convolutional Pose Machines. Main Concepts 1. CNN with regressor](https://reader034.vdocuments.us/reader034/viewer/2022042915/5f538fa137cbdf6d2b262af1/html5/thumbnails/5.jpg)
Object localization
![Page 6: DeepPose & Convolutional Pose Machinescseweb.ucsd.edu/.../Presentations/12_Human_Pose_Estimation_Rant… · DeepPose & Convolutional Pose Machines. Main Concepts 1. CNN with regressor](https://reader034.vdocuments.us/reader034/viewer/2022042915/5f538fa137cbdf6d2b262af1/html5/thumbnails/6.jpg)
Object localization
![Page 7: DeepPose & Convolutional Pose Machinescseweb.ucsd.edu/.../Presentations/12_Human_Pose_Estimation_Rant… · DeepPose & Convolutional Pose Machines. Main Concepts 1. CNN with regressor](https://reader034.vdocuments.us/reader034/viewer/2022042915/5f538fa137cbdf6d2b262af1/html5/thumbnails/7.jpg)
Human Pose as Object Localization but our object is non-rigid
![Page 8: DeepPose & Convolutional Pose Machinescseweb.ucsd.edu/.../Presentations/12_Human_Pose_Estimation_Rant… · DeepPose & Convolutional Pose Machines. Main Concepts 1. CNN with regressor](https://reader034.vdocuments.us/reader034/viewer/2022042915/5f538fa137cbdf6d2b262af1/html5/thumbnails/8.jpg)
Bigger to small or Small to Big(View)1. First paper takes First Coarse view of full image then goes for magnified view
of parts.2. Second Paper first takes small Receptive field and as depth increases image
view increases.
![Page 9: DeepPose & Convolutional Pose Machinescseweb.ucsd.edu/.../Presentations/12_Human_Pose_Estimation_Rant… · DeepPose & Convolutional Pose Machines. Main Concepts 1. CNN with regressor](https://reader034.vdocuments.us/reader034/viewer/2022042915/5f538fa137cbdf6d2b262af1/html5/thumbnails/9.jpg)
Deep Supervision
![Page 10: DeepPose & Convolutional Pose Machinescseweb.ucsd.edu/.../Presentations/12_Human_Pose_Estimation_Rant… · DeepPose & Convolutional Pose Machines. Main Concepts 1. CNN with regressor](https://reader034.vdocuments.us/reader034/viewer/2022042915/5f538fa137cbdf6d2b262af1/html5/thumbnails/10.jpg)
What’s Deep Supervision?
![Page 11: DeepPose & Convolutional Pose Machinescseweb.ucsd.edu/.../Presentations/12_Human_Pose_Estimation_Rant… · DeepPose & Convolutional Pose Machines. Main Concepts 1. CNN with regressor](https://reader034.vdocuments.us/reader034/viewer/2022042915/5f538fa137cbdf6d2b262af1/html5/thumbnails/11.jpg)
Why it’s needed? Vanishing Gradient
Generally |wj|<1,So, gradient will decrease, As depth increases.
![Page 12: DeepPose & Convolutional Pose Machinescseweb.ucsd.edu/.../Presentations/12_Human_Pose_Estimation_Rant… · DeepPose & Convolutional Pose Machines. Main Concepts 1. CNN with regressor](https://reader034.vdocuments.us/reader034/viewer/2022042915/5f538fa137cbdf6d2b262af1/html5/thumbnails/12.jpg)
Are there other solutions?1. Yes, Proper weight initialization.
2. What LSTMS did for RNN, ResNet’s did for FeedForward NN.
![Page 13: DeepPose & Convolutional Pose Machinescseweb.ucsd.edu/.../Presentations/12_Human_Pose_Estimation_Rant… · DeepPose & Convolutional Pose Machines. Main Concepts 1. CNN with regressor](https://reader034.vdocuments.us/reader034/viewer/2022042915/5f538fa137cbdf6d2b262af1/html5/thumbnails/13.jpg)
Paper 1: DeepPose
![Page 14: DeepPose & Convolutional Pose Machinescseweb.ucsd.edu/.../Presentations/12_Human_Pose_Estimation_Rant… · DeepPose & Convolutional Pose Machines. Main Concepts 1. CNN with regressor](https://reader034.vdocuments.us/reader034/viewer/2022042915/5f538fa137cbdf6d2b262af1/html5/thumbnails/14.jpg)
Why is it Hard?
![Page 15: DeepPose & Convolutional Pose Machinescseweb.ucsd.edu/.../Presentations/12_Human_Pose_Estimation_Rant… · DeepPose & Convolutional Pose Machines. Main Concepts 1. CNN with regressor](https://reader034.vdocuments.us/reader034/viewer/2022042915/5f538fa137cbdf6d2b262af1/html5/thumbnails/15.jpg)
Background1. Pictorial structures: parts and tree based
relations between them based on some priors.
2. Non tree models: The first layer acts as a discriminative, independent body part classifier. The second layer takes the estimated class distributions of the first one into account and is thereby able to predict joint locations by modeling the interdependence and co-occurrence of the parts
![Page 16: DeepPose & Convolutional Pose Machinescseweb.ucsd.edu/.../Presentations/12_Human_Pose_Estimation_Rant… · DeepPose & Convolutional Pose Machines. Main Concepts 1. CNN with regressor](https://reader034.vdocuments.us/reader034/viewer/2022042915/5f538fa137cbdf6d2b262af1/html5/thumbnails/16.jpg)
Problem Specification
Encode the locations of all “k” body joints in pose vector defined as y = (...,yi ,...) ,i ∈ {1,...,k},
where yi contains the x and y coordinates of the ith joint.
![Page 17: DeepPose & Convolutional Pose Machinescseweb.ucsd.edu/.../Presentations/12_Human_Pose_Estimation_Rant… · DeepPose & Convolutional Pose Machines. Main Concepts 1. CNN with regressor](https://reader034.vdocuments.us/reader034/viewer/2022042915/5f538fa137cbdf6d2b262af1/html5/thumbnails/17.jpg)
Bounding Box TrickLets define a bounding box, b = (bc,bw,bh) in absolute image coordinates
Where bc = center of box
bh = height of box
bw = width of box
![Page 18: DeepPose & Convolutional Pose Machinescseweb.ucsd.edu/.../Presentations/12_Human_Pose_Estimation_Rant… · DeepPose & Convolutional Pose Machines. Main Concepts 1. CNN with regressor](https://reader034.vdocuments.us/reader034/viewer/2022042915/5f538fa137cbdf6d2b262af1/html5/thumbnails/18.jpg)
Transporting point to BOX coordinate from Absolute image coordinate
Any point Yi from absolute to BOX coordinate system(b).
![Page 19: DeepPose & Convolutional Pose Machinescseweb.ucsd.edu/.../Presentations/12_Human_Pose_Estimation_Rant… · DeepPose & Convolutional Pose Machines. Main Concepts 1. CNN with regressor](https://reader034.vdocuments.us/reader034/viewer/2022042915/5f538fa137cbdf6d2b262af1/html5/thumbnails/19.jpg)
ExampleYi=(4,4) in absolute frame
BOX b=(bc=(2,2),bw=4,bh=4)
Yi in BOX coordinate=(½,1/2)
Similarly Y=(0,0) becomes(-½,-1/2)
![Page 20: DeepPose & Convolutional Pose Machinescseweb.ucsd.edu/.../Presentations/12_Human_Pose_Estimation_Rant… · DeepPose & Convolutional Pose Machines. Main Concepts 1. CNN with regressor](https://reader034.vdocuments.us/reader034/viewer/2022042915/5f538fa137cbdf6d2b262af1/html5/thumbnails/20.jpg)
Why Box Coordinate?For our network it becomes very easy to work in this normalized environment,
Variance in Images. Large, Small etc.
Can always take inverse to get back Absolute Coordinates wrt to image.
![Page 21: DeepPose & Convolutional Pose Machinescseweb.ucsd.edu/.../Presentations/12_Human_Pose_Estimation_Rant… · DeepPose & Convolutional Pose Machines. Main Concepts 1. CNN with regressor](https://reader034.vdocuments.us/reader034/viewer/2022042915/5f538fa137cbdf6d2b262af1/html5/thumbnails/21.jpg)
Some AssumptionsFrom Now On We will work in Normalized box coordinate system.
Our network will give us predictions in Normalized BOX coordinate system.
Our loss function(L2) will also work in BOX frame.
We will take inverse to get back absolute coordinates.
![Page 22: DeepPose & Convolutional Pose Machinescseweb.ucsd.edu/.../Presentations/12_Human_Pose_Estimation_Rant… · DeepPose & Convolutional Pose Machines. Main Concepts 1. CNN with regressor](https://reader034.vdocuments.us/reader034/viewer/2022042915/5f538fa137cbdf6d2b262af1/html5/thumbnails/22.jpg)
Algo: we will work in stages to improveBasicly AlexNet with K regressor heads
![Page 23: DeepPose & Convolutional Pose Machinescseweb.ucsd.edu/.../Presentations/12_Human_Pose_Estimation_Rant… · DeepPose & Convolutional Pose Machines. Main Concepts 1. CNN with regressor](https://reader034.vdocuments.us/reader034/viewer/2022042915/5f538fa137cbdf6d2b262af1/html5/thumbnails/23.jpg)
Stage:1Where N(x,b) in transformation in box b coordinate system,
B_0 is full image or by person detector
Ψ is neural Net with parameters Ө1
![Page 24: DeepPose & Convolutional Pose Machinescseweb.ucsd.edu/.../Presentations/12_Human_Pose_Estimation_Rant… · DeepPose & Convolutional Pose Machines. Main Concepts 1. CNN with regressor](https://reader034.vdocuments.us/reader034/viewer/2022042915/5f538fa137cbdf6d2b262af1/html5/thumbnails/24.jpg)
Stage 2 onwards:Note: For next stages we are only predicting displacement of improvement.
Note: bounding box evaluation, from prev state
Where σ is hyperparameter, diam(y) is distance btw opposite joints in human torso. Left shoulder right hip,
![Page 25: DeepPose & Convolutional Pose Machinescseweb.ucsd.edu/.../Presentations/12_Human_Pose_Estimation_Rant… · DeepPose & Convolutional Pose Machines. Main Concepts 1. CNN with regressor](https://reader034.vdocuments.us/reader034/viewer/2022042915/5f538fa137cbdf6d2b262af1/html5/thumbnails/25.jpg)
LOSS function:Everything happening in BOX frame
![Page 26: DeepPose & Convolutional Pose Machinescseweb.ucsd.edu/.../Presentations/12_Human_Pose_Estimation_Rant… · DeepPose & Convolutional Pose Machines. Main Concepts 1. CNN with regressor](https://reader034.vdocuments.us/reader034/viewer/2022042915/5f538fa137cbdf6d2b262af1/html5/thumbnails/26.jpg)
![Page 27: DeepPose & Convolutional Pose Machinescseweb.ucsd.edu/.../Presentations/12_Human_Pose_Estimation_Rant… · DeepPose & Convolutional Pose Machines. Main Concepts 1. CNN with regressor](https://reader034.vdocuments.us/reader034/viewer/2022042915/5f538fa137cbdf6d2b262af1/html5/thumbnails/27.jpg)
Dataset setupFLIC: Hollywood
Used Body detector to get initial bounding box. Face based.
LSP: Sports
Directly Used. Since Humans are tightly cropped in image.
![Page 28: DeepPose & Convolutional Pose Machinescseweb.ucsd.edu/.../Presentations/12_Human_Pose_Estimation_Rant… · DeepPose & Convolutional Pose Machines. Main Concepts 1. CNN with regressor](https://reader034.vdocuments.us/reader034/viewer/2022042915/5f538fa137cbdf6d2b262af1/html5/thumbnails/28.jpg)
Dataset ValidationFLIC: Hollywood
Scalar sigma=1.0
LSP: Sports
Scalar sigma=2.0
Number of stages:
S=3 after algo stopped improvement on validation dataset.
![Page 29: DeepPose & Convolutional Pose Machinescseweb.ucsd.edu/.../Presentations/12_Human_Pose_Estimation_Rant… · DeepPose & Convolutional Pose Machines. Main Concepts 1. CNN with regressor](https://reader034.vdocuments.us/reader034/viewer/2022042915/5f538fa137cbdf6d2b262af1/html5/thumbnails/29.jpg)
Metrics: Percent of Detected Joint (PDJ)A joint is considered to be detected if the distance between the predicted and the true joint is within a certain fraction of the torso diameter.
By varying this fraction detection rates are obtained for varying degrees of localization.
![Page 30: DeepPose & Convolutional Pose Machinescseweb.ucsd.edu/.../Presentations/12_Human_Pose_Estimation_Rant… · DeepPose & Convolutional Pose Machines. Main Concepts 1. CNN with regressor](https://reader034.vdocuments.us/reader034/viewer/2022042915/5f538fa137cbdf6d2b262af1/html5/thumbnails/30.jpg)
Metrics Percent Correct Parts
A candidate body part is labeled as correct if its segment endpoints lie within 50% of the length of the ground-truth annotated endpoints.
Penalizes shorter limbs a lot which are harder to detect
![Page 31: DeepPose & Convolutional Pose Machinescseweb.ucsd.edu/.../Presentations/12_Human_Pose_Estimation_Rant… · DeepPose & Convolutional Pose Machines. Main Concepts 1. CNN with regressor](https://reader034.vdocuments.us/reader034/viewer/2022042915/5f538fa137cbdf6d2b262af1/html5/thumbnails/31.jpg)
Some Results
![Page 32: DeepPose & Convolutional Pose Machinescseweb.ucsd.edu/.../Presentations/12_Human_Pose_Estimation_Rant… · DeepPose & Convolutional Pose Machines. Main Concepts 1. CNN with regressor](https://reader034.vdocuments.us/reader034/viewer/2022042915/5f538fa137cbdf6d2b262af1/html5/thumbnails/32.jpg)
Effects of iterative refinement
![Page 33: DeepPose & Convolutional Pose Machinescseweb.ucsd.edu/.../Presentations/12_Human_Pose_Estimation_Rant… · DeepPose & Convolutional Pose Machines. Main Concepts 1. CNN with regressor](https://reader034.vdocuments.us/reader034/viewer/2022042915/5f538fa137cbdf6d2b262af1/html5/thumbnails/33.jpg)
Effects of iterative refinement
![Page 34: DeepPose & Convolutional Pose Machinescseweb.ucsd.edu/.../Presentations/12_Human_Pose_Estimation_Rant… · DeepPose & Convolutional Pose Machines. Main Concepts 1. CNN with regressor](https://reader034.vdocuments.us/reader034/viewer/2022042915/5f538fa137cbdf6d2b262af1/html5/thumbnails/34.jpg)
Take AwayUse DNN regression for accurate object localization.
Very Easier to implement
Iterative refinement works for non rigid objects as well.
![Page 35: DeepPose & Convolutional Pose Machinescseweb.ucsd.edu/.../Presentations/12_Human_Pose_Estimation_Rant… · DeepPose & Convolutional Pose Machines. Main Concepts 1. CNN with regressor](https://reader034.vdocuments.us/reader034/viewer/2022042915/5f538fa137cbdf6d2b262af1/html5/thumbnails/35.jpg)
Paper 2: Convolutional Pose Machines slides Shih-En Wei et al
Where gt is multiclass(k) predictor making belief maps.
![Page 36: DeepPose & Convolutional Pose Machinescseweb.ucsd.edu/.../Presentations/12_Human_Pose_Estimation_Rant… · DeepPose & Convolutional Pose Machines. Main Concepts 1. CNN with regressor](https://reader034.vdocuments.us/reader034/viewer/2022042915/5f538fa137cbdf6d2b262af1/html5/thumbnails/36.jpg)
![Page 37: DeepPose & Convolutional Pose Machinescseweb.ucsd.edu/.../Presentations/12_Human_Pose_Estimation_Rant… · DeepPose & Convolutional Pose Machines. Main Concepts 1. CNN with regressor](https://reader034.vdocuments.us/reader034/viewer/2022042915/5f538fa137cbdf6d2b262af1/html5/thumbnails/37.jpg)
![Page 38: DeepPose & Convolutional Pose Machinescseweb.ucsd.edu/.../Presentations/12_Human_Pose_Estimation_Rant… · DeepPose & Convolutional Pose Machines. Main Concepts 1. CNN with regressor](https://reader034.vdocuments.us/reader034/viewer/2022042915/5f538fa137cbdf6d2b262af1/html5/thumbnails/38.jpg)
Stage 1conv1_stage1 = layers.conv2d(image, 64, 9(kernel), 1(stride))
pool1_stage1 = layers.max_pool2d(conv1_stage1, 2(kernel), 2(stride))
conv2_stage1 = layers.conv2d(pool1_stage1, 64, 9(kernel), 1(stride))
pool2_stage1 = layers.max_pool2d(conv2_stage1, 2(kernel), 2(stride))
conv3_stage1 = layers.conv2d(pool2_stage1, 64, 9(kernel), 1(stride))
pool3_stage1 = layers.max_pool2d(conv3_stage1, 2(kernel), 2(stride))
conv4_stage1 = layers.conv2d(pool3_stage1, 64, 5(kernel), 1(stride))
conv5_stage1 = layers.conv2d(conv4_stage1, 64, 5(kernel), 1(stride))
conv6_stage1 = layers.conv2d(conv5_stage1, 64, 1(kernel), 1(stride))
conv7_stage1 = layers.conv2d(conv6_stage1, 10(p+1), 1(kernel), 1(stride))
![Page 39: DeepPose & Convolutional Pose Machinescseweb.ucsd.edu/.../Presentations/12_Human_Pose_Estimation_Rant… · DeepPose & Convolutional Pose Machines. Main Concepts 1. CNN with regressor](https://reader034.vdocuments.us/reader034/viewer/2022042915/5f538fa137cbdf6d2b262af1/html5/thumbnails/39.jpg)
Stage 2conv4_stage2 = layers.conv2d(pool3_stage2, 64, 5(kernel), 1(stride))
concat_stage2 = tf.concat([conv7_stage1, conv4_stage2])
Mconv1_stage2 = layers.conv2d(concat_stage2, 64, 11(kernel),1(stride))
conv6_stage3=layers.conv2d(Mconv1_stage2,10(p+1),1(kernel),1(stride))
model = Model(input=image, output=[conv6_stage3,conv7_stage1])
model.compile(loss='mse') #mse= L2 loss ,mean square error,
![Page 40: DeepPose & Convolutional Pose Machinescseweb.ucsd.edu/.../Presentations/12_Human_Pose_Estimation_Rant… · DeepPose & Convolutional Pose Machines. Main Concepts 1. CNN with regressor](https://reader034.vdocuments.us/reader034/viewer/2022042915/5f538fa137cbdf6d2b262af1/html5/thumbnails/40.jpg)
![Page 41: DeepPose & Convolutional Pose Machinescseweb.ucsd.edu/.../Presentations/12_Human_Pose_Estimation_Rant… · DeepPose & Convolutional Pose Machines. Main Concepts 1. CNN with regressor](https://reader034.vdocuments.us/reader034/viewer/2022042915/5f538fa137cbdf6d2b262af1/html5/thumbnails/41.jpg)
Note: feature sharing from stage 2 onwards
![Page 42: DeepPose & Convolutional Pose Machinescseweb.ucsd.edu/.../Presentations/12_Human_Pose_Estimation_Rant… · DeepPose & Convolutional Pose Machines. Main Concepts 1. CNN with regressor](https://reader034.vdocuments.us/reader034/viewer/2022042915/5f538fa137cbdf6d2b262af1/html5/thumbnails/42.jpg)
![Page 43: DeepPose & Convolutional Pose Machinescseweb.ucsd.edu/.../Presentations/12_Human_Pose_Estimation_Rant… · DeepPose & Convolutional Pose Machines. Main Concepts 1. CNN with regressor](https://reader034.vdocuments.us/reader034/viewer/2022042915/5f538fa137cbdf6d2b262af1/html5/thumbnails/43.jpg)
![Page 44: DeepPose & Convolutional Pose Machinescseweb.ucsd.edu/.../Presentations/12_Human_Pose_Estimation_Rant… · DeepPose & Convolutional Pose Machines. Main Concepts 1. CNN with regressor](https://reader034.vdocuments.us/reader034/viewer/2022042915/5f538fa137cbdf6d2b262af1/html5/thumbnails/44.jpg)
![Page 45: DeepPose & Convolutional Pose Machinescseweb.ucsd.edu/.../Presentations/12_Human_Pose_Estimation_Rant… · DeepPose & Convolutional Pose Machines. Main Concepts 1. CNN with regressor](https://reader034.vdocuments.us/reader034/viewer/2022042915/5f538fa137cbdf6d2b262af1/html5/thumbnails/45.jpg)
Training styles
![Page 46: DeepPose & Convolutional Pose Machinescseweb.ucsd.edu/.../Presentations/12_Human_Pose_Estimation_Rant… · DeepPose & Convolutional Pose Machines. Main Concepts 1. CNN with regressor](https://reader034.vdocuments.us/reader034/viewer/2022042915/5f538fa137cbdf6d2b262af1/html5/thumbnails/46.jpg)
![Page 47: DeepPose & Convolutional Pose Machinescseweb.ucsd.edu/.../Presentations/12_Human_Pose_Estimation_Rant… · DeepPose & Convolutional Pose Machines. Main Concepts 1. CNN with regressor](https://reader034.vdocuments.us/reader034/viewer/2022042915/5f538fa137cbdf6d2b262af1/html5/thumbnails/47.jpg)
![Page 48: DeepPose & Convolutional Pose Machinescseweb.ucsd.edu/.../Presentations/12_Human_Pose_Estimation_Rant… · DeepPose & Convolutional Pose Machines. Main Concepts 1. CNN with regressor](https://reader034.vdocuments.us/reader034/viewer/2022042915/5f538fa137cbdf6d2b262af1/html5/thumbnails/48.jpg)
Dataset CroppingTrain:
Roughly resize images to have same scale.
Crop or Pad image according to center positions of human to make it 368*368.
Rough scale estimates are provided if not estimate them using joints.
Testing:
They do resizing and cropping(padding)
![Page 49: DeepPose & Convolutional Pose Machinescseweb.ucsd.edu/.../Presentations/12_Human_Pose_Estimation_Rant… · DeepPose & Convolutional Pose Machines. Main Concepts 1. CNN with regressor](https://reader034.vdocuments.us/reader034/viewer/2022042915/5f538fa137cbdf6d2b262af1/html5/thumbnails/49.jpg)
MPII dataset
![Page 50: DeepPose & Convolutional Pose Machinescseweb.ucsd.edu/.../Presentations/12_Human_Pose_Estimation_Rant… · DeepPose & Convolutional Pose Machines. Main Concepts 1. CNN with regressor](https://reader034.vdocuments.us/reader034/viewer/2022042915/5f538fa137cbdf6d2b262af1/html5/thumbnails/50.jpg)
MPII dataset with different viewpoints
![Page 51: DeepPose & Convolutional Pose Machinescseweb.ucsd.edu/.../Presentations/12_Human_Pose_Estimation_Rant… · DeepPose & Convolutional Pose Machines. Main Concepts 1. CNN with regressor](https://reader034.vdocuments.us/reader034/viewer/2022042915/5f538fa137cbdf6d2b262af1/html5/thumbnails/51.jpg)
FLICBeats the prev paper, see black is way above blue.
![Page 52: DeepPose & Convolutional Pose Machinescseweb.ucsd.edu/.../Presentations/12_Human_Pose_Estimation_Rant… · DeepPose & Convolutional Pose Machines. Main Concepts 1. CNN with regressor](https://reader034.vdocuments.us/reader034/viewer/2022042915/5f538fa137cbdf6d2b262af1/html5/thumbnails/52.jpg)
![Page 53: DeepPose & Convolutional Pose Machinescseweb.ucsd.edu/.../Presentations/12_Human_Pose_Estimation_Rant… · DeepPose & Convolutional Pose Machines. Main Concepts 1. CNN with regressor](https://reader034.vdocuments.us/reader034/viewer/2022042915/5f538fa137cbdf6d2b262af1/html5/thumbnails/53.jpg)
Summary:Iterative refining is the key.
Vanishing Gradients, Use Deep Supervision
This architecture can be extended to multiple persons.
I don’t know why authors didn’t used ResNet.
Try to model human 3d model from single image.