a scale and rotation invariant approach to tracking human body part regions in videos
DESCRIPTION
A Scale and Rotation Invariant Approach to Tracking Human Body Part Regions in Videos. Yihang Bo. Hao Jiang. Institute of Automation, CAS Boston College. Boston College. Challenges. Previous Rectangular Part Methods. Templates with Different scales . Templates with - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: A Scale and Rotation Invariant Approach to Tracking Human Body Part Regions in Videos](https://reader035.vdocuments.us/reader035/viewer/2022062323/5681674f550346895ddc00bb/html5/thumbnails/1.jpg)
A Scale and Rotation Invariant Approach to Tracking Human Body
Part Regions in Videos
Yihang Bo Hao JiangInstitute of Automation, CASBoston CollegeBoston College
![Page 2: A Scale and Rotation Invariant Approach to Tracking Human Body Part Regions in Videos](https://reader035.vdocuments.us/reader035/viewer/2022062323/5681674f550346895ddc00bb/html5/thumbnails/2.jpg)
Challenges
![Page 3: A Scale and Rotation Invariant Approach to Tracking Human Body Part Regions in Videos](https://reader035.vdocuments.us/reader035/viewer/2022062323/5681674f550346895ddc00bb/html5/thumbnails/3.jpg)
Previous Rectangular Part Methods
Templates with Different scales
Templates with Different rotations
If the target scale and rotation are unknown, local part extraction becomes a very slow process.
![Page 4: A Scale and Rotation Invariant Approach to Tracking Human Body Part Regions in Videos](https://reader035.vdocuments.us/reader035/viewer/2022062323/5681674f550346895ddc00bb/html5/thumbnails/4.jpg)
Solution: Finding Body Part Regions
![Page 5: A Scale and Rotation Invariant Approach to Tracking Human Body Part Regions in Videos](https://reader035.vdocuments.us/reader035/viewer/2022062323/5681674f550346895ddc00bb/html5/thumbnails/5.jpg)
Overview of the Method
We track human body part regions (arm, leg and torso) in videos.
Our model considers spatial and temporal coupling among parts.
It is invariant to scale and rotation.
![Page 6: A Scale and Rotation Invariant Approach to Tracking Human Body Part Regions in Videos](https://reader035.vdocuments.us/reader035/viewer/2022062323/5681674f550346895ddc00bb/html5/thumbnails/6.jpg)
Tracking Body Part Regions
![Page 7: A Scale and Rotation Invariant Approach to Tracking Human Body Part Regions in Videos](https://reader035.vdocuments.us/reader035/viewer/2022062323/5681674f550346895ddc00bb/html5/thumbnails/7.jpg)
The Non-tree Model
Body part coupling between two successive video frames
![Page 8: A Scale and Rotation Invariant Approach to Tracking Human Body Part Regions in Videos](https://reader035.vdocuments.us/reader035/viewer/2022062323/5681674f550346895ddc00bb/html5/thumbnails/8.jpg)
Part Region Candidates
Object class independent Region ProposalsSuperpixels
Ian Endres, and Derek Hoiem, “Category Independent Object Proposals”, ECCV 2010.
P.F. Felzenszwalb and D.P. Huttenlocher, Efficient Graph-Based Image SegmentationInternational Journal of Computer Vision, Volume 59, Number 2, September 2004.
![Page 9: A Scale and Rotation Invariant Approach to Tracking Human Body Part Regions in Videos](https://reader035.vdocuments.us/reader035/viewer/2022062323/5681674f550346895ddc00bb/html5/thumbnails/9.jpg)
3D Superpixels
Video segmentation (3D superpixels) usually do not directly give human part regions.
![Page 10: A Scale and Rotation Invariant Approach to Tracking Human Body Part Regions in Videos](https://reader035.vdocuments.us/reader035/viewer/2022062323/5681674f550346895ddc00bb/html5/thumbnails/10.jpg)
Partial Background Removal (Optional)warping
warpingwarpingwarping
……
![Page 11: A Scale and Rotation Invariant Approach to Tracking Human Body Part Regions in Videos](https://reader035.vdocuments.us/reader035/viewer/2022062323/5681674f550346895ddc00bb/html5/thumbnails/11.jpg)
Criteria
Shape Matching Part Distance Part Overlap Relative Ratio
Shape Changes Position Changes
Appearance Changes
![Page 12: A Scale and Rotation Invariant Approach to Tracking Human Body Part Regions in Videos](https://reader035.vdocuments.us/reader035/viewer/2022062323/5681674f550346895ddc00bb/html5/thumbnails/12.jpg)
Distance Term
Lj
kkk tfjfdfG ))(),(()(
![Page 13: A Scale and Rotation Invariant Approach to Tracking Human Body Part Regions in Videos](https://reader035.vdocuments.us/reader035/viewer/2022062323/5681674f550346895ddc00bb/html5/thumbnails/13.jpg)
Nji kk
kkk jFiF
jFiFfO},{ ))()((
))()(()(
Overlap
RegionOverlap
RegionOverlap
![Page 14: A Scale and Rotation Invariant Approach to Tracking Human Body Part Regions in Videos](https://reader035.vdocuments.us/reader035/viewer/2022062323/5681674f550346895ddc00bb/html5/thumbnails/14.jpg)
Pi Pj ji
jikkk
jfifrfA 2
,
2, )))(),(((
)(
Size Ratio
Part SizeRatio
![Page 15: A Scale and Rotation Invariant Approach to Tracking Human Body Part Regions in Videos](https://reader035.vdocuments.us/reader035/viewer/2022062323/5681674f550346895ddc00bb/html5/thumbnails/15.jpg)
Piffififkk kkkkssssffS ||||||||),(
11 )()(1
Shape Consistency Across Frames
ShapeConsistency
![Page 16: A Scale and Rotation Invariant Approach to Tracking Human Body Part Regions in Videos](https://reader035.vdocuments.us/reader035/viewer/2022062323/5681674f550346895ddc00bb/html5/thumbnails/16.jpg)
Piififkk kk
llffL ||||),( )()(1 1
Motion Smoothness
MotionContinuity
![Page 17: A Scale and Rotation Invariant Approach to Tracking Human Body Part Regions in Videos](https://reader035.vdocuments.us/reader035/viewer/2022062323/5681674f550346895ddc00bb/html5/thumbnails/17.jpg)
Piififkk kk
hhffH ||||),( )()(1 1
Color Consistency
AppearanceConsistency
![Page 18: A Scale and Rotation Invariant Approach to Tracking Human Body Part Regions in Videos](https://reader035.vdocuments.us/reader035/viewer/2022062323/5681674f550346895ddc00bb/html5/thumbnails/18.jpg)
Inference on a Loopy Graph
…
We assign region candidates to each of the body part nodeso that the objective function is minimized.
![Page 19: A Scale and Rotation Invariant Approach to Tracking Human Body Part Regions in Videos](https://reader035.vdocuments.us/reader035/viewer/2022062323/5681674f550346895ddc00bb/html5/thumbnails/19.jpg)
Convert to a Chain
…
…
Linear meta-graph
![Page 20: A Scale and Rotation Invariant Approach to Tracking Human Body Part Regions in Videos](https://reader035.vdocuments.us/reader035/viewer/2022062323/5681674f550346895ddc00bb/html5/thumbnails/20.jpg)
Convert to a Chain
…
…
Unfortunately, there are too many whole body configurations in each video frame.
![Page 21: A Scale and Rotation Invariant Approach to Tracking Human Body Part Regions in Videos](https://reader035.vdocuments.us/reader035/viewer/2022062323/5681674f550346895ddc00bb/html5/thumbnails/21.jpg)
Convert to a Chain
…
…
Solution: we find the best-N whole body configurationsin each video frame.
![Page 22: A Scale and Rotation Invariant Approach to Tracking Human Body Part Regions in Videos](https://reader035.vdocuments.us/reader035/viewer/2022062323/5681674f550346895ddc00bb/html5/thumbnails/22.jpg)
Cycle Removal
![Page 23: A Scale and Rotation Invariant Approach to Tracking Human Body Part Regions in Videos](https://reader035.vdocuments.us/reader035/viewer/2022062323/5681674f550346895ddc00bb/html5/thumbnails/23.jpg)
Cycle Breaking
![Page 24: A Scale and Rotation Invariant Approach to Tracking Human Body Part Regions in Videos](https://reader035.vdocuments.us/reader035/viewer/2022062323/5681674f550346895ddc00bb/html5/thumbnails/24.jpg)
Find Best-N Body Configurations on a Cycle
Best-N (with torso1)
Best-N (with torso2) +
Best-N (with torso1,2)
Best-N (with torso3) +
Best-N (with torso1,2,3)
…
Best-N (with torso M) +
Best-N (with torso1..M)
![Page 25: A Scale and Rotation Invariant Approach to Tracking Human Body Part Regions in Videos](https://reader035.vdocuments.us/reader035/viewer/2022062323/5681674f550346895ddc00bb/html5/thumbnails/25.jpg)
Region Tracking on a Trellis
Frame 1 Frame 2 Frame k
Best-NBody configurations
![Page 26: A Scale and Rotation Invariant Approach to Tracking Human Body Part Regions in Videos](https://reader035.vdocuments.us/reader035/viewer/2022062323/5681674f550346895ddc00bb/html5/thumbnails/26.jpg)
Sample Results on Five Test Videos
V1
V2
V3
V4
V5
![Page 27: A Scale and Rotation Invariant Approach to Tracking Human Body Part Regions in Videos](https://reader035.vdocuments.us/reader035/viewer/2022062323/5681674f550346895ddc00bb/html5/thumbnails/27.jpg)
Comparison Result
[N-best] D. Park, D. Ramanan. "N-Best Maximal Decoders for Part Models”, ICCV 2011.
![Page 28: A Scale and Rotation Invariant Approach to Tracking Human Body Part Regions in Videos](https://reader035.vdocuments.us/reader035/viewer/2022062323/5681674f550346895ddc00bb/html5/thumbnails/28.jpg)
Quantitative resultsComparison Result
![Page 29: A Scale and Rotation Invariant Approach to Tracking Human Body Part Regions in Videos](https://reader035.vdocuments.us/reader035/viewer/2022062323/5681674f550346895ddc00bb/html5/thumbnails/29.jpg)
Conclusion
• By tracking body part regions, we can achieve efficient scale and rotation invariant human pose tracking.
• This method can be used for human tracking in complex sports videos.
![Page 30: A Scale and Rotation Invariant Approach to Tracking Human Body Part Regions in Videos](https://reader035.vdocuments.us/reader035/viewer/2022062323/5681674f550346895ddc00bb/html5/thumbnails/30.jpg)
Thank You