![Page 1: Fast 3D Recognition and Pose. Viewpoint Feature Histogramdownload.ros.org/data/Events/CoTeSys-ROS-School/vfh.pdf · 2010. 11. 2. · Rusu et al, Learning Informative Point Classes](https://reader035.vdocuments.us/reader035/viewer/2022081403/60af635c8e0738140559f3ec/html5/thumbnails/1.jpg)
Motivation F/PFH Priors VFH Examples/Results Conclusion
Fast 3D Recognition and Pose.Viewpoint Feature Histogram
Radu Bogdan RUSUNovember 2, 2010
![Page 2: Fast 3D Recognition and Pose. Viewpoint Feature Histogramdownload.ros.org/data/Events/CoTeSys-ROS-School/vfh.pdf · 2010. 11. 2. · Rusu et al, Learning Informative Point Classes](https://reader035.vdocuments.us/reader035/viewer/2022081403/60af635c8e0738140559f3ec/html5/thumbnails/2.jpg)
[Motivation] F/PFH Priors VFH Examples/Results Conclusion
Outline
1. Motivation
2. F/PFH
3. Priors
4. VFH
5. Examples/Results
6. Conclusion
VFH :: Fast 3D Recognition and PoseRadu Bogdan RUSU
![Page 3: Fast 3D Recognition and Pose. Viewpoint Feature Histogramdownload.ros.org/data/Events/CoTeSys-ROS-School/vfh.pdf · 2010. 11. 2. · Rusu et al, Learning Informative Point Classes](https://reader035.vdocuments.us/reader035/viewer/2022081403/60af635c8e0738140559f3ec/html5/thumbnails/3.jpg)
[Motivation] F/PFH Priors VFH Examples/Results Conclusion
Motivation (1/2)Got Robots. Now what?
VFH :: Fast 3D Recognition and PoseRadu Bogdan RUSU
![Page 4: Fast 3D Recognition and Pose. Viewpoint Feature Histogramdownload.ros.org/data/Events/CoTeSys-ROS-School/vfh.pdf · 2010. 11. 2. · Rusu et al, Learning Informative Point Classes](https://reader035.vdocuments.us/reader035/viewer/2022081403/60af635c8e0738140559f3ec/html5/thumbnails/4.jpg)
[Motivation] F/PFH Priors VFH Examples/Results Conclusion
Motivation (2/2)Wanted: Recognition and 6D Pose
Something along the lines of:
I Input:
I Output:
Object 1: ID=XX, Pose=YYObject 2: ID=XX, Pose=YY
VFH :: Fast 3D Recognition and PoseRadu Bogdan RUSU
![Page 5: Fast 3D Recognition and Pose. Viewpoint Feature Histogramdownload.ros.org/data/Events/CoTeSys-ROS-School/vfh.pdf · 2010. 11. 2. · Rusu et al, Learning Informative Point Classes](https://reader035.vdocuments.us/reader035/viewer/2022081403/60af635c8e0738140559f3ec/html5/thumbnails/5.jpg)
Motivation [F/PFH] Priors VFH Examples/Results Conclusion
Outline
1. Motivation
2. F/PFH
3. Priors
4. VFH
5. Examples/Results
6. Conclusion
VFH :: Fast 3D Recognition and PoseRadu Bogdan RUSU
![Page 6: Fast 3D Recognition and Pose. Viewpoint Feature Histogramdownload.ros.org/data/Events/CoTeSys-ROS-School/vfh.pdf · 2010. 11. 2. · Rusu et al, Learning Informative Point Classes](https://reader035.vdocuments.us/reader035/viewer/2022081403/60af635c8e0738140559f3ec/html5/thumbnails/6.jpg)
Motivation [F/PFH] Priors VFH Examples/Results Conclusion
Point Feature Histograms (PFH) (1-4/4)Basic Concepts :: PCL/Features/(F)PFHEstimation
Rusu et al, Learning Informative Point Classes for the Acquisition of Object Model Maps, ICARCV’08
Rusu et al, Fast Point Feature Histograms (FPFH) for 3D Registration, ICRA’09
I For every point pair〈(ps,ns); (pt ,nt)〉, letu = ns, v = (pt−ps)×u, w = u×v
f0 = 〈v ,nj〉f1 = 〈u, pj − pi〉/||pj − pi ||f2 = ||pj − pi ||f3 = atan(〈w ,nj〉, 〈u,nj〉)
ihist =∑x≤3
x=0
⌊fx ·d
fxmax −fxmin
⌋· dx
VFH :: Fast 3D Recognition and PoseRadu Bogdan RUSU
![Page 7: Fast 3D Recognition and Pose. Viewpoint Feature Histogramdownload.ros.org/data/Events/CoTeSys-ROS-School/vfh.pdf · 2010. 11. 2. · Rusu et al, Learning Informative Point Classes](https://reader035.vdocuments.us/reader035/viewer/2022081403/60af635c8e0738140559f3ec/html5/thumbnails/7.jpg)
Motivation [F/PFH] Priors VFH Examples/Results Conclusion
Point Feature Histograms (PFH) (1-4/4)Basic Concepts :: PCL/Features/(F)PFHEstimation
f0 = 〈v ,nj〉f1 = 〈u, pj − pi〉/||pj − pi ||f2 = ||pj − pi ||f3 = atan(〈w ,nj〉, 〈u,nj〉)
ihist =∑x≤3
x=0
⌊fx ·d
fxmax −fxmin
⌋· dx
VFH :: Fast 3D Recognition and PoseRadu Bogdan RUSU
![Page 8: Fast 3D Recognition and Pose. Viewpoint Feature Histogramdownload.ros.org/data/Events/CoTeSys-ROS-School/vfh.pdf · 2010. 11. 2. · Rusu et al, Learning Informative Point Classes](https://reader035.vdocuments.us/reader035/viewer/2022081403/60af635c8e0738140559f3ec/html5/thumbnails/8.jpg)
Motivation [F/PFH] Priors VFH Examples/Results Conclusion
Point Feature Histograms (PFH) (1-4/4)Basic Concepts :: PCL/Features/(F)PFHEstimation
I For every point pair〈(ps,ns); (pt ,nt)〉, letu = ns, v = (pt−ps)×u, w = u×v
VFH :: Fast 3D Recognition and PoseRadu Bogdan RUSU
![Page 9: Fast 3D Recognition and Pose. Viewpoint Feature Histogramdownload.ros.org/data/Events/CoTeSys-ROS-School/vfh.pdf · 2010. 11. 2. · Rusu et al, Learning Informative Point Classes](https://reader035.vdocuments.us/reader035/viewer/2022081403/60af635c8e0738140559f3ec/html5/thumbnails/9.jpg)
Motivation [F/PFH] Priors VFH Examples/Results Conclusion
Point Feature Histograms (PFH) (1-4/4)Points lying on different geometric primitives
VFH :: Fast 3D Recognition and PoseRadu Bogdan RUSU
![Page 10: Fast 3D Recognition and Pose. Viewpoint Feature Histogramdownload.ros.org/data/Events/CoTeSys-ROS-School/vfh.pdf · 2010. 11. 2. · Rusu et al, Learning Informative Point Classes](https://reader035.vdocuments.us/reader035/viewer/2022081403/60af635c8e0738140559f3ec/html5/thumbnails/10.jpg)
Motivation [F/PFH] Priors VFH Examples/Results Conclusion
Fast Point Feature Histograms (FPFH) (1/2)Basic Concepts
Re-formulate: FPFH(p) = SPF (p) + 1k∑k
i=11ωk· SPF (pk )
Point Feature Histograms (PFH) Fast Point Feature Histograms (FPFH)
VFH :: Fast 3D Recognition and PoseRadu Bogdan RUSU
![Page 11: Fast 3D Recognition and Pose. Viewpoint Feature Histogramdownload.ros.org/data/Events/CoTeSys-ROS-School/vfh.pdf · 2010. 11. 2. · Rusu et al, Learning Informative Point Classes](https://reader035.vdocuments.us/reader035/viewer/2022081403/60af635c8e0738140559f3ec/html5/thumbnails/11.jpg)
Motivation [F/PFH] Priors VFH Examples/Results Conclusion
Fast Point Feature Histograms (FPFH) (2/2)Theoretical formulation
VFH :: Fast 3D Recognition and PoseRadu Bogdan RUSU
![Page 12: Fast 3D Recognition and Pose. Viewpoint Feature Histogramdownload.ros.org/data/Events/CoTeSys-ROS-School/vfh.pdf · 2010. 11. 2. · Rusu et al, Learning Informative Point Classes](https://reader035.vdocuments.us/reader035/viewer/2022081403/60af635c8e0738140559f3ec/html5/thumbnails/12.jpg)
Motivation [F/PFH] Priors VFH Examples/Results Conclusion
(F)PFH Usage (1/3)Classification results using FPFH and CRF: 97.36%
VFH :: Fast 3D Recognition and PoseRadu Bogdan RUSU
![Page 13: Fast 3D Recognition and Pose. Viewpoint Feature Histogramdownload.ros.org/data/Events/CoTeSys-ROS-School/vfh.pdf · 2010. 11. 2. · Rusu et al, Learning Informative Point Classes](https://reader035.vdocuments.us/reader035/viewer/2022081403/60af635c8e0738140559f3ec/html5/thumbnails/13.jpg)
Motivation [F/PFH] Priors VFH Examples/Results Conclusion
(F)PFH Usage (2/3)Classification results using FPFH and CRF: 98.27%
VFH :: Fast 3D Recognition and PoseRadu Bogdan RUSU
![Page 14: Fast 3D Recognition and Pose. Viewpoint Feature Histogramdownload.ros.org/data/Events/CoTeSys-ROS-School/vfh.pdf · 2010. 11. 2. · Rusu et al, Learning Informative Point Classes](https://reader035.vdocuments.us/reader035/viewer/2022081403/60af635c8e0738140559f3ec/html5/thumbnails/14.jpg)
Motivation [F/PFH] Priors VFH Examples/Results Conclusion
(F)PFH Usage (3/3)Multiple Scans :: PCL/Registration/IterativeClosestPoint
points on similar surfaces
VFH :: Fast 3D Recognition and PoseRadu Bogdan RUSU
![Page 15: Fast 3D Recognition and Pose. Viewpoint Feature Histogramdownload.ros.org/data/Events/CoTeSys-ROS-School/vfh.pdf · 2010. 11. 2. · Rusu et al, Learning Informative Point Classes](https://reader035.vdocuments.us/reader035/viewer/2022081403/60af635c8e0738140559f3ec/html5/thumbnails/15.jpg)
Motivation F/PFH [Priors] VFH Examples/Results Conclusion
Outline
1. Motivation
2. F/PFH
3. Priors
4. VFH
5. Examples/Results
6. Conclusion
VFH :: Fast 3D Recognition and PoseRadu Bogdan RUSU
![Page 16: Fast 3D Recognition and Pose. Viewpoint Feature Histogramdownload.ros.org/data/Events/CoTeSys-ROS-School/vfh.pdf · 2010. 11. 2. · Rusu et al, Learning Informative Point Classes](https://reader035.vdocuments.us/reader035/viewer/2022081403/60af635c8e0738140559f3ec/html5/thumbnails/16.jpg)
Motivation F/PFH [Priors] VFH Examples/Results Conclusion
PriorsAssumptions and Constraints
I Dense depth + decent surface normal estimatesI Some form of prior segmentation, attention filter, etc (light
clutter segmentation - not the purpose of this paper)I the acquisition viewpoint
I VFH is meta-local and operates on point clusters (!)VFH :: Fast 3D Recognition and PoseRadu Bogdan RUSU
![Page 17: Fast 3D Recognition and Pose. Viewpoint Feature Histogramdownload.ros.org/data/Events/CoTeSys-ROS-School/vfh.pdf · 2010. 11. 2. · Rusu et al, Learning Informative Point Classes](https://reader035.vdocuments.us/reader035/viewer/2022081403/60af635c8e0738140559f3ec/html5/thumbnails/17.jpg)
Motivation F/PFH Priors [VFH] Examples/Results Conclusion
Outline
1. Motivation
2. F/PFH
3. Priors
4. VFH
5. Examples/Results
6. Conclusion
VFH :: Fast 3D Recognition and PoseRadu Bogdan RUSU
![Page 18: Fast 3D Recognition and Pose. Viewpoint Feature Histogramdownload.ros.org/data/Events/CoTeSys-ROS-School/vfh.pdf · 2010. 11. 2. · Rusu et al, Learning Informative Point Classes](https://reader035.vdocuments.us/reader035/viewer/2022081403/60af635c8e0738140559f3ec/html5/thumbnails/18.jpg)
Motivation F/PFH Priors [VFH] Examples/Results Conclusion
VFHTheoretical formulation: first component
I 1 fixed point c = p = 1k ·
∑ki=1 pi
I α = v · nj , φ = u · (pj−pi )
d , θ = arctan(w · nj ,u · nj)
VFH :: Fast 3D Recognition and PoseRadu Bogdan RUSU
![Page 19: Fast 3D Recognition and Pose. Viewpoint Feature Histogramdownload.ros.org/data/Events/CoTeSys-ROS-School/vfh.pdf · 2010. 11. 2. · Rusu et al, Learning Informative Point Classes](https://reader035.vdocuments.us/reader035/viewer/2022081403/60af635c8e0738140559f3ec/html5/thumbnails/19.jpg)
Motivation F/PFH Priors [VFH] Examples/Results Conclusion
VFHTheoretical formulation: second component
I add angles between viewpoint Vp and centroid normal:β = n · Vp
VFH :: Fast 3D Recognition and PoseRadu Bogdan RUSU
![Page 20: Fast 3D Recognition and Pose. Viewpoint Feature Histogramdownload.ros.org/data/Events/CoTeSys-ROS-School/vfh.pdf · 2010. 11. 2. · Rusu et al, Learning Informative Point Classes](https://reader035.vdocuments.us/reader035/viewer/2022081403/60af635c8e0738140559f3ec/html5/thumbnails/20.jpg)
Motivation F/PFH Priors [VFH] Examples/Results Conclusion
VFHTheoretical formulation
VFH :: Fast 3D Recognition and PoseRadu Bogdan RUSU
![Page 21: Fast 3D Recognition and Pose. Viewpoint Feature Histogramdownload.ros.org/data/Events/CoTeSys-ROS-School/vfh.pdf · 2010. 11. 2. · Rusu et al, Learning Informative Point Classes](https://reader035.vdocuments.us/reader035/viewer/2022081403/60af635c8e0738140559f3ec/html5/thumbnails/21.jpg)
Motivation F/PFH Priors VFH [Examples/Results] Conclusion
Outline
1. Motivation
2. F/PFH
3. Priors
4. VFH
5. Examples/Results
6. Conclusion
VFH :: Fast 3D Recognition and PoseRadu Bogdan RUSU
![Page 22: Fast 3D Recognition and Pose. Viewpoint Feature Histogramdownload.ros.org/data/Events/CoTeSys-ROS-School/vfh.pdf · 2010. 11. 2. · Rusu et al, Learning Informative Point Classes](https://reader035.vdocuments.us/reader035/viewer/2022081403/60af635c8e0738140559f3ec/html5/thumbnails/22.jpg)
Motivation F/PFH Priors VFH [Examples/Results] Conclusion
Usage Examples/Results (1/9)“Training”
How to acquire data:I Something like a pan-tilt unit works well...
VFH :: Fast 3D Recognition and PoseRadu Bogdan RUSU
![Page 23: Fast 3D Recognition and Pose. Viewpoint Feature Histogramdownload.ros.org/data/Events/CoTeSys-ROS-School/vfh.pdf · 2010. 11. 2. · Rusu et al, Learning Informative Point Classes](https://reader035.vdocuments.us/reader035/viewer/2022081403/60af635c8e0738140559f3ec/html5/thumbnails/23.jpg)
Motivation F/PFH Priors VFH [Examples/Results] Conclusion
Usage Examples/Results (2/9)“Training”
How to acquire data:I Simulated camera views for a similar camera model +
identical stereo image processing code
VFH :: Fast 3D Recognition and PoseRadu Bogdan RUSU
![Page 24: Fast 3D Recognition and Pose. Viewpoint Feature Histogramdownload.ros.org/data/Events/CoTeSys-ROS-School/vfh.pdf · 2010. 11. 2. · Rusu et al, Learning Informative Point Classes](https://reader035.vdocuments.us/reader035/viewer/2022081403/60af635c8e0738140559f3ec/html5/thumbnails/24.jpg)
Motivation F/PFH Priors VFH [Examples/Results] Conclusion
Usage Examples/Results (3/9)“Training”
VFH :: Fast 3D Recognition and PoseRadu Bogdan RUSU
![Page 25: Fast 3D Recognition and Pose. Viewpoint Feature Histogramdownload.ros.org/data/Events/CoTeSys-ROS-School/vfh.pdf · 2010. 11. 2. · Rusu et al, Learning Informative Point Classes](https://reader035.vdocuments.us/reader035/viewer/2022081403/60af635c8e0738140559f3ec/html5/thumbnails/25.jpg)
Motivation F/PFH Priors VFH [Examples/Results] Conclusion
Usage Examples/Results (4/9)Test on Train
VFH :: Fast 3D Recognition and PoseRadu Bogdan RUSU
![Page 26: Fast 3D Recognition and Pose. Viewpoint Feature Histogramdownload.ros.org/data/Events/CoTeSys-ROS-School/vfh.pdf · 2010. 11. 2. · Rusu et al, Learning Informative Point Classes](https://reader035.vdocuments.us/reader035/viewer/2022081403/60af635c8e0738140559f3ec/html5/thumbnails/26.jpg)
Motivation F/PFH Priors VFH [Examples/Results] Conclusion
Usage Examples/Results (5/9)How to use and interpret the results
VFH :: Fast 3D Recognition and PoseRadu Bogdan RUSU
![Page 27: Fast 3D Recognition and Pose. Viewpoint Feature Histogramdownload.ros.org/data/Events/CoTeSys-ROS-School/vfh.pdf · 2010. 11. 2. · Rusu et al, Learning Informative Point Classes](https://reader035.vdocuments.us/reader035/viewer/2022081403/60af635c8e0738140559f3ec/html5/thumbnails/27.jpg)
Motivation F/PFH Priors VFH [Examples/Results] Conclusion
Usage Examples/Results (6/9)How to use and interpret the results
VFH :: Fast 3D Recognition and PoseRadu Bogdan RUSU
![Page 28: Fast 3D Recognition and Pose. Viewpoint Feature Histogramdownload.ros.org/data/Events/CoTeSys-ROS-School/vfh.pdf · 2010. 11. 2. · Rusu et al, Learning Informative Point Classes](https://reader035.vdocuments.us/reader035/viewer/2022081403/60af635c8e0738140559f3ec/html5/thumbnails/28.jpg)
Motivation F/PFH Priors VFH [Examples/Results] Conclusion
Usage Examples/Results (7/9)How to use and interpret the results
VFH :: Fast 3D Recognition and PoseRadu Bogdan RUSU
![Page 29: Fast 3D Recognition and Pose. Viewpoint Feature Histogramdownload.ros.org/data/Events/CoTeSys-ROS-School/vfh.pdf · 2010. 11. 2. · Rusu et al, Learning Informative Point Classes](https://reader035.vdocuments.us/reader035/viewer/2022081403/60af635c8e0738140559f3ec/html5/thumbnails/29.jpg)
Motivation F/PFH Priors VFH [Examples/Results] Conclusion
Usage Examples/Results (8/9)How to use and interpret the results
VFH
Spin
VFH :: Fast 3D Recognition and PoseRadu Bogdan RUSU
![Page 30: Fast 3D Recognition and Pose. Viewpoint Feature Histogramdownload.ros.org/data/Events/CoTeSys-ROS-School/vfh.pdf · 2010. 11. 2. · Rusu et al, Learning Informative Point Classes](https://reader035.vdocuments.us/reader035/viewer/2022081403/60af635c8e0738140559f3ec/html5/thumbnails/30.jpg)
Motivation F/PFH Priors VFH [Examples/Results] Conclusion
Usage Examples/Results (9/9)Performance Indicators
Numbers are not relevant - use them as indicators:I extremely fast (SSE optimized implementation in PCL
http://pcl.ros.org): ≈0.3ms / point clusterI scales very well (kudos to FLANN): tried it on ≈55000
scenesI recognition performance (%50 + 1, first 10 NN, capped on
dth = 50:Object Pose
Method Recognition EstimationVFH 98.52% 98.52%Spin 75.3% 61.2%
I good behavior for synthetic data simulation (advantage ifCAD models already exist)
VFH :: Fast 3D Recognition and PoseRadu Bogdan RUSU
![Page 31: Fast 3D Recognition and Pose. Viewpoint Feature Histogramdownload.ros.org/data/Events/CoTeSys-ROS-School/vfh.pdf · 2010. 11. 2. · Rusu et al, Learning Informative Point Classes](https://reader035.vdocuments.us/reader035/viewer/2022081403/60af635c8e0738140559f3ec/html5/thumbnails/31.jpg)
Motivation F/PFH Priors VFH Examples/Results [Conclusion]
Outline
1. Motivation
2. F/PFH
3. Priors
4. VFH
5. Examples/Results
6. Conclusion
VFH :: Fast 3D Recognition and PoseRadu Bogdan RUSU
![Page 32: Fast 3D Recognition and Pose. Viewpoint Feature Histogramdownload.ros.org/data/Events/CoTeSys-ROS-School/vfh.pdf · 2010. 11. 2. · Rusu et al, Learning Informative Point Classes](https://reader035.vdocuments.us/reader035/viewer/2022081403/60af635c8e0738140559f3ec/html5/thumbnails/32.jpg)
Motivation F/PFH Priors VFH Examples/Results [Conclusion]
Questions?
Open Source! Go try it out and let us if it’s useful or not!
http://pcl.ros.orgVFH :: Fast 3D Recognition and PoseRadu Bogdan RUSU