change detection of 3d scene with 3d and 2d information for environment checking
DESCRIPTION
presentation slides for PhD degree of Baowei Lin @Hiroshima University. 20130812TRANSCRIPT
![Page 1: Change Detection of 3D Scene with 3D and 2D Information for Environment Checking](https://reader033.vdocuments.us/reader033/viewer/2022052907/55931a081a28abdc428b459e/html5/thumbnails/1.jpg)
1
Change Detection of 3D Scene with 3D and 2D Information for
Environment Checking
PhD Candidate: Baowei Lin August 12th, 2013
![Page 2: Change Detection of 3D Scene with 3D and 2D Information for Environment Checking](https://reader033.vdocuments.us/reader033/viewer/2022052907/55931a081a28abdc428b459e/html5/thumbnails/2.jpg)
1. Introduction Research Motivation Change Detection
2. 3D Keypoints Based 3D-2D Matching Background 3D Keypoints Detection Evaluation
3. 3D-2D Based Change Detection Background Image Based Change Detection Evaluation
4. 3D-3D Based Change Detection Background Scale Estimation of a Single Point Cloud Scale Ratio Estimation of Two Point Clouds Evaluation
5. Conclusions
2
![Page 3: Change Detection of 3D Scene with 3D and 2D Information for Environment Checking](https://reader033.vdocuments.us/reader033/viewer/2022052907/55931a081a28abdc428b459e/html5/thumbnails/3.jpg)
1. Introduction Research Motivation Change Detection
2. 3D Keypoints Based 3D-2D Matching Background 3D Keypoints Detection Evaluation
3. 3D-2D Based Change Detection Background Image Based Change Detection Evaluation
4. 3D-3D Based Change Detection Background Scale Estimation of a Single Point Cloud Scale Ratio Estimation of Two Point Clouds Evaluation
5. Conclusions
3
![Page 4: Change Detection of 3D Scene with 3D and 2D Information for Environment Checking](https://reader033.vdocuments.us/reader033/viewer/2022052907/55931a081a28abdc428b459e/html5/thumbnails/4.jpg)
4
Changes should be alerted at these areas.
![Page 5: Change Detection of 3D Scene with 3D and 2D Information for Environment Checking](https://reader033.vdocuments.us/reader033/viewer/2022052907/55931a081a28abdc428b459e/html5/thumbnails/5.jpg)
5
Original configuration
Damaged configuration
wave washing
if changed
dangerous
![Page 6: Change Detection of 3D Scene with 3D and 2D Information for Environment Checking](https://reader033.vdocuments.us/reader033/viewer/2022052907/55931a081a28abdc428b459e/html5/thumbnails/6.jpg)
6
• Impossible to check manually Wide range Huge number of blocks
• Important to check automatically
![Page 7: Change Detection of 3D Scene with 3D and 2D Information for Environment Checking](https://reader033.vdocuments.us/reader033/viewer/2022052907/55931a081a28abdc428b459e/html5/thumbnails/7.jpg)
7
• Impractical to check by fixed cameras
![Page 8: Change Detection of 3D Scene with 3D and 2D Information for Environment Checking](https://reader033.vdocuments.us/reader033/viewer/2022052907/55931a081a28abdc428b459e/html5/thumbnails/8.jpg)
8
• possible to check by hand-held devices
![Page 9: Change Detection of 3D Scene with 3D and 2D Information for Environment Checking](https://reader033.vdocuments.us/reader033/viewer/2022052907/55931a081a28abdc428b459e/html5/thumbnails/9.jpg)
9
Finding potential change area.
Sub-goal 1:
![Page 10: Change Detection of 3D Scene with 3D and 2D Information for Environment Checking](https://reader033.vdocuments.us/reader033/viewer/2022052907/55931a081a28abdc428b459e/html5/thumbnails/10.jpg)
10
Estimating accurate changes.
Sub-goal 2: offline
Finding potential change area. online
Sub-goal 1:
![Page 11: Change Detection of 3D Scene with 3D and 2D Information for Environment Checking](https://reader033.vdocuments.us/reader033/viewer/2022052907/55931a081a28abdc428b459e/html5/thumbnails/11.jpg)
1. Introduction Research Motivation Change Detection
2. 3D Keypoints Based 3D-2D Matching Background 3D Keypoints Detection Evaluation
3. 3D-2D Based Change Detection Background Image Based Change Detection Evaluation
4. 3D-3D Based Change Detection Background Scale Estimation of a Single Point Cloud Scale Ratio Estimation of Two Point Clouds Evaluation
5. Conclusions
11
![Page 12: Change Detection of 3D Scene with 3D and 2D Information for Environment Checking](https://reader033.vdocuments.us/reader033/viewer/2022052907/55931a081a28abdc428b459e/html5/thumbnails/12.jpg)
A change is a difference of objects in the scene at time A and at time B.
12
Time A Time B
![Page 13: Change Detection of 3D Scene with 3D and 2D Information for Environment Checking](https://reader033.vdocuments.us/reader033/viewer/2022052907/55931a081a28abdc428b459e/html5/thumbnails/13.jpg)
13
3D point cloud
Training images (2D images)
![Page 14: Change Detection of 3D Scene with 3D and 2D Information for Environment Checking](https://reader033.vdocuments.us/reader033/viewer/2022052907/55931a081a28abdc428b459e/html5/thumbnails/14.jpg)
1. 2D-2D Method
14
Need Fixed camera input
output
Original image
Change image
Changed area
2D-2D
![Page 15: Change Detection of 3D Scene with 3D and 2D Information for Environment Checking](https://reader033.vdocuments.us/reader033/viewer/2022052907/55931a081a28abdc428b459e/html5/thumbnails/15.jpg)
input
output
2. 3D-2D Method
15
Detection is fast but not accurate
Original point cloud
Change image
Changed area
3D-2D
![Page 16: Change Detection of 3D Scene with 3D and 2D Information for Environment Checking](https://reader033.vdocuments.us/reader033/viewer/2022052907/55931a081a28abdc428b459e/html5/thumbnails/16.jpg)
input
output
3. 3D-3D Method
16
Detection is accurate but slow
Original point cloud
Change point cloud
Changed area
3D-3D
![Page 17: Change Detection of 3D Scene with 3D and 2D Information for Environment Checking](https://reader033.vdocuments.us/reader033/viewer/2022052907/55931a081a28abdc428b459e/html5/thumbnails/17.jpg)
17
Potential changed areas 3D information
Camera poses
Online system Offline system
3D-2D based change detection
3D-2D based camera pose estimation 3D-3D based
change detection
Chapter 2 Chapter 3 Chapter 4
3D-3D 3D-2D
![Page 18: Change Detection of 3D Scene with 3D and 2D Information for Environment Checking](https://reader033.vdocuments.us/reader033/viewer/2022052907/55931a081a28abdc428b459e/html5/thumbnails/18.jpg)
1. Introduction Research Motivation Change Detection
2. 3D Keypoints Based 3D-2D Matching Background 3D Keypoints Detection Evaluation
3. 3D-2D Based Change Detection Background Image Based Change Detection Evaluation
4. 3D-3D Based Change Detection Background Scale Estimation of a Single Point Cloud Scale Ratio Estimation of Two Point Clouds Evaluation
5. Conclusions
18
![Page 19: Change Detection of 3D Scene with 3D and 2D Information for Environment Checking](https://reader033.vdocuments.us/reader033/viewer/2022052907/55931a081a28abdc428b459e/html5/thumbnails/19.jpg)
19
3D interesting points
2D interesting points
Camera pose=[R,t]
3D point cloud
2D training images
![Page 20: Change Detection of 3D Scene with 3D and 2D Information for Environment Checking](https://reader033.vdocuments.us/reader033/viewer/2022052907/55931a081a28abdc428b459e/html5/thumbnails/20.jpg)
20
2D-2D
3D-3D
SIFT[Lowe 2004], SURF [Bay 2006], etc.
spin image[Johnson 1998], NARF[Steder 2010], etc.
detector descriptor
detector descriptor
![Page 21: Change Detection of 3D Scene with 3D and 2D Information for Environment Checking](https://reader033.vdocuments.us/reader033/viewer/2022052907/55931a081a28abdc428b459e/html5/thumbnails/21.jpg)
21
3D point cloud
2D training images
3D detector and 2D detector can not be corresponded.
![Page 22: Change Detection of 3D Scene with 3D and 2D Information for Environment Checking](https://reader033.vdocuments.us/reader033/viewer/2022052907/55931a081a28abdc428b459e/html5/thumbnails/22.jpg)
22
Image patch
Point distribution
Can not match
2D image
3D point cloud
![Page 23: Change Detection of 3D Scene with 3D and 2D Information for Environment Checking](https://reader033.vdocuments.us/reader033/viewer/2022052907/55931a081a28abdc428b459e/html5/thumbnails/23.jpg)
23
• Detect keypoints correctly
• Describe keypoints appropriately
2D image
3D point cloud
![Page 24: Change Detection of 3D Scene with 3D and 2D Information for Environment Checking](https://reader033.vdocuments.us/reader033/viewer/2022052907/55931a081a28abdc428b459e/html5/thumbnails/24.jpg)
Introduction ◦ Research Motivation ◦ Change Detection
3D Keypoints Based 3D-2D Matching ◦ Background ◦ 3D Keypoints Detection ◦ Evaluation
3D-2D Based Change Detection ◦ Background ◦ Image Based Change Detection ◦ Evaluation
3D-3D Based Change Detection ◦ Background ◦ Scale Estimation of a Single Point Cloud ◦ Scale Ratio Estimation of Two Point Clouds ◦ Evaluation
Conclusions
24
![Page 25: Change Detection of 3D Scene with 3D and 2D Information for Environment Checking](https://reader033.vdocuments.us/reader033/viewer/2022052907/55931a081a28abdc428b459e/html5/thumbnails/25.jpg)
Detected 2D interesting points
25
Feature matching
Obviously, the SIFT features could be used in 3D keypoints detection and description.
![Page 26: Change Detection of 3D Scene with 3D and 2D Information for Environment Checking](https://reader033.vdocuments.us/reader033/viewer/2022052907/55931a081a28abdc428b459e/html5/thumbnails/26.jpg)
Point cloud
26
P1
P2
P3 P4
P5
P6 Camera position
3D keypoint
Projected 3D points
2D images number threshold used for 3D keypoints decision.
the points which can appear on multiple training images
Back face points are not used for computation
3D keypoints
th_v
![Page 27: Change Detection of 3D Scene with 3D and 2D Information for Environment Checking](https://reader033.vdocuments.us/reader033/viewer/2022052907/55931a081a28abdc428b459e/html5/thumbnails/27.jpg)
27
th_v = 1 #3D keypoints ≅10,000
27 training images 105,779 3D points
th_v = 7 #3D keypoints ≅ 1,000
Reconstructed 3D points #3D points ≅30,000
Too many for real time calculating
Smaller number and good distribution
ours
orig
inal
![Page 28: Change Detection of 3D Scene with 3D and 2D Information for Environment Checking](https://reader033.vdocuments.us/reader033/viewer/2022052907/55931a081a28abdc428b459e/html5/thumbnails/28.jpg)
28
2D SIFT keypoints and descriptors
3
3D keypoint& descriptor
Projected 3D points should overlapped to 2D SIFT keypoint
-Keep all 2D descriptors Accurate but slow
Description methods: -Average and Median
SIFT features are different when view directions are different.
![Page 29: Change Detection of 3D Scene with 3D and 2D Information for Environment Checking](https://reader033.vdocuments.us/reader033/viewer/2022052907/55931a081a28abdc428b459e/html5/thumbnails/29.jpg)
Introduction ◦ Research Motivation ◦ Change Detection
3D Keypoints Based 3D-2D Matching ◦ Background ◦ 3D Keypoints Detection ◦ Evaluation
3D-2D Based Change Detection ◦ Background ◦ Image Based Change Detection ◦ Evaluation
3D-3D Based Change Detection ◦ Background ◦ Scale Estimation of a Single Point Cloud ◦ Scale Ratio Estimation of Two Point Clouds ◦ Evaluation
Conclusions
29
![Page 30: Change Detection of 3D Scene with 3D and 2D Information for Environment Checking](https://reader033.vdocuments.us/reader033/viewer/2022052907/55931a081a28abdc428b459e/html5/thumbnails/30.jpg)
30
P1 P2 P3 P4 P5 P6
3D point cloud
Ground truth Camera
positions
……
Training images
![Page 31: Change Detection of 3D Scene with 3D and 2D Information for Environment Checking](https://reader033.vdocuments.us/reader033/viewer/2022052907/55931a081a28abdc428b459e/html5/thumbnails/31.jpg)
31
P1 P2 P3 P4 P5 P6
P6’
Camera pose estimation
3D keypoints generation
![Page 32: Change Detection of 3D Scene with 3D and 2D Information for Environment Checking](https://reader033.vdocuments.us/reader033/viewer/2022052907/55931a081a28abdc428b459e/html5/thumbnails/32.jpg)
32
P1
P2
P3 P4
P5
P6
P6’
P6’ =[R ’ |t ’]
P6 =[R | t ]
![Page 33: Change Detection of 3D Scene with 3D and 2D Information for Environment Checking](https://reader033.vdocuments.us/reader033/viewer/2022052907/55931a081a28abdc428b459e/html5/thumbnails/33.jpg)
Tra
nsla
tion e
rror
Rota
tion e
rror[ra
d]
33
1. Our method is accurate 2. th_v does not affect the result
th_v is used for 3D keypoints selection
2 degrees Dataset: 27 training images Image resolution:2256x1504 3D points number:105,779 3D scene size:40x25x5cm Bounding box size:10.6x5.7x1.4
0.24cm
![Page 34: Change Detection of 3D Scene with 3D and 2D Information for Environment Checking](https://reader033.vdocuments.us/reader033/viewer/2022052907/55931a081a28abdc428b459e/html5/thumbnails/34.jpg)
34
3D point cloud Query image
project 3D points
![Page 35: Change Detection of 3D Scene with 3D and 2D Information for Environment Checking](https://reader033.vdocuments.us/reader033/viewer/2022052907/55931a081a28abdc428b459e/html5/thumbnails/35.jpg)
35
![Page 36: Change Detection of 3D Scene with 3D and 2D Information for Environment Checking](https://reader033.vdocuments.us/reader033/viewer/2022052907/55931a081a28abdc428b459e/html5/thumbnails/36.jpg)
Introduction ◦ Research Motivation ◦ Change Detection
3D Keypoints Based 3D-2D Matching ◦ Background ◦ 3D Keypoints Detection ◦ Evaluation
3D-2D Based Change Detection ◦ Background ◦ Image Based Change Detection ◦ Evaluation
3D-3D Based Change Detection ◦ Background ◦ Scale Estimation of a Single Point Cloud ◦ Scale Ratio Estimation of Two Point Clouds ◦ Evaluation
Conclusions
36
![Page 37: Change Detection of 3D Scene with 3D and 2D Information for Environment Checking](https://reader033.vdocuments.us/reader033/viewer/2022052907/55931a081a28abdc428b459e/html5/thumbnails/37.jpg)
37
Potential changed areas 3D information
Camera poses
Online system Offline system
3D-2D based change detection
3D-2D based camera pose estimation
3D-3D based change detection
![Page 38: Change Detection of 3D Scene with 3D and 2D Information for Environment Checking](https://reader033.vdocuments.us/reader033/viewer/2022052907/55931a081a28abdc428b459e/html5/thumbnails/38.jpg)
38
Our method: 1. Use local feature instead of color 2. Detect any shape of object
Using laser range finder [Goncalves 2010,
Ryle 2011 and Neuman
2011].
Not for wide area targets.
Not applicable for our round shape or natural scenes.
Matching 3D line segments [Eden 2008].
Using color differences [Sato
2006, Pollard 2007 and Taneja 2011]
Not stable for illumination changes.
![Page 39: Change Detection of 3D Scene with 3D and 2D Information for Environment Checking](https://reader033.vdocuments.us/reader033/viewer/2022052907/55931a081a28abdc428b459e/html5/thumbnails/39.jpg)
Introduction ◦ Research Motivation ◦ Change Detection
3D Keypoints Based 3D-2D Matching ◦ Background ◦ 3D Keypoints Detection ◦ Evaluation
3D-2D Based Change Detection ◦ Background ◦ Image Based Change Detection ◦ Evaluation
3D-3D Based Change Detection ◦ Background ◦ Scale Estimation of a Single Point Cloud ◦ Scale Ratio Estimation of Two Point Clouds ◦ Evaluation
Conclusions
39
![Page 40: Change Detection of 3D Scene with 3D and 2D Information for Environment Checking](https://reader033.vdocuments.us/reader033/viewer/2022052907/55931a081a28abdc428b459e/html5/thumbnails/40.jpg)
40
36
1. Find the nearest image
Query image
Nearest image
2. Find changed area Nearest image Query image
changed area
3. Visualization
Project 3D points onto changed area
![Page 41: Change Detection of 3D Scene with 3D and 2D Information for Environment Checking](https://reader033.vdocuments.us/reader033/viewer/2022052907/55931a081a28abdc428b459e/html5/thumbnails/41.jpg)
1st Nearest Query image
41
P1 P2 P3 P4 P5
3D keypoints generation
Need fixed camera
Smallest distance
Ground truth
……
Training images 2nd 3rd
P
![Page 42: Change Detection of 3D Scene with 3D and 2D Information for Environment Checking](https://reader033.vdocuments.us/reader033/viewer/2022052907/55931a081a28abdc428b459e/html5/thumbnails/42.jpg)
42
the 1st nearest image
Query image
Points: 2D keypoints
Blue: correspondence
Red: no correspondence
Blue: correspondence
Red: no correspondence
Non-change area
Uncovered area is the changed area
Estimated changed area
![Page 43: Change Detection of 3D Scene with 3D and 2D Information for Environment Checking](https://reader033.vdocuments.us/reader033/viewer/2022052907/55931a081a28abdc428b459e/html5/thumbnails/43.jpg)
3D point cloud
Visualized 3D points 43
change area
projection
Detected result
Projected 3D points
![Page 44: Change Detection of 3D Scene with 3D and 2D Information for Environment Checking](https://reader033.vdocuments.us/reader033/viewer/2022052907/55931a081a28abdc428b459e/html5/thumbnails/44.jpg)
Introduction ◦ Research Motivation ◦ Change Detection
3D Keypoints Based 3D-2D Matching ◦ Background ◦ 3D Keypoints Detection ◦ Evaluation
3D-2D Based Change Detection ◦ Background ◦ Image Based Change Detection ◦ Evaluation
3D-3D Based Change Detection ◦ Background ◦ Scale Estimation of a Single Point Cloud ◦ Scale Ratio Estimation of Two Point Clouds ◦ Evaluation
Conclusions
44
![Page 45: Change Detection of 3D Scene with 3D and 2D Information for Environment Checking](https://reader033.vdocuments.us/reader033/viewer/2022052907/55931a081a28abdc428b459e/html5/thumbnails/45.jpg)
3D point cloud Query image
45
Quantitative results visualization
Changed 3D points Changed area
![Page 46: Change Detection of 3D Scene with 3D and 2D Information for Environment Checking](https://reader033.vdocuments.us/reader033/viewer/2022052907/55931a081a28abdc428b459e/html5/thumbnails/46.jpg)
Results for different thresholds 46
Set as:0, 5, 10, 20, 30, 50, 70 and 90 pixels
0 5 10 20
30 50 70 90
Image resolution:2256x1504 The number of Image: 54 The number of 3D points: 190,845
![Page 47: Change Detection of 3D Scene with 3D and 2D Information for Environment Checking](https://reader033.vdocuments.us/reader033/viewer/2022052907/55931a081a28abdc428b459e/html5/thumbnails/47.jpg)
TP rate= True Positive Ground Positive
FP rate= Ground Negative
False Positive
47
Receiver operating characteristic (ROC) plot
threshold = 30
threshold = 30
Ground truth is set manually
We expect the 1st nearest image perform better than others, but the best result is the 2nd nearest image.
Good performance
Bad performance
![Page 48: Change Detection of 3D Scene with 3D and 2D Information for Environment Checking](https://reader033.vdocuments.us/reader033/viewer/2022052907/55931a081a28abdc428b459e/html5/thumbnails/48.jpg)
48 It is the parameter left for users.
1st
2nd
Query image Detection results
![Page 49: Change Detection of 3D Scene with 3D and 2D Information for Environment Checking](https://reader033.vdocuments.us/reader033/viewer/2022052907/55931a081a28abdc428b459e/html5/thumbnails/49.jpg)
Introduction ◦ Research Motivation ◦ Change Detection
3D Keypoints Based 3D-2D Matching ◦ Background ◦ 3D Keypoints Detection ◦ Evaluation
3D-2D Based Change Detection ◦ Background ◦ Image Based Change Detection ◦ Evaluation
3D-3D Based Change Detection ◦ Background ◦ Scale Estimation of a Single Point Cloud ◦ Scale Ratio Estimation of Two Point Clouds ◦ Evaluation
Conclusions
49
![Page 50: Change Detection of 3D Scene with 3D and 2D Information for Environment Checking](https://reader033.vdocuments.us/reader033/viewer/2022052907/55931a081a28abdc428b459e/html5/thumbnails/50.jpg)
50
Potential changed areas 3D information
Camera poses
Online system Offline system
3D-2D based change detection
3D-2D based camera pose estimation
3D-3D based change detection
![Page 51: Change Detection of 3D Scene with 3D and 2D Information for Environment Checking](https://reader033.vdocuments.us/reader033/viewer/2022052907/55931a081a28abdc428b459e/html5/thumbnails/51.jpg)
Different size scale because of the character of Structure-from-Motion (SfM)
51
3D-3D registration is actually, the scale registration
3D point cloud 3D point cloud
3D point cloud
Change points
registration
Point clouds of same scene with different size
3D point cloud 3D point cloud
![Page 52: Change Detection of 3D Scene with 3D and 2D Information for Environment Checking](https://reader033.vdocuments.us/reader033/viewer/2022052907/55931a081a28abdc428b459e/html5/thumbnails/52.jpg)
52
Iterative closest point (ICP) based alignment [Besl 1991].
-Need simple scenes -Need initial pose and scale -Not robust to clutters, occlusions and missing part
spin images [Johnson 1998],
NARF [Steder 2010], shape context [Belongie 2002], etc.
Feature based alignment
-Need appropriate neighborhood size
3D SIFT [Scovanner 2007], 3D SURF [Knopp 2010], etc.
-Not robust to clutters, occlusions and missing part
Easy data
Different data
Fixed scale
Adaptive scale
![Page 53: Change Detection of 3D Scene with 3D and 2D Information for Environment Checking](https://reader033.vdocuments.us/reader033/viewer/2022052907/55931a081a28abdc428b459e/html5/thumbnails/53.jpg)
53
1. Scale estimation
2. Scale Ratio estimation
Keyscale1=0.5 Keyscale2=0.1
Scale ratio=5
![Page 54: Change Detection of 3D Scene with 3D and 2D Information for Environment Checking](https://reader033.vdocuments.us/reader033/viewer/2022052907/55931a081a28abdc428b459e/html5/thumbnails/54.jpg)
Introduction ◦ Research Motivation ◦ Change Detection
3D Keypoints Based 3D-2D Matching ◦ Background ◦ 3D Keypoints Detection ◦ Evaluation
3D-2D Based Change Detection ◦ Background ◦ Image Based Change Detection ◦ Evaluation
3D-3D Based Change Detection ◦ Background ◦ Scale Estimation of a Single Point Cloud ◦ Scale Ratio Estimation of Two Point Clouds ◦ Evaluation
Conclusions
54
![Page 55: Change Detection of 3D Scene with 3D and 2D Information for Environment Checking](https://reader033.vdocuments.us/reader033/viewer/2022052907/55931a081a28abdc428b459e/html5/thumbnails/55.jpg)
Bunny point cloud
Width=0.001
55
Similar to each other
Different to each other
Width=0.1
Width=1.0
3D keypoints
Spin images
the minimum of similarity between spin images when the width changes.
Similar to each other
Keyscale
Robust to clutters, occlusions and missing part
Calculate similarity of collected spin images
![Page 56: Change Detection of 3D Scene with 3D and 2D Information for Environment Checking](https://reader033.vdocuments.us/reader033/viewer/2022052907/55931a081a28abdc428b459e/html5/thumbnails/56.jpg)
56
Decide which set of spin images are different to each other by using Contribution rate.
PCA Robust to order of extracted spin images.
Robust to detail
![Page 57: Change Detection of 3D Scene with 3D and 2D Information for Environment Checking](https://reader033.vdocuments.us/reader033/viewer/2022052907/55931a081a28abdc428b459e/html5/thumbnails/57.jpg)
57 minimum
1 5 10 15
sim
ilarity
sim
ilarity
d
w
Similar to each other
Different
Similar to each other
![Page 58: Change Detection of 3D Scene with 3D and 2D Information for Environment Checking](https://reader033.vdocuments.us/reader033/viewer/2022052907/55931a081a28abdc428b459e/html5/thumbnails/58.jpg)
minimum is not unique Finding them is not stable
58
minimum minimum
sim
ilarity
w
![Page 59: Change Detection of 3D Scene with 3D and 2D Information for Environment Checking](https://reader033.vdocuments.us/reader033/viewer/2022052907/55931a081a28abdc428b459e/html5/thumbnails/59.jpg)
59
Bunny point cloud
Finding minimum is not stable
sim
ilarity
w
![Page 60: Change Detection of 3D Scene with 3D and 2D Information for Environment Checking](https://reader033.vdocuments.us/reader033/viewer/2022052907/55931a081a28abdc428b459e/html5/thumbnails/60.jpg)
Introduction ◦ Research Motivation ◦ Change Detection
3D Keypoints Based 3D-2D Matching ◦ Background ◦ 3D Keypoints Detection ◦ Evaluation
3D-2D Based Change Detection ◦ Background ◦ Image Based Change Detection ◦ Evaluation
3D-3D Based Change Detection ◦ Background ◦ Scale Estimation of a Single Point Cloud ◦ Scale Ratio Estimation of Two Point Clouds ◦ Evaluation
Conclusions
60
![Page 61: Change Detection of 3D Scene with 3D and 2D Information for Environment Checking](https://reader033.vdocuments.us/reader033/viewer/2022052907/55931a081a28abdc428b459e/html5/thumbnails/61.jpg)
Point clouds
61
Register two plots to get scale ratio
Scale ratio ICP
similarity plots
![Page 62: Change Detection of 3D Scene with 3D and 2D Information for Environment Checking](https://reader033.vdocuments.us/reader033/viewer/2022052907/55931a081a28abdc428b459e/html5/thumbnails/62.jpg)
Overlapping parts
Original bunny curves 5 times larger bunny curves
62
![Page 63: Change Detection of 3D Scene with 3D and 2D Information for Environment Checking](https://reader033.vdocuments.us/reader033/viewer/2022052907/55931a081a28abdc428b459e/html5/thumbnails/63.jpg)
Original bunny curves
5 times larger bunny curves
63
![Page 64: Change Detection of 3D Scene with 3D and 2D Information for Environment Checking](https://reader033.vdocuments.us/reader033/viewer/2022052907/55931a081a28abdc428b459e/html5/thumbnails/64.jpg)
Scale ratio t
Displaced Original bunny curves
5 times larger bunny curves
64
![Page 65: Change Detection of 3D Scene with 3D and 2D Information for Environment Checking](https://reader033.vdocuments.us/reader033/viewer/2022052907/55931a081a28abdc428b459e/html5/thumbnails/65.jpg)
65
Similarity estimation
3D registration
Scale
ratio
estim
atio
n
input
Similarity plots
Scale ratio
alignment
![Page 66: Change Detection of 3D Scene with 3D and 2D Information for Environment Checking](https://reader033.vdocuments.us/reader033/viewer/2022052907/55931a081a28abdc428b459e/html5/thumbnails/66.jpg)
Introduction ◦ Research Motivation ◦ Change Detection
3D Keypoints Based 3D-2D Matching ◦ Background ◦ 3D Keypoints Detection ◦ Evaluation
3D-2D Based Change Detection ◦ Background ◦ Image Based Change Detection ◦ Evaluation
3D-3D Based Change Detection ◦ Background ◦ Scale Estimation of a Single Point Cloud ◦ Scale Ratio Estimation of Two Point Clouds ◦ Evaluation
Conclusions
66
![Page 67: Change Detection of 3D Scene with 3D and 2D Information for Environment Checking](https://reader033.vdocuments.us/reader033/viewer/2022052907/55931a081a28abdc428b459e/html5/thumbnails/67.jpg)
67
Original point cloud
The number of points:207,583 Scene size: 35m x 5m
![Page 68: Change Detection of 3D Scene with 3D and 2D Information for Environment Checking](https://reader033.vdocuments.us/reader033/viewer/2022052907/55931a081a28abdc428b459e/html5/thumbnails/68.jpg)
overlapping area
68
Created 1st point cloud
Created 2nd point cloud
estimate scale ratio
Original point cloud
![Page 69: Change Detection of 3D Scene with 3D and 2D Information for Environment Checking](https://reader033.vdocuments.us/reader033/viewer/2022052907/55931a081a28abdc428b459e/html5/thumbnails/69.jpg)
69
The method provides perfect result when the overlap rate is larger than 70%.
Ground truth = 1
![Page 70: Change Detection of 3D Scene with 3D and 2D Information for Environment Checking](https://reader033.vdocuments.us/reader033/viewer/2022052907/55931a081a28abdc428b459e/html5/thumbnails/70.jpg)
Small blocks point clouds
70
Changed block
![Page 71: Change Detection of 3D Scene with 3D and 2D Information for Environment Checking](https://reader033.vdocuments.us/reader033/viewer/2022052907/55931a081a28abdc428b459e/html5/thumbnails/71.jpg)
Introduction ◦ Research Motivation ◦ Change Detection
3D Keypoints Based 3D-2D Matching ◦ Background ◦ 3D Keypoints Detection ◦ Evaluation
3D-2D Based Change Detection ◦ Background ◦ Image Based Change Detection ◦ Evaluation
3D-3D Based Change Detection ◦ Background ◦ Scale Estimation of a Single Point Cloud ◦ Scale Ratio Estimation of Two Point Clouds ◦ Evaluation
Conclusions
71
![Page 72: Change Detection of 3D Scene with 3D and 2D Information for Environment Checking](https://reader033.vdocuments.us/reader033/viewer/2022052907/55931a081a28abdc428b459e/html5/thumbnails/72.jpg)
We have proposed three methods for a surveillance system to detect change.
2. Online 3D-2D based change detection
3. Offline 3D-3D based change detection
1. 3D-2D matching
In future: Find a more systematic way for choosing parameters. Improve computation time.
In future: Find the nearest image by considering FOVs. Implement the method on mobile devices.
In future: Accelerate the computation in order to handle much larger number of points.
72
Potential changed areas 3D information
Camera poses
Online system Offline system
3D-2D based change detection
3D-2D based camera pose estimation
3D-3D based change detection
![Page 73: Change Detection of 3D Scene with 3D and 2D Information for Environment Checking](https://reader033.vdocuments.us/reader033/viewer/2022052907/55931a081a28abdc428b459e/html5/thumbnails/73.jpg)
73
Future work:
1. Improve computation speed and detection accuracy for online system. -current computation time: 20 seconds per image
2. Optimize algorithm to operate with huge size data for offline system. -current computation time: 10 minutes for 100,000 points