tum - joint optimization of pose and depth using a prox-linear … · 2019. 12. 28. · master...
TRANSCRIPT
Problem Formulation Optimization Results
Joint Optimization of Pose And Depth Using aProx-Linear ApproachMaster Thesis Presentation
Florian Hofherr
20. Dezember 2019
Florian Hofherr
Joint Optimization of Pose And Depth Using a Prox-Linear Approach
Problem Formulation Optimization Results
1 Problem Formulation
2 Optimization
3 Results
Florian Hofherr
Joint Optimization of Pose And Depth Using a Prox-Linear Approach
Problem Formulation Optimization Results
1 Problem Formulation
2 Optimization
3 Results
Florian Hofherr
Joint Optimization of Pose And Depth Using a Prox-Linear Approach
Problem Formulation Optimization Results
SLAM Overview
X0
Y0
Z0
Scene
CameraTrajectory
Florian Hofherr
Joint Optimization of Pose And Depth Using a Prox-Linear Approach
Problem Formulation Optimization Results
Warping Between two Frames
X
X0
Y0
Z0
x0y0
0xX1
Y1
Z1
x1y1
1x
R (w) ,T
w: Exponential CoordinatesPinhole Camera: hx̃ = KX
1x = ω0x (R,T , 0h)
Florian Hofherr
Joint Optimization of Pose And Depth Using a Prox-Linear Approach
Problem Formulation Optimization Results
Photometric Data Term
X
I0
x0y0
0x
X0
Y0
Z0
I1
x1y11x
X1
Y1
Z1
Photometric → Compare image intensities
Bilinear/Bicubic interpolation in second image
Assume Lambertian surfaces
Dense → All valid pixels
Florian Hofherr
Joint Optimization of Pose And Depth Using a Prox-Linear Approach
Problem Formulation Optimization Results
Photometric Data Term
Data term for two images
Edata (w,T ,h) =∑x∈VI1
` (J I1 (ωx (R (w) ,T ,h))− I0 (x))
VI1 : Set of valid pixels
`: Loss function
J : Interpolation operator
Extendable for more images
Florian Hofherr
Joint Optimization of Pose And Depth Using a Prox-Linear Approach
Problem Formulation Optimization Results
Regularization
Isotropic Huber Regularization with image-driven weights
Ereg (h) =∑x∈P
γ (x) ‖Dh (x)‖h
P: Set of all pixels
γ (x) = e−α‖DI0(x)‖β2
D: Discrete gradient operator (forward differences)
Huber norm: ‖x‖h =
{12h ‖x‖
22 if ‖x‖2 ≤ h
‖x‖2 −h2 else
Florian Hofherr
Joint Optimization of Pose And Depth Using a Prox-Linear Approach
Problem Formulation Optimization Results
Joint Optimization Problem
Joint Optimization Problem
minw,T,h
{Edata (w,T,h) + λregEreg (h)}
Advantages
Only one optimization for pose and depth
No keypoint selection, all pixels for pose
Difficulties
Non-linear, Non-Convex
High-Dimensional
Florian Hofherr
Joint Optimization of Pose And Depth Using a Prox-Linear Approach
Problem Formulation Optimization Results
1 Problem Formulation
2 Optimization
3 Results
Florian Hofherr
Joint Optimization of Pose And Depth Using a Prox-Linear Approach
Problem Formulation Optimization Results
Composite Optimization Problem
Composite Optimization Problem
minx
F (x) := g (x) + h (c (x))
g : Rd → R and h : Rm → R proper, closed and convex
c : Rd → Rm be a C 1-smooth
Florian Hofherr
Joint Optimization of Pose And Depth Using a Prox-Linear Approach
Problem Formulation Optimization Results
Composite Optimization Problem
Composite Optimization Problem
minx
F (x) := g (x) + h (c (x))
g : Rd → R and h : Rm → R proper, closed and convex
c : Rd → Rm be a C 1-smooth
g (·) ↔ λregEreg (h) = λreg∑x∈P
γ (x) ‖Dh (x)‖h
Florian Hofherr
Joint Optimization of Pose And Depth Using a Prox-Linear Approach
Problem Formulation Optimization Results
Composite Optimization Problem
Composite Optimization Problem
minx
F (x) := g (x) + h (c (x))
g : Rd → R and h : Rm → R proper, closed and convex
c : Rd → Rm be a C 1-smooth
h (c (·)) ↔ Edata (w,T ,h) =∑x∈VI1
` (J I1 (ωx (R (w) ,T ,h))− I0 (x))
Florian Hofherr
Joint Optimization of Pose And Depth Using a Prox-Linear Approach
Problem Formulation Optimization Results
The Standard Prox-Linear Algorithm
Composite Optimization Problem
minx
F (x) := g (x) + h (c (x))
Iterative update
xk+1 = arg minx∈Rd
{g (x) + h
(c(xk)
+ Jc(xk) (
x − xk))
+1
2t
∥∥x − xk∥∥22
}
Jc(xk): Jacobian Matrix of c
Florian Hofherr
Joint Optimization of Pose And Depth Using a Prox-Linear Approach
Problem Formulation Optimization Results
The Standard Prox-Linear Algorithm
Composite Optimization Problem
minx
F (x) := g (x) + h (c (x))
Iterative update
xk+1 = arg minx∈Rd
{g (x) + h
(c(xk)
+ Jc(xk) (
x − xk))
+1
2t
∥∥x − xk∥∥22
}
Jc(xk): Jacobian Matrix of c
g ≡ 0, h = 12
∑x2i ⇒ Levenberg-Marquardt
h (x) = x ⇒ Prox-gradient algorithm
Florian Hofherr
Joint Optimization of Pose And Depth Using a Prox-Linear Approach
Problem Formulation Optimization Results
Weighted Proximal Mapping
f : Rn → R closed, proper and convex
M ∈ Rn×n symmetric positive definite matrix (→ diagonal)
Weighted norm: ‖u‖2M = 〈M−1u, u〉
proxMf (u) := arg minv∈Rn
{f (v) +
1
2‖v − u‖2M
}
Florian Hofherr
Joint Optimization of Pose And Depth Using a Prox-Linear Approach
Problem Formulation Optimization Results
The Weighted Prox-Linear Algorithm
Composite Optimization Problem
minx
F (x) := g (x) + h (c (x))
Iterative update
xk+1 = arg minx∈Rd
{g (x) + h
(c(xk)
+ Jc(xk) (
x − xk))
+1
2
∥∥x − xk∥∥2Mk
}
Step widths:(Mk)−1
= 1ζkstep
M−10 + diag(Jk
TJk)
Florian Hofherr
Joint Optimization of Pose And Depth Using a Prox-Linear Approach
Problem Formulation Optimization Results
The Weighted Prox-Linear Algorithm
Composite Optimization Problem
minx
F (x) := g (x) + h (c (x))
Iterative update
xk+1 = arg minx∈Rd
{g (x) + h
(c(xk)
+ Jc(xk) (
x − xk))
+1
2
∥∥x − xk∥∥2Mk
}
Step widths:(Mk)−1
= 1ζkstep
M−10 + diag(Jk
TJk)
Florian Hofherr
Joint Optimization of Pose And Depth Using a Prox-Linear Approach
Problem Formulation Optimization Results
Solution of the Sub-Problems
Sub-Problem
uk+1 = arg minu
{h(Jku − bk
)+ λregEreg (u) +
1
2
∥∥∥u − uk∥∥∥2Mk
}
General Case
h: Absolute loss, Huber loss
Standard TV regularization possible
Solve using preconditioned PDHG
Florian Hofherr
Joint Optimization of Pose And Depth Using a Prox-Linear Approach
Problem Formulation Optimization Results
Solution of the Sub-Problems
Sub-Problem
uk+1 = arg minu
{h(Jku − bk
)+ λregEreg (u) +
1
2
∥∥∥u − uk∥∥∥2Mk
}
Special Case: h (x) = 12 ‖x‖
2
Linearize Regularization
⇒ Analytic Solution
Mixture: Levenberg-Marquardt on data term, gradient descenton regularization
Florian Hofherr
Joint Optimization of Pose And Depth Using a Prox-Linear Approach
Problem Formulation Optimization Results
1 Problem Formulation
2 Optimization
3 Results
Florian Hofherr
Joint Optimization of Pose And Depth Using a Prox-Linear Approach
Problem Formulation Optimization Results
Experiment 1
Compare joint estimation to pure depth/pose estimation
Absolute data loss
Pure estimation: Use ground truth
New Tsukuba data set
Florian Hofherr
Joint Optimization of Pose And Depth Using a Prox-Linear Approach
Problem Formulation Optimization Results
Comparison Joint Approach vs. Pure Pose/Depth
Image Pair
Florian Hofherr
Joint Optimization of Pose And Depth Using a Prox-Linear Approach
Problem Formulation Optimization Results
Comparison Joint Approach vs. Pure Pose/Depth
True depth and initial value
100
200
300
Florian Hofherr
Joint Optimization of Pose And Depth Using a Prox-Linear Approach
Problem Formulation Optimization Results
Comparison Joint Approach vs. Pure Pose/Depth
Result Joint Optimization
100
200
300
−1
−0.5
0
0.5
1
Florian Hofherr
Joint Optimization of Pose And Depth Using a Prox-Linear Approach
Problem Formulation Optimization Results
Comparison Joint Approach vs. Pure Pose/Depth
Result Pure Depth Estimation
100
200
300
−1
−0.5
0
0.5
1
Florian Hofherr
Joint Optimization of Pose And Depth Using a Prox-Linear Approach
Problem Formulation Optimization Results
Comparison Joint Approach vs. Pure Pose/Depth
Comparison Results Pose
blue: joint optimization
orange: pure pose optimization
0 10 20 300
1
2
3
Iterations
δ win
[◦]
0 10 20 300
0.05
0.1
0.15
Iterations
δ T
Florian Hofherr
Joint Optimization of Pose And Depth Using a Prox-Linear Approach
Problem Formulation Optimization Results
Experiment 2
Compare data loss functions
Absolute loss and special case (quadratic data loss)
Reduced step widths for special case
Florian Hofherr
Joint Optimization of Pose And Depth Using a Prox-Linear Approach
Problem Formulation Optimization Results
Comparison Absolute Loss vs. Special Case
Image Pair
Florian Hofherr
Joint Optimization of Pose And Depth Using a Prox-Linear Approach
Problem Formulation Optimization Results
Comparison Absolute Loss vs. Special Case
Result Absolute Loss
100
200
300
−1
−0.5
0
0.5
1
Florian Hofherr
Joint Optimization of Pose And Depth Using a Prox-Linear Approach
Problem Formulation Optimization Results
Comparison Absolute Loss vs. Special Case
Result Special Case
100
200
300
−1
−0.5
0
0.5
1
Florian Hofherr
Joint Optimization of Pose And Depth Using a Prox-Linear Approach
Problem Formulation Optimization Results
Comparison Absolute Loss vs. Special Case
Comparison Results Pose
blue: absolute loss
yellow: special case (quadratic loss)
0 50 100 150 2000
1
2
Iterations
δ win
[◦]
0 50 100 150 2000
0.02
0.04
0.06
0.08
0.1
Iterations
δ T
Florian Hofherr
Joint Optimization of Pose And Depth Using a Prox-Linear Approach
Problem Formulation Optimization Results
Comparison Absolute Loss vs. Special Case
Energy absolute loss
green: data part
cyan: Regularization part
0 20 40 60 80 100 120 140 160 180 200 220
105
106
107
Iterations
En
ergy
Florian Hofherr
Joint Optimization of Pose And Depth Using a Prox-Linear Approach
Problem Formulation Optimization Results
Comparison Absolute Loss vs. Special Case
Energy special case (quadratic loss)
green: data part
cyan: Regularization part
0 20 40 60 80 100 120 140 160 180 200 220106
107
108
Iterations
En
ergy
Florian Hofherr
Joint Optimization of Pose And Depth Using a Prox-Linear Approach
Problem Formulation Optimization Results
Questions?
Florian Hofherr
Joint Optimization of Pose And Depth Using a Prox-Linear Approach
Problem Formulation Optimization Results
Thank YouAnd
Merry Christmas
Florian Hofherr
Joint Optimization of Pose And Depth Using a Prox-Linear Approach