lecture 8: registration - stanford...
TRANSCRIPT
![Page 1: Lecture 8: Registration - Stanford Universityweb.stanford.edu/class/me328/lectures/lecture8-registration.pdf · K. S. Arun, T. S. Huang, S. D. Blostein. Least-Squares Fitting of Two](https://reader036.vdocuments.us/reader036/viewer/2022090607/605caaa5190050108f3f3ff6/html5/thumbnails/1.jpg)
ME 328: Medical Robotics Winter 2019
Lecture 8:Registration
Allison OkamuraStanford University
![Page 2: Lecture 8: Registration - Stanford Universityweb.stanford.edu/class/me328/lectures/lecture8-registration.pdf · K. S. Arun, T. S. Huang, S. D. Blostein. Least-Squares Fitting of Two](https://reader036.vdocuments.us/reader036/viewer/2022090607/605caaa5190050108f3f3ff6/html5/thumbnails/2.jpg)
Updates
Assignment 4
Sign up for teams/ultrasound by noon today at:
https://tinyurl.com/ME328-USlab
Main parts:Readings/questions
Image acquisition and analysis (ultrasound lab)Image registration
Needle insertion (robot control)
Remainder of assignment will be posted this morninghttps://tinyurl.com/ME328-USlab
![Page 3: Lecture 8: Registration - Stanford Universityweb.stanford.edu/class/me328/lectures/lecture8-registration.pdf · K. S. Arun, T. S. Huang, S. D. Blostein. Least-Squares Fitting of Two](https://reader036.vdocuments.us/reader036/viewer/2022090607/605caaa5190050108f3f3ff6/html5/thumbnails/3.jpg)
registrationthe alignment of multiple data sets into a single
coordinate system such that the spatial locations of corresponding points coincide
i.e., establish a quantitative relationship between different reference frames
essential for the quantitative integration of various sources of information:1. pre-, intra-, and post-operative images2. trackers (tracking data not useful unless registered)3. robots (cannot use images unless registered)
Image-Guided Procedures: A Review, by Ziv Yaniv and Kevin Cleary (2006)
![Page 4: Lecture 8: Registration - Stanford Universityweb.stanford.edu/class/me328/lectures/lecture8-registration.pdf · K. S. Arun, T. S. Huang, S. D. Blostein. Least-Squares Fitting of Two](https://reader036.vdocuments.us/reader036/viewer/2022090607/605caaa5190050108f3f3ff6/html5/thumbnails/4.jpg)
Copyright L. Joskowicz, 2011
Registration examplesexamples
L. Joskowicz 2011c�
![Page 5: Lecture 8: Registration - Stanford Universityweb.stanford.edu/class/me328/lectures/lecture8-registration.pdf · K. S. Arun, T. S. Huang, S. D. Blostein. Least-Squares Fitting of Two](https://reader036.vdocuments.us/reader036/viewer/2022090607/605caaa5190050108f3f3ff6/html5/thumbnails/5.jpg)
suitability for image-guided procedures
• accuracy: target registration error (TRE) quantifies how far the predicted position of the anatomical target is from its actual position~1 mm typically satisfactory
• speed: how long does it take the algorithm to produce the solutionintraoperatively, need on the order of seconds to minutes
• robustness: how well does the algorithm deal with noise and outliersdepends on how noisy the data is
![Page 6: Lecture 8: Registration - Stanford Universityweb.stanford.edu/class/me328/lectures/lecture8-registration.pdf · K. S. Arun, T. S. Huang, S. D. Blostein. Least-Squares Fitting of Two](https://reader036.vdocuments.us/reader036/viewer/2022090607/605caaa5190050108f3f3ff6/html5/thumbnails/6.jpg)
navigation and registration example
http://www.youtube.com/watch?v=jYCiKOERYD8http://www.youtube.com/watch?v=c58etXWWoDA
![Page 7: Lecture 8: Registration - Stanford Universityweb.stanford.edu/class/me328/lectures/lecture8-registration.pdf · K. S. Arun, T. S. Huang, S. D. Blostein. Least-Squares Fitting of Two](https://reader036.vdocuments.us/reader036/viewer/2022090607/605caaa5190050108f3f3ff6/html5/thumbnails/7.jpg)
a typical robot registration problem
Preoperative model from a CT scan gives the location of a tumor in
the brain...
Now we are in the OR and
want to operate with a robot based on this pre-operative
imageregistration example and graphics provided by Russell H. Taylor, JHU
![Page 8: Lecture 8: Registration - Stanford Universityweb.stanford.edu/class/me328/lectures/lecture8-registration.pdf · K. S. Arun, T. S. Huang, S. D. Blostein. Least-Squares Fitting of Two](https://reader036.vdocuments.us/reader036/viewer/2022090607/605caaa5190050108f3f3ff6/html5/thumbnails/8.jpg)
Russell H. Taylor, 2011c�
preoperative model
intraoperative reality
a typical robot registration problem
What is ?Treg
vptrvCT = Tregvptr
Treg
![Page 9: Lecture 8: Registration - Stanford Universityweb.stanford.edu/class/me328/lectures/lecture8-registration.pdf · K. S. Arun, T. S. Huang, S. D. Blostein. Least-Squares Fitting of Two](https://reader036.vdocuments.us/reader036/viewer/2022090607/605caaa5190050108f3f3ff6/html5/thumbnails/9.jpg)
coordinate systems and transformations
{A}xA
yA
zA
e.g., robot basecoordinate system
e.g., image or patient coordinate
system
{B}
zB
yB
xB
T
T�1
![Page 10: Lecture 8: Registration - Stanford Universityweb.stanford.edu/class/me328/lectures/lecture8-registration.pdf · K. S. Arun, T. S. Huang, S. D. Blostein. Least-Squares Fitting of Two](https://reader036.vdocuments.us/reader036/viewer/2022090607/605caaa5190050108f3f3ff6/html5/thumbnails/10.jpg)
{A}xA
yA
zA
e.g., robot basecoordinate system
e.g., image or patient coordinate
system
{B}
zB
yB
xB
T
T�1
vA
vB
vA = F (vB)
vA = TvB
vA = RvB + p
T (R, p) =
R3⇥3 p3⇥1
0 0 0 1
�
coordinate systems and transformations
![Page 11: Lecture 8: Registration - Stanford Universityweb.stanford.edu/class/me328/lectures/lecture8-registration.pdf · K. S. Arun, T. S. Huang, S. D. Blostein. Least-Squares Fitting of Two](https://reader036.vdocuments.us/reader036/viewer/2022090607/605caaa5190050108f3f3ff6/html5/thumbnails/11.jpg)
{A}
{B}
vA
vB
vA = F (vB)
vA = TvB
vA = RvB + p
T (R, p) =
R3⇥3 p3⇥1
0 0 0 1
�
first rotate
coordinate systems and transformations
![Page 12: Lecture 8: Registration - Stanford Universityweb.stanford.edu/class/me328/lectures/lecture8-registration.pdf · K. S. Arun, T. S. Huang, S. D. Blostein. Least-Squares Fitting of Two](https://reader036.vdocuments.us/reader036/viewer/2022090607/605caaa5190050108f3f3ff6/html5/thumbnails/12.jpg)
{A}
{B}
vA
vB
vA = F (vB)
vA = TvB
vA = RvB + p
T (R, p) =
R3⇥3 p3⇥1
0 0 0 1
�
then translate
coordinate systems and transformations
![Page 13: Lecture 8: Registration - Stanford Universityweb.stanford.edu/class/me328/lectures/lecture8-registration.pdf · K. S. Arun, T. S. Huang, S. D. Blostein. Least-Squares Fitting of Two](https://reader036.vdocuments.us/reader036/viewer/2022090607/605caaa5190050108f3f3ff6/html5/thumbnails/13.jpg)
interesting properties of T and R
forward transformation
vA = RvB + p
vA = TvB
inverse transformation
T�1vA = vBvB = R�1(vA � p)
vB = R�1vA �R�1p
composition of transformations
T3 = T1T2
R3 = R1R2
p3 = R1p2 + p1rotation matrices are orthogonal matrices
RT = R�1
RTR = RRT = I
and members of the group SO(3)
det(R) = +1
![Page 14: Lecture 8: Registration - Stanford Universityweb.stanford.edu/class/me328/lectures/lecture8-registration.pdf · K. S. Arun, T. S. Huang, S. D. Blostein. Least-Squares Fitting of Two](https://reader036.vdocuments.us/reader036/viewer/2022090607/605caaa5190050108f3f3ff6/html5/thumbnails/14.jpg)
FGH = F�1BG
FBEFEH
FB
FC FD
FH
FU
FG
FE
FCA
FDU
FUA
FGC
FBG
FBE
FBD
FEH
FCH
Russell H. Taylor, 2011c�
FGH?
composition of transformations
![Page 15: Lecture 8: Registration - Stanford Universityweb.stanford.edu/class/me328/lectures/lecture8-registration.pdf · K. S. Arun, T. S. Huang, S. D. Blostein. Least-Squares Fitting of Two](https://reader036.vdocuments.us/reader036/viewer/2022090607/605caaa5190050108f3f3ff6/html5/thumbnails/15.jpg)
feature-based registration
Russell H. Taylor, 2011c�
preoperative model
intraoperative reality
trackedpointer/robot end-effector
![Page 16: Lecture 8: Registration - Stanford Universityweb.stanford.edu/class/me328/lectures/lecture8-registration.pdf · K. S. Arun, T. S. Huang, S. D. Blostein. Least-Squares Fitting of Two](https://reader036.vdocuments.us/reader036/viewer/2022090607/605caaa5190050108f3f3ff6/html5/thumbnails/16.jpg)
• Point fiducials (markers)
• Point anatomical landmarks
• Ridge curves
• Contours
• Surfaces
• Line fiducials
features used for feature-based registration
given two features (either the same features in two different reference frames, of two different features in the same reference
frame), we need to know the “distance” between them
![Page 17: Lecture 8: Registration - Stanford Universityweb.stanford.edu/class/me328/lectures/lecture8-registration.pdf · K. S. Arun, T. S. Huang, S. D. Blostein. Least-Squares Fitting of Two](https://reader036.vdocuments.us/reader036/viewer/2022090607/605caaa5190050108f3f3ff6/html5/thumbnails/17.jpg)
what the computer knows
Russell H. Taylor, 2011c�
preoperative model
intraoperative reality
![Page 18: Lecture 8: Registration - Stanford Universityweb.stanford.edu/class/me328/lectures/lecture8-registration.pdf · K. S. Arun, T. S. Huang, S. D. Blostein. Least-Squares Fitting of Two](https://reader036.vdocuments.us/reader036/viewer/2022090607/605caaa5190050108f3f3ff6/html5/thumbnails/18.jpg)
identify corresponding points
Russell H. Taylor, 2011c�
preoperative model
intraoperative reality
![Page 19: Lecture 8: Registration - Stanford Universityweb.stanford.edu/class/me328/lectures/lecture8-registration.pdf · K. S. Arun, T. S. Huang, S. D. Blostein. Least-Squares Fitting of Two](https://reader036.vdocuments.us/reader036/viewer/2022090607/605caaa5190050108f3f3ff6/html5/thumbnails/19.jpg)
how to identify correspondences
manualvs.
part of the optimization/minimization process
ak
?
bi{ }
... but we will assume manual is possible
state of the art is the “iterative closest point”
(ICP) algorithm
![Page 20: Lecture 8: Registration - Stanford Universityweb.stanford.edu/class/me328/lectures/lecture8-registration.pdf · K. S. Arun, T. S. Huang, S. D. Blostein. Least-Squares Fitting of Two](https://reader036.vdocuments.us/reader036/viewer/2022090607/605caaa5190050108f3f3ff6/html5/thumbnails/20.jpg)
find the best rigid transformation
Russell H. Taylor, 2011c�
preoperative model
intraoperative reality
minTreg
X
i
wiD(Tregai, bi)
a disparity functionweighting depending
on which points are being considered
minimize over all possible values of Treg
![Page 21: Lecture 8: Registration - Stanford Universityweb.stanford.edu/class/me328/lectures/lecture8-registration.pdf · K. S. Arun, T. S. Huang, S. D. Blostein. Least-Squares Fitting of Two](https://reader036.vdocuments.us/reader036/viewer/2022090607/605caaa5190050108f3f3ff6/html5/thumbnails/21.jpg)
disparity function, D
a metric for the error between two feature sets
minTreg
X
i
wiD(Tregai, bi)
a disparity functionweighting depending
on which points are being considered
minimize over all possible values of Treg
minTreg
X
i
wi||(Tregai � bi)||2sum of squares of residuals is common:
other D possibilities include: maximum distancemedian distancecardinality depending in threshold
![Page 22: Lecture 8: Registration - Stanford Universityweb.stanford.edu/class/me328/lectures/lecture8-registration.pdf · K. S. Arun, T. S. Huang, S. D. Blostein. Least-Squares Fitting of Two](https://reader036.vdocuments.us/reader036/viewer/2022090607/605caaa5190050108f3f3ff6/html5/thumbnails/22.jpg)
how to do the optimization for rigid registration
Options:• global vs. local• numerical vs. direct (analytical)• ways of dealing with local minima
Given points in two different coordinate systems(e.g., a set of points and a set of points ){ai} {bi}
Find the transformation matrix T (R, p)
That minimizes
Where ei = (Rai + p)� bi
X
i
eTi ei
this is tricky because of R
![Page 23: Lecture 8: Registration - Stanford Universityweb.stanford.edu/class/me328/lectures/lecture8-registration.pdf · K. S. Arun, T. S. Huang, S. D. Blostein. Least-Squares Fitting of Two](https://reader036.vdocuments.us/reader036/viewer/2022090607/605caaa5190050108f3f3ff6/html5/thumbnails/23.jpg)
minimizing registration errors
Step 1: compute means and residuals of known points
a =1
N
NX
i=1
ai
ai = ai � a
b =1
N
NX
i=1
bi
bi = bi � b
Step 2: Find that minimizesRX
i
(Rai � bi)2
Step 3: Find p
Step 4: Desired transformation is
p = b�Ra
T (R, p) =
R p
0 0 0 1
�
How???There are4 substeps
withinthis step
![Page 24: Lecture 8: Registration - Stanford Universityweb.stanford.edu/class/me328/lectures/lecture8-registration.pdf · K. S. Arun, T. S. Huang, S. D. Blostein. Least-Squares Fitting of Two](https://reader036.vdocuments.us/reader036/viewer/2022090607/605caaa5190050108f3f3ff6/html5/thumbnails/24.jpg)
minimization substeps: direct method(there is also an iterative method)
K. S. Arun, T. S. Huang, S. D. Blostein. Least-Squares Fitting of Two 3-D Point Sets. IEEE Transactions on Pattern Analysis and Machine Intelligence, 9(5): 698-700, 1987.
Step 1: Compute
H =
!ai,x!bi,x
!ai,x!bi,y
!ai,x!bi,z
!ai,y!bi,x
!ai,y!bi,y
!ai,y!bi,z
!ai,z!bi,x
!ai,z!bi,y
!ai,z!bi,z
⎡
⎣
⎢⎢⎢⎢
⎤
⎦
⎥⎥⎥⎥
i∑
Step 2: Compute the SVD of H = USVt
Step 3: R = VUt
Step 4: Verify Det(R) = 1. If not, then algorithm may fail.
outer productbetween a and b
![Page 25: Lecture 8: Registration - Stanford Universityweb.stanford.edu/class/me328/lectures/lecture8-registration.pdf · K. S. Arun, T. S. Huang, S. D. Blostein. Least-Squares Fitting of Two](https://reader036.vdocuments.us/reader036/viewer/2022090607/605caaa5190050108f3f3ff6/html5/thumbnails/25.jpg)
Extra Slides:An Iterative Method for
Finding the Rotation Matrix
(I don’t recommend using this for your assignment, though. The SVD method is
much simpler.)
![Page 26: Lecture 8: Registration - Stanford Universityweb.stanford.edu/class/me328/lectures/lecture8-registration.pdf · K. S. Arun, T. S. Huang, S. D. Blostein. Least-Squares Fitting of Two](https://reader036.vdocuments.us/reader036/viewer/2022090607/605caaa5190050108f3f3ff6/html5/thumbnails/26.jpg)
Step 0: Make an initial guess
Step 1: Given , compute
Step 2: Compute that minimizes
Step 3: Set
Step 4: Iterate Steps 1-3 until residual error is sufficiently small (or other termination condition)
R = arg minX
i
(Rai � bi)2
Goal: given paired point sets and , find{ai} {bi}
R0
Rk bi = R�1k bi
�RX
i
(�Rai � bi)2
Rk+1 = Rk�R
iterative method: solving for R
![Page 27: Lecture 8: Registration - Stanford Universityweb.stanford.edu/class/me328/lectures/lecture8-registration.pdf · K. S. Arun, T. S. Huang, S. D. Blostein. Least-Squares Fitting of Two](https://reader036.vdocuments.us/reader036/viewer/2022090607/605caaa5190050108f3f3ff6/html5/thumbnails/27.jpg)
more mathematical preliminaries
matrix representation of cross product
a =⇥ax ay az
⇤T
a⇥ v = skew(a)v
skew(a) =
2
40 �az ayaz 0 �ax�ay ax 0
3
5
representations/constructions of rotation matrices
angle-axis: is a rotation about axisrot(a,↵) ↵ a
Rodrigues’ formula: c = rot(a,↵)b = b cos(↵) + a⇥ b sin(↵) + a(aT b)(1� cos(↵))
exponential: rot(a,↵) = eskew(a)↵ = I + ↵skew(a) +↵2
2!skew(a)2 + . . .
rot(a,↵) ⇡ I + ↵skew(a)
more notation: rot(a,↵) = Ra(↵) R(a) = rot(a, ||a||)
![Page 28: Lecture 8: Registration - Stanford Universityweb.stanford.edu/class/me328/lectures/lecture8-registration.pdf · K. S. Arun, T. S. Huang, S. D. Blostein. Least-Squares Fitting of Two](https://reader036.vdocuments.us/reader036/viewer/2022090607/605caaa5190050108f3f3ff6/html5/thumbnails/28.jpg)
“small” transformationsuseful for linear approximations to
�R
Ra(�↵)
rot(a, ||a||)b ⇡ a⇥ b+ b ||a||
�R(a)
�R(�a)�R(µb) ⇡ �R(�a+ µb)
a small rotation
a rotation by a small angle about axis�↵ a
for sufficiently small
a rotation that is small enough so that any errorintroduced by this approximation is negligible
linearity for small rotations
exercise: work out the linearity proposition by substitution
represent small pose shifts �Tv = �Rv +�p
![Page 29: Lecture 8: Registration - Stanford Universityweb.stanford.edu/class/me328/lectures/lecture8-registration.pdf · K. S. Arun, T. S. Huang, S. D. Blostein. Least-Squares Fitting of Two](https://reader036.vdocuments.us/reader036/viewer/2022090607/605caaa5190050108f3f3ff6/html5/thumbnails/29.jpg)
“small” transformations
�Tv = �R(a)v +�p
�Tv ⇡ v + a⇥ v +�p
a⇥ v = skew(a)v =
2
40 �az ayaz 0 �ax�ay ax 0
3
5
2
4vxvyvz
3
5
skew(a)a = 0
�R(a) ⇡ I + skew(a)
�R(a)�1 ⇡ I � skew(a) = I + skew(�a)
![Page 30: Lecture 8: Registration - Stanford Universityweb.stanford.edu/class/me328/lectures/lecture8-registration.pdf · K. S. Arun, T. S. Huang, S. D. Blostein. Least-Squares Fitting of Two](https://reader036.vdocuments.us/reader036/viewer/2022090607/605caaa5190050108f3f3ff6/html5/thumbnails/30.jpg)
iterative method: solving for
Step 0: Make an initial guess
Step 1: Given , compute
Step 2: Compute that minimizes
Step 3: Set
Step 4: Iterate Steps 1-3 until residual error is sufficiently small (or other termination condition)
R = arg minX
i
(Rai � bi)2
Goal: given paired point sets and , find{ai} {bi}
R0
Rk bi = R�1k bi
�RX
i
(�Rai � bi)2
Rk+1 = Rk�R
R
![Page 31: Lecture 8: Registration - Stanford Universityweb.stanford.edu/class/me328/lectures/lecture8-registration.pdf · K. S. Arun, T. S. Huang, S. D. Blostein. Least-Squares Fitting of Two](https://reader036.vdocuments.us/reader036/viewer/2022090607/605caaa5190050108f3f3ff6/html5/thumbnails/31.jpg)
iterative method: solving for
Approximate as
�R
�R (I + skew(↵))
Which is equivalent to �Rv ⇡ v + ↵⇥ v
remember: multiplying by a skew-symmetric matrix is equivalent to taking cross product
Then our least squares problem becomes
min�R
X
i
(�Rai � bi)2 ⇡ min
↵
X
i
(ai � bi + ↵⇥ ai)2
This is a linear least squares problem in ↵
Then compute �R(↵)