medical image registration kumar rajamani. registration spatial transform that maps points from one...
TRANSCRIPT
![Page 1: Medical Image Registration Kumar Rajamani. Registration Spatial transform that maps points from one image to corresponding points in another image](https://reader035.vdocuments.us/reader035/viewer/2022062409/56649c7b5503460f9492f551/html5/thumbnails/1.jpg)
Medical Image Registration
Kumar Rajamani
![Page 2: Medical Image Registration Kumar Rajamani. Registration Spatial transform that maps points from one image to corresponding points in another image](https://reader035.vdocuments.us/reader035/viewer/2022062409/56649c7b5503460f9492f551/html5/thumbnails/2.jpg)
Registration
Spatial transform that maps points from one image to corresponding points in another image
![Page 3: Medical Image Registration Kumar Rajamani. Registration Spatial transform that maps points from one image to corresponding points in another image](https://reader035.vdocuments.us/reader035/viewer/2022062409/56649c7b5503460f9492f551/html5/thumbnails/3.jpg)
xy
zTime axis: aging, development…
………
… … …
Anatomy
Label
Function
Pathology,Histology…
Cross-subject, same modality
Sam
e s
ub
ject,
cro
ss m
od
ality
…
Commoncoordinatesystem
Image Registration
Data Fusion
Atlases for population studies
Motion Correction for image reconstruction
Surgery Planning
![Page 4: Medical Image Registration Kumar Rajamani. Registration Spatial transform that maps points from one image to corresponding points in another image](https://reader035.vdocuments.us/reader035/viewer/2022062409/56649c7b5503460f9492f551/html5/thumbnails/4.jpg)
Affine Scale / Skew
Example Registration Transformations
Rigid Deformable
SimilarityRigid + Scale
Original
![Page 5: Medical Image Registration Kumar Rajamani. Registration Spatial transform that maps points from one image to corresponding points in another image](https://reader035.vdocuments.us/reader035/viewer/2022062409/56649c7b5503460f9492f551/html5/thumbnails/5.jpg)
5
Image comparison
Difference beforeregistration
Difference afterregistration
Registeredmoving image
![Page 6: Medical Image Registration Kumar Rajamani. Registration Spatial transform that maps points from one image to corresponding points in another image](https://reader035.vdocuments.us/reader035/viewer/2022062409/56649c7b5503460f9492f551/html5/thumbnails/6.jpg)
Before Registration
B-Splines based Registration
After Registration
*GE Owned
![Page 7: Medical Image Registration Kumar Rajamani. Registration Spatial transform that maps points from one image to corresponding points in another image](https://reader035.vdocuments.us/reader035/viewer/2022062409/56649c7b5503460f9492f551/html5/thumbnails/7.jpg)
B-Splines based Registration (Hill et al.)
![Page 8: Medical Image Registration Kumar Rajamani. Registration Spatial transform that maps points from one image to corresponding points in another image](https://reader035.vdocuments.us/reader035/viewer/2022062409/56649c7b5503460f9492f551/html5/thumbnails/8.jpg)
Dense NR Registration (Hill et al.)
![Page 9: Medical Image Registration Kumar Rajamani. Registration Spatial transform that maps points from one image to corresponding points in another image](https://reader035.vdocuments.us/reader035/viewer/2022062409/56649c7b5503460f9492f551/html5/thumbnails/9.jpg)
Registration Criteria
Landmark-based– Features selected by the user
Segmentation-based– Rigidly or deformably align binary
structures Intensity-based
– Minimize intensity difference over entire image
![Page 10: Medical Image Registration Kumar Rajamani. Registration Spatial transform that maps points from one image to corresponding points in another image](https://reader035.vdocuments.us/reader035/viewer/2022062409/56649c7b5503460f9492f551/html5/thumbnails/10.jpg)
Spatial Transformation
Rigid– Rotations and translations
Affine– Also, skew and scaling
Deformable– Free-form mapping
![Page 11: Medical Image Registration Kumar Rajamani. Registration Spatial transform that maps points from one image to corresponding points in another image](https://reader035.vdocuments.us/reader035/viewer/2022062409/56649c7b5503460f9492f551/html5/thumbnails/11.jpg)
Registration framework pieces
2 input images, fixed and moving Metric - determines the “fitness” of the
current registration iteration Optimizer - adjusts the transform in an
attempt to improve the metric Interpolator - applies transform to image
and computes sub-pixel values
11
![Page 12: Medical Image Registration Kumar Rajamani. Registration Spatial transform that maps points from one image to corresponding points in another image](https://reader035.vdocuments.us/reader035/viewer/2022062409/56649c7b5503460f9492f551/html5/thumbnails/12.jpg)
Registration — General Problem
Problem: Find transform T such that image f[p] matches m[T(p)]
Domain ofan individual
subject
Commoncoordinatesystem
f m
)( pq T
transformmetricoptimizer
)]([m],[fminarg ppd T
T
Interpolate
![Page 13: Medical Image Registration Kumar Rajamani. Registration Spatial transform that maps points from one image to corresponding points in another image](https://reader035.vdocuments.us/reader035/viewer/2022062409/56649c7b5503460f9492f551/html5/thumbnails/13.jpg)
Registration Framework
![Page 14: Medical Image Registration Kumar Rajamani. Registration Spatial transform that maps points from one image to corresponding points in another image](https://reader035.vdocuments.us/reader035/viewer/2022062409/56649c7b5503460f9492f551/html5/thumbnails/14.jpg)
Transforms
x’=T(x|p)=T(x,y|tx,ty,θ)
Goal: Find parameter values that optimize image similarity metric
y
x
t
t
y
x
y
xx
cossin
sincos
'
''
![Page 15: Medical Image Registration Kumar Rajamani. Registration Spatial transform that maps points from one image to corresponding points in another image](https://reader035.vdocuments.us/reader035/viewer/2022062409/56649c7b5503460f9492f551/html5/thumbnails/15.jpg)
Optimizer
Often require derivative of image similarity metric (S)
j j
j
ji
x
x
MFSMFS
p
p
p
p '
'
),,|(),,|(
m
nn
m
m
p
x
p
x
p
x
p
x
p
x
p
xp
x
p
x
p
x
J
'''
'''
'''
1
1
1
2
2
2
1
2
1
2
1
1
1
![Page 16: Medical Image Registration Kumar Rajamani. Registration Spatial transform that maps points from one image to corresponding points in another image](https://reader035.vdocuments.us/reader035/viewer/2022062409/56649c7b5503460f9492f551/html5/thumbnails/16.jpg)
Understanding the Transform Jacobian
J shows how changing p shifts a transformed point in the moving image space.
This allows efficient use of a pre-computed moving-image gradient to infer changes in corresponding-pixel intensities for changes in p
Now we can update dS/dp by just updating J
16
![Page 17: Medical Image Registration Kumar Rajamani. Registration Spatial transform that maps points from one image to corresponding points in another image](https://reader035.vdocuments.us/reader035/viewer/2022062409/56649c7b5503460f9492f551/html5/thumbnails/17.jpg)
Jacobian and Image Gradient
![Page 18: Medical Image Registration Kumar Rajamani. Registration Spatial transform that maps points from one image to corresponding points in another image](https://reader035.vdocuments.us/reader035/viewer/2022062409/56649c7b5503460f9492f551/html5/thumbnails/18.jpg)
Transforms
Before we discuss specific transforms, let’s discuss the…
Fixed Set = the set of points (i.e. physical coordinates) that are unchanged by the transform
The fixed set is a very important property of a transform
18
![Page 19: Medical Image Registration Kumar Rajamani. Registration Spatial transform that maps points from one image to corresponding points in another image](https://reader035.vdocuments.us/reader035/viewer/2022062409/56649c7b5503460f9492f551/html5/thumbnails/19.jpg)
Identity Transform
Maps every point to itself Only used for testing Fixed set = everything (i.e., the entire
space)
![Page 20: Medical Image Registration Kumar Rajamani. Registration Spatial transform that maps points from one image to corresponding points in another image](https://reader035.vdocuments.us/reader035/viewer/2022062409/56649c7b5503460f9492f551/html5/thumbnails/20.jpg)
Translation Transform Fixed set = empty set Translation can be closely approximated by:
– Small rotation about distant origin, and/or…– Small scale about distant origin– Both of these do have a fixed point
Optimizers will frequently (accidently) do translation by using either rotation or scale– This makes the optimization space harder to use– The final transform may be harder to understand
20
![Page 21: Medical Image Registration Kumar Rajamani. Registration Spatial transform that maps points from one image to corresponding points in another image](https://reader035.vdocuments.us/reader035/viewer/2022062409/56649c7b5503460f9492f551/html5/thumbnails/21.jpg)
Translation Transform
Fixed set is an empty set
![Page 22: Medical Image Registration Kumar Rajamani. Registration Spatial transform that maps points from one image to corresponding points in another image](https://reader035.vdocuments.us/reader035/viewer/2022062409/56649c7b5503460f9492f551/html5/thumbnails/22.jpg)
Scaling Transform
Isotropic vs. anisotropic Fixed set is the origin of the coordinates
C
C
![Page 23: Medical Image Registration Kumar Rajamani. Registration Spatial transform that maps points from one image to corresponding points in another image](https://reader035.vdocuments.us/reader035/viewer/2022062409/56649c7b5503460f9492f551/html5/thumbnails/23.jpg)
2D Rotation Transform
Rotation transforms are typically specific to either 2D or 3D
Fixed set = origin = “center” = C
24
C
C
![Page 24: Medical Image Registration Kumar Rajamani. Registration Spatial transform that maps points from one image to corresponding points in another image](https://reader035.vdocuments.us/reader035/viewer/2022062409/56649c7b5503460f9492f551/html5/thumbnails/24.jpg)
Rotations in Polar Coordinates
y
x
y
xx
cossin
sincos
'
''
iii
i i
erereyx
rrreyx
)()','(
)sin,cos(),(
![Page 25: Medical Image Registration Kumar Rajamani. Registration Spatial transform that maps points from one image to corresponding points in another image](https://reader035.vdocuments.us/reader035/viewer/2022062409/56649c7b5503460f9492f551/html5/thumbnails/25.jpg)
Transforms
Affine TransformS=Id reduces to Rigid case,
(e.g. Head motion correction)
Affine Transforms also encode scale/skew factors
tAx
txRSxT tS
.],,[
Scaling Rotation Translation
Original
Rigid
![Page 26: Medical Image Registration Kumar Rajamani. Registration Spatial transform that maps points from one image to corresponding points in another image](https://reader035.vdocuments.us/reader035/viewer/2022062409/56649c7b5503460f9492f551/html5/thumbnails/26.jpg)
Optimization
Search for value of θ that minimizes cost function S
Gradient descent algorithm– Update of parameter
– G is the variation from the gradient of the cost function
is step length of algorithm
)exp(
'
' Geeee
S
Si
iii
![Page 27: Medical Image Registration Kumar Rajamani. Registration Spatial transform that maps points from one image to corresponding points in another image](https://reader035.vdocuments.us/reader035/viewer/2022062409/56649c7b5503460f9492f551/html5/thumbnails/27.jpg)
Sequence of translations and metric values at each iteration of the optimizer
![Page 28: Medical Image Registration Kumar Rajamani. Registration Spatial transform that maps points from one image to corresponding points in another image](https://reader035.vdocuments.us/reader035/viewer/2022062409/56649c7b5503460f9492f551/html5/thumbnails/28.jpg)
Scaling, Rotation,Translation
P=arbitrary point C=fixed point of transformation D=scaling factor Θ=rotation angle P and C are complex numbers (x+iy) or reiθ
Store derivates of P in Jacobian matrix for optimizer
Rigid if D=1, otherwise similarity transform
CDeCPPTP i)()('
![Page 29: Medical Image Registration Kumar Rajamani. Registration Spatial transform that maps points from one image to corresponding points in another image](https://reader035.vdocuments.us/reader035/viewer/2022062409/56649c7b5503460f9492f551/html5/thumbnails/29.jpg)
Affine Transformation
Collinearity is preserved x’=A x + T A is a complex matrix of coefficients With fixed point
– x’=A (x–C) + C A is optimized similar to the scaling
factor
![Page 30: Medical Image Registration Kumar Rajamani. Registration Spatial transform that maps points from one image to corresponding points in another image](https://reader035.vdocuments.us/reader035/viewer/2022062409/56649c7b5503460f9492f551/html5/thumbnails/30.jpg)
Quaternions
Quotient of two vectors– Q= A / B
Operator that produces second vector– A= Q B
Represents orientation of one vector with respect to another, as well as ratio of their magnitudes– Versor-rotates vector– Tensor-changes vector magnitude
![Page 31: Medical Image Registration Kumar Rajamani. Registration Spatial transform that maps points from one image to corresponding points in another image](https://reader035.vdocuments.us/reader035/viewer/2022062409/56649c7b5503460f9492f551/html5/thumbnails/31.jpg)
Scalars and Versors
Quaternion represented by 4 numbers– Versor
• Direction – parallel to axis of rotation• Rotation angle• Norm – function of rotation angle
– Tensor• Magnitude
![Page 32: Medical Image Registration Kumar Rajamani. Registration Spatial transform that maps points from one image to corresponding points in another image](https://reader035.vdocuments.us/reader035/viewer/2022062409/56649c7b5503460f9492f551/html5/thumbnails/32.jpg)
Rigid Transform in 3D
Use quaternions instead of phasors P’=V*(P-C)+C P’=V*P+T, T=C-V*C P=point, V=Versor, T=Translation, C=fixed
point Transform represented by 6 parameters
– Three numbers representing versor– Three components of fixed coordinate system
![Page 33: Medical Image Registration Kumar Rajamani. Registration Spatial transform that maps points from one image to corresponding points in another image](https://reader035.vdocuments.us/reader035/viewer/2022062409/56649c7b5503460f9492f551/html5/thumbnails/33.jpg)
Image Interpolators
2 functions– Compute interpolated intensity at
requested position– Detect whether or not requested position
lies within moving-image domain
![Page 34: Medical Image Registration Kumar Rajamani. Registration Spatial transform that maps points from one image to corresponding points in another image](https://reader035.vdocuments.us/reader035/viewer/2022062409/56649c7b5503460f9492f551/html5/thumbnails/34.jpg)
Nearest Neighbor
Uses intensity of nearest grid position Computationally cheap Doesn’t require floating point
calculations
![Page 35: Medical Image Registration Kumar Rajamani. Registration Spatial transform that maps points from one image to corresponding points in another image](https://reader035.vdocuments.us/reader035/viewer/2022062409/56649c7b5503460f9492f551/html5/thumbnails/35.jpg)
Linear Interpolation
Computed as the weighted sum of 2n-1 neighbors
n=dimensionality of image Weighting is based on distance
between requested position and neighbors
![Page 36: Medical Image Registration Kumar Rajamani. Registration Spatial transform that maps points from one image to corresponding points in another image](https://reader035.vdocuments.us/reader035/viewer/2022062409/56649c7b5503460f9492f551/html5/thumbnails/36.jpg)
B-spline Interpolation
Intensity calculated by multiplying B-spline coefficients with shifted B-spline kernels
Higher spline orders require more pixels to computer interpolated value
Third-order B-spline kernels typically used because good tradeoff between smoothness and computational burden
![Page 37: Medical Image Registration Kumar Rajamani. Registration Spatial transform that maps points from one image to corresponding points in another image](https://reader035.vdocuments.us/reader035/viewer/2022062409/56649c7b5503460f9492f551/html5/thumbnails/37.jpg)
Metrics
Scalar function of the set of transform parameters for a given fixed image, moving image, and transformation type
Typically samples points within fixed image to compute the measure
![Page 38: Medical Image Registration Kumar Rajamani. Registration Spatial transform that maps points from one image to corresponding points in another image](https://reader035.vdocuments.us/reader035/viewer/2022062409/56649c7b5503460f9492f551/html5/thumbnails/38.jpg)
Mean Squares
Mean squared difference over all the pixels in an image
Intensities are interpolated for the moving image
For gradient-based optimization, derivative of metric is also required
2))],(()([1
),,|( pxMxFN
MFpS i
N
i i TT
![Page 39: Medical Image Registration Kumar Rajamani. Registration Spatial transform that maps points from one image to corresponding points in another image](https://reader035.vdocuments.us/reader035/viewer/2022062409/56649c7b5503460f9492f551/html5/thumbnails/39.jpg)
Mean Squares
Optimal value of zero Interpolator will affect computation time
and smoothness of metric plot Assumes intensity representing the
same homologous point is in both images
Images must be from same modality
![Page 40: Medical Image Registration Kumar Rajamani. Registration Spatial transform that maps points from one image to corresponding points in another image](https://reader035.vdocuments.us/reader035/viewer/2022062409/56649c7b5503460f9492f551/html5/thumbnails/40.jpg)
Mean Squares
Smoothness affected by interpolator
![Page 41: Medical Image Registration Kumar Rajamani. Registration Spatial transform that maps points from one image to corresponding points in another image](https://reader035.vdocuments.us/reader035/viewer/2022062409/56649c7b5503460f9492f551/html5/thumbnails/41.jpg)
Normalized Correlation
Computes pixel-wise cross-correlation between the intensity of the two images, normalized by the square root of the autocorrelation of each image
For two identical images, metric =1 Misalignment, metric <1
![Page 42: Medical Image Registration Kumar Rajamani. Registration Spatial transform that maps points from one image to corresponding points in another image](https://reader035.vdocuments.us/reader035/viewer/2022062409/56649c7b5503460f9492f551/html5/thumbnails/42.jpg)
Normalized Correlation
-1 added for minimum-seeking optimizers
N
i
N
iii
N
iii
xTMxF
xTMxFMFpS
))(()(
))(()(1),,|(
22
T
![Page 43: Medical Image Registration Kumar Rajamani. Registration Spatial transform that maps points from one image to corresponding points in another image](https://reader035.vdocuments.us/reader035/viewer/2022062409/56649c7b5503460f9492f551/html5/thumbnails/43.jpg)
Normalized Correlation
![Page 44: Medical Image Registration Kumar Rajamani. Registration Spatial transform that maps points from one image to corresponding points in another image](https://reader035.vdocuments.us/reader035/viewer/2022062409/56649c7b5503460f9492f551/html5/thumbnails/44.jpg)
Difference Density
Each pixel’s contribution is calculated using bell-shaped function
f(d) has a maximum of 1 at d=0 and minimum of zero at d=+/-infinity
d is difference in intensity b/w F and M
2)/(1
1)(
ddf
![Page 45: Medical Image Registration Kumar Rajamani. Registration Spatial transform that maps points from one image to corresponding points in another image](https://reader035.vdocuments.us/reader035/viewer/2022062409/56649c7b5503460f9492f551/html5/thumbnails/45.jpg)
Multi-modal Volume Registration by Maximization of Mutual Information
Wells W, Viola P, Atsumi H, Nakajima S, Kikinis R
![Page 46: Medical Image Registration Kumar Rajamani. Registration Spatial transform that maps points from one image to corresponding points in another image](https://reader035.vdocuments.us/reader035/viewer/2022062409/56649c7b5503460f9492f551/html5/thumbnails/46.jpg)
Registering Images from Same Modality Typical measure of error is sum of
squared differences between voxels values
This measure is directly proportional to the likelihood that the images are correctly registered
Same measure is NOT effective for images of different modalities
![Page 47: Medical Image Registration Kumar Rajamani. Registration Spatial transform that maps points from one image to corresponding points in another image](https://reader035.vdocuments.us/reader035/viewer/2022062409/56649c7b5503460f9492f551/html5/thumbnails/47.jpg)
Figure 8.9 from the ITK Software Guide v 2.4, by Luis Ibáñez, et al.62
MI Inputs
T1 MRI Proton density MRI
![Page 48: Medical Image Registration Kumar Rajamani. Registration Spatial transform that maps points from one image to corresponding points in another image](https://reader035.vdocuments.us/reader035/viewer/2022062409/56649c7b5503460f9492f551/html5/thumbnails/48.jpg)
![Page 49: Medical Image Registration Kumar Rajamani. Registration Spatial transform that maps points from one image to corresponding points in another image](https://reader035.vdocuments.us/reader035/viewer/2022062409/56649c7b5503460f9492f551/html5/thumbnails/49.jpg)
![Page 50: Medical Image Registration Kumar Rajamani. Registration Spatial transform that maps points from one image to corresponding points in another image](https://reader035.vdocuments.us/reader035/viewer/2022062409/56649c7b5503460f9492f551/html5/thumbnails/50.jpg)
Relationship Between Images of Different Modalities Example: We should be able to construct a
function F() that predicts CT voxel value from corresponding MRI value
Registration could be evaluated by computing F(MR) and comparing it to the CT image– Via sum of squared differences (or correlation)
In practice, this is a difficult and under-determined problem
![Page 51: Medical Image Registration Kumar Rajamani. Registration Spatial transform that maps points from one image to corresponding points in another image](https://reader035.vdocuments.us/reader035/viewer/2022062409/56649c7b5503460f9492f551/html5/thumbnails/51.jpg)
Mutual Information
Theory: Since MR and CT both describe the underlying anatomy, there will be mutual information between the two images
Find the best registration by maximizing the information that one image provides about the other
Requires no a priori model of the relationship Assumes that max. info. is provided when the
images are correctly registered
![Page 52: Medical Image Registration Kumar Rajamani. Registration Spatial transform that maps points from one image to corresponding points in another image](https://reader035.vdocuments.us/reader035/viewer/2022062409/56649c7b5503460f9492f551/html5/thumbnails/52.jpg)
Notation
Reference (fixed) volume: u(x) Test (moving) volume: v(x) x: coordinates of the volume T: transformation from coordinate frame
of reference volume to test volume v(T(x)): test volume voxel associated
with reference volume voxel u(x)
![Page 53: Medical Image Registration Kumar Rajamani. Registration Spatial transform that maps points from one image to corresponding points in another image](https://reader035.vdocuments.us/reader035/viewer/2022062409/56649c7b5503460f9492f551/html5/thumbnails/53.jpg)
Mutual Information
Defined in terms of entropies
If there are any dependencies, H(A,B)<H(A)+H(B)
)BA,()B()A()BA,(
),(log),()BA,(
)(log)()B(
)(log)()A(
HHHI
dabapbapH
dbbpbpH
daapapH
ABAB
BB
AA
![Page 54: Medical Image Registration Kumar Rajamani. Registration Spatial transform that maps points from one image to corresponding points in another image](https://reader035.vdocuments.us/reader035/viewer/2022062409/56649c7b5503460f9492f551/html5/thumbnails/54.jpg)
Maximizing Mutual Information
h(v(T(x))) encourages transformations that project u into complex parts of v
Last term of MI eqn contributes when u and v are functionally related
Together, last two terms of MI eqn identify transforms that find complexity and explain it well
)))((),(()))((())(()))((),((
)))((),((maxargˆ
xTvxuhxTvhxuhxTvxuI
xTvxuITT
![Page 55: Medical Image Registration Kumar Rajamani. Registration Spatial transform that maps points from one image to corresponding points in another image](https://reader035.vdocuments.us/reader035/viewer/2022062409/56649c7b5503460f9492f551/html5/thumbnails/55.jpg)
Parzen Windowing
Used to estimate probability density P*(z)
Entropy estimated based on P*(z)
![Page 56: Medical Image Registration Kumar Rajamani. Registration Spatial transform that maps points from one image to corresponding points in another image](https://reader035.vdocuments.us/reader035/viewer/2022062409/56649c7b5503460f9492f551/html5/thumbnails/56.jpg)
Finding Maximum of I(T)
To find maximum of mutual information, calculate its derivative:
Derivative of reference volume is 0, b/c not a function of T
Entropies depend on covariance of Parzen window functions
)))((),((*)))(((*))((*)( xTvxuhdT
dxTvh
dT
dxuh
dT
dTI
dT
d
![Page 57: Medical Image Registration Kumar Rajamani. Registration Spatial transform that maps points from one image to corresponding points in another image](https://reader035.vdocuments.us/reader035/viewer/2022062409/56649c7b5503460f9492f551/html5/thumbnails/57.jpg)
Stochastic Maximization of Mutual Information
Similar to gradient descent Steps are taken that are proportional to dI/dT Repeat:
– A {sample of size NA drawn from x}
– B {sample of size NB drawn from x}
– T T+λ(dI/dT)
λ is the learning rate Repeated a fixed # of times, or until
convergence
![Page 58: Medical Image Registration Kumar Rajamani. Registration Spatial transform that maps points from one image to corresponding points in another image](https://reader035.vdocuments.us/reader035/viewer/2022062409/56649c7b5503460f9492f551/html5/thumbnails/58.jpg)
Stochastic Approximation
Uses noisy derivative estimates instead of the true derivative for optimizing a function
Authors have found that technique always converges to a transformation estimate that is close to locally optimal– NA=NB=50 has been successful
The noise introduced by the sampling can effectively penetrate small local minima
![Page 59: Medical Image Registration Kumar Rajamani. Registration Spatial transform that maps points from one image to corresponding points in another image](https://reader035.vdocuments.us/reader035/viewer/2022062409/56649c7b5503460f9492f551/html5/thumbnails/59.jpg)
MRI-CT Example
Coarse to fine registration Images were smoothed by convolving with
binomial kernel Rigid transform represented by displacement
vectors and quaternions Images were sampled and tri-linear
interpolation was used 5 levels of resolution
– 10000, 5000(*4) iterations
![Page 60: Medical Image Registration Kumar Rajamani. Registration Spatial transform that maps points from one image to corresponding points in another image](https://reader035.vdocuments.us/reader035/viewer/2022062409/56649c7b5503460f9492f551/html5/thumbnails/60.jpg)
Initial Condition of MR-CT Registration
![Page 61: Medical Image Registration Kumar Rajamani. Registration Spatial transform that maps points from one image to corresponding points in another image](https://reader035.vdocuments.us/reader035/viewer/2022062409/56649c7b5503460f9492f551/html5/thumbnails/61.jpg)
Final Configuration for MR-CT Registration
![Page 62: Medical Image Registration Kumar Rajamani. Registration Spatial transform that maps points from one image to corresponding points in another image](https://reader035.vdocuments.us/reader035/viewer/2022062409/56649c7b5503460f9492f551/html5/thumbnails/62.jpg)
Initial Condition of MR-PET Registration
![Page 63: Medical Image Registration Kumar Rajamani. Registration Spatial transform that maps points from one image to corresponding points in another image](https://reader035.vdocuments.us/reader035/viewer/2022062409/56649c7b5503460f9492f551/html5/thumbnails/63.jpg)
Final Configuration for MR-PET Registration
![Page 64: Medical Image Registration Kumar Rajamani. Registration Spatial transform that maps points from one image to corresponding points in another image](https://reader035.vdocuments.us/reader035/viewer/2022062409/56649c7b5503460f9492f551/html5/thumbnails/64.jpg)
Application Register 2 MRIs of brain (SPGR and
T2-weighted) to visualize anatomy and tumor– Create at 3-D model for surgical planning
and visualization
![Page 65: Medical Image Registration Kumar Rajamani. Registration Spatial transform that maps points from one image to corresponding points in another image](https://reader035.vdocuments.us/reader035/viewer/2022062409/56649c7b5503460f9492f551/html5/thumbnails/65.jpg)
3-D Model
Tumor(green), Vessels(red), Ventricles(blue), Edema (orange)
![Page 66: Medical Image Registration Kumar Rajamani. Registration Spatial transform that maps points from one image to corresponding points in another image](https://reader035.vdocuments.us/reader035/viewer/2022062409/56649c7b5503460f9492f551/html5/thumbnails/66.jpg)
Correlation Conventional correlation aligns two
signals by minimizing a summed quadratic difference between their intensities
If intensity of one signal is negated, then intensities no longer agree, and alignment by correlation will fail
Mutual information is not affected by negation of either signal
![Page 67: Medical Image Registration Kumar Rajamani. Registration Spatial transform that maps points from one image to corresponding points in another image](https://reader035.vdocuments.us/reader035/viewer/2022062409/56649c7b5503460f9492f551/html5/thumbnails/67.jpg)
Occlusion
Correlation is significantly affected by occlusion because intensity is substantially different
Occlusion will reduce mutual information at alignment– But “mutual information measure degrades
gracefully when subject to partially occluded imagery”
![Page 68: Medical Image Registration Kumar Rajamani. Registration Spatial transform that maps points from one image to corresponding points in another image](https://reader035.vdocuments.us/reader035/viewer/2022062409/56649c7b5503460f9492f551/html5/thumbnails/68.jpg)
Comparison to Other Methods
Many researchers use surface-based methods to register MRI and PET imagery– Need for reliable segmentation is a drawback
Others use joint entropy to characterize registration– “not robust”: difficulty describing partial overlap– Mutual Information is better because it has a larger
capture range• Additional influence from term that rewards for complexity
in portion of test volume into which reference volume is transformed
![Page 69: Medical Image Registration Kumar Rajamani. Registration Spatial transform that maps points from one image to corresponding points in another image](https://reader035.vdocuments.us/reader035/viewer/2022062409/56649c7b5503460f9492f551/html5/thumbnails/69.jpg)
Comparison to Other Methods
Woods et al. register MR and PET by minimizing range of PET values associated with a particular MR intensity value– Effective when test volume distribution is
Gaussian– Mutual Information can handle data that is multi-
modal– Woods’ measure is sensitive to noise and outliers
![Page 70: Medical Image Registration Kumar Rajamani. Registration Spatial transform that maps points from one image to corresponding points in another image](https://reader035.vdocuments.us/reader035/viewer/2022062409/56649c7b5503460f9492f551/html5/thumbnails/70.jpg)
Registration Tools/ Softwares
Elastix (ITK) ANTS (ITK) Plastimatch Slicer3D (ITK) MevisLab Fiji
![Page 71: Medical Image Registration Kumar Rajamani. Registration Spatial transform that maps points from one image to corresponding points in another image](https://reader035.vdocuments.us/reader035/viewer/2022062409/56649c7b5503460f9492f551/html5/thumbnails/71.jpg)
Conclusions
Intensity based techniques work directly with volumetric data (vs. segmentation)
Mutual information does not rely on assumptions about nature of imaging modalities
Have also used this technique to register 3D volumetric images to video images of patients