automated medical image registration using global and conditional mutual information
DESCRIPTION
Automated Medical Image Registration using Global and Conditional Mutual Information. Dirk Loeckx Frederik Maes, Dirk Vandermeulen, Paul Suetens. Medical Imaging Research Center. Medical Image Computing. Group of Biomedical Sciences. Group of Science, Engineering and Technology. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Automated Medical Image Registration using Global and Conditional Mutual Information](https://reader034.vdocuments.us/reader034/viewer/2022052701/56813949550346895da0e671/html5/thumbnails/1.jpg)
Automated Medical Image Registration using Global and Conditional Mutual
Information
Dirk LoeckxFrederik Maes, Dirk Vandermeulen, Paul Suetens
![Page 2: Automated Medical Image Registration using Global and Conditional Mutual Information](https://reader034.vdocuments.us/reader034/viewer/2022052701/56813949550346895da0e671/html5/thumbnails/2.jpg)
Group of Science, Engineeringand Technology
Group of BiomedicalSciences
Radiology, Nuclear Medicine, Cardiology,
Radiotherapy
Processing of Speech & Images
Medical Image Computing
Medical ImagingResearch Center
![Page 3: Automated Medical Image Registration using Global and Conditional Mutual Information](https://reader034.vdocuments.us/reader034/viewer/2022052701/56813949550346895da0e671/html5/thumbnails/3.jpg)
Image RegistrationFind geometrical relationship between
images
![Page 4: Automated Medical Image Registration using Global and Conditional Mutual Information](https://reader034.vdocuments.us/reader034/viewer/2022052701/56813949550346895da0e671/html5/thumbnails/4.jpg)
Why?• Wealth of images• Registration
– Integrate information from different images– Different modality, time, patient, pose, contrast– Automatic ↔ implicit
• Clinical workflow– Fully digital
• Acquisition, transmission, storage, retrieval, reading– Processing time, visualisation– Validation
![Page 5: Automated Medical Image Registration using Global and Conditional Mutual Information](https://reader034.vdocuments.us/reader034/viewer/2022052701/56813949550346895da0e671/html5/thumbnails/5.jpg)
Building BlocksSimilarity Measure
Rr Ffs fprp
frpfrpE
μ
μμ
;
;,log;, xμxg RF ;
x
yji ,
xu
v
djdi ,
Transformation Penalties
xμxxμ dPwEV ;
SmoothnessVolumeRigidity
R
Input data
F
Deformed Image
F’
OptimizerMultiresolutionAnalytic DerivativesLBFGSB (quasi Newton)
Visualization
![Page 6: Automated Medical Image Registration using Global and Conditional Mutual Information](https://reader034.vdocuments.us/reader034/viewer/2022052701/56813949550346895da0e671/html5/thumbnails/6.jpg)
Similarity Measure
Cross Correlation(CC)
Linear Relation
Mutual Information(MI)
Statistical Relation
Sum of Squared Differences (SSD)
Constant Relation
Rx
xFxRn
SSD 21g
2
22
1
1
1
RxFR
FxFRxRn
CC g
frRxfr fprp
frpfrpMIxFfwxRrw
nfrp
,
,ln,,
1, g
![Page 7: Automated Medical Image Registration using Global and Conditional Mutual Information](https://reader034.vdocuments.us/reader034/viewer/2022052701/56813949550346895da0e671/html5/thumbnails/7.jpg)
Mutual Information• ‘Information’ R carries about F and vice
versa• Based on joint histogram
– Combine intensity R(x) with F(g(x;m)) in discrete bins
– If I know p(r), how good can I predict p(f)?– Joint and marginal probability p(r;m), p(f;m),
p(r,f;m)– F(g(x;m)) unknown (digital images, partial
volume effect)• No spatial information!
![Page 8: Automated Medical Image Registration using Global and Conditional Mutual Information](https://reader034.vdocuments.us/reader034/viewer/2022052701/56813949550346895da0e671/html5/thumbnails/8.jpg)
Mutual Information
RH FRH ,
FH
+ – =
H R H F
, ,I R F H R H F H R F
logx
H X p x p xF : Reference image (histogram) : Floating image (histogram) : Entropy
R
![Page 9: Automated Medical Image Registration using Global and Conditional Mutual Information](https://reader034.vdocuments.us/reader034/viewer/2022052701/56813949550346895da0e671/html5/thumbnails/9.jpg)
Mutual Information• Excellent results for rigid
registration–Only translation/rotation (6
parameters)–Parameters influence whole image
• Sometimes works for nonrigid registration–100’s to 1000’s of parameters–Local influence
![Page 10: Automated Medical Image Registration using Global and Conditional Mutual Information](https://reader034.vdocuments.us/reader034/viewer/2022052701/56813949550346895da0e671/html5/thumbnails/10.jpg)
Some Examples
![Page 11: Automated Medical Image Registration using Global and Conditional Mutual Information](https://reader034.vdocuments.us/reader034/viewer/2022052701/56813949550346895da0e671/html5/thumbnails/11.jpg)
Rigid CT/MR
F. Maes, A. Collignon, D. Vandermeulen, G. Marchal, and P. Suetens, “Multimodality image registration by maximization of mutual information,” IEEE Trans. Med. Imag., vol. 16, no. 2, pp. 187–198, Apr 1997.
![Page 12: Automated Medical Image Registration using Global and Conditional Mutual Information](https://reader034.vdocuments.us/reader034/viewer/2022052701/56813949550346895da0e671/html5/thumbnails/12.jpg)
Nonrigid Cardiac US• Left ventricle– Manual delineation– Triangulation
• Propagate delineation– Mutual information
image registration– Backward and
forward– Constant
connectivity• 4D path of each
vector• Visual validation:
ok
![Page 13: Automated Medical Image Registration using Global and Conditional Mutual Information](https://reader034.vdocuments.us/reader034/viewer/2022052701/56813949550346895da0e671/html5/thumbnails/13.jpg)
Subtraction CT angiography
• 3D acquisitions of region of interest– Without / with contrast
• Registration– Nonrigid mutual information– Compensate for movement artefacts
• Subtraction– Only contrast remains– Clear view of vasculature
![Page 14: Automated Medical Image Registration using Global and Conditional Mutual Information](https://reader034.vdocuments.us/reader034/viewer/2022052701/56813949550346895da0e671/html5/thumbnails/14.jpg)
sCTA: Examples
![Page 15: Automated Medical Image Registration using Global and Conditional Mutual Information](https://reader034.vdocuments.us/reader034/viewer/2022052701/56813949550346895da0e671/html5/thumbnails/15.jpg)
MI for nonrigid registration• Ongoing field of research• Global Histogram
– Multimodal registration• Minimise minor in favour of major peaks• Reduce smaller image details
– Bias field• Register bias fields, not image features (also
rigid)• Local Histogram (image subdivision)
– Limited ‘hits’: statistical power?– Solution: overlapping subregions
![Page 16: Automated Medical Image Registration using Global and Conditional Mutual Information](https://reader034.vdocuments.us/reader034/viewer/2022052701/56813949550346895da0e671/html5/thumbnails/16.jpg)
Transformation model• -
– Reference position– Floating position
• Tensor-product B-splines– Rectangular mesh– Displacement vectors– Weighted sum– B-splines
• Multiresolution, limited span, analytical derivative
zkR
yjR
xiR
ijkijkR kzkykx
zyx 222; μμrg
![Page 17: Automated Medical Image Registration using Global and Conditional Mutual Information](https://reader034.vdocuments.us/reader034/viewer/2022052701/56813949550346895da0e671/html5/thumbnails/17.jpg)
Local Mutual Information• Overlapping subregions
– – : Spatial label, spatial bins
• Multidimensional mutual information?– Total correlation, 3 channel regional MI
• ‘Amount of redundancy in a set of variables’•
– Conditional MI• ‘MI between R and F when X is known’•
– …
XFRHXHFHRHXFRC ,,)(,,
x,,, frpfrp
xp
XFRHXFHXRHXFRI ,,
![Page 18: Automated Medical Image Registration using Global and Conditional Mutual Information](https://reader034.vdocuments.us/reader034/viewer/2022052701/56813949550346895da0e671/html5/thumbnails/18.jpg)
Conditional MI (cMI) XRH XFH
XFRH ,
, ,I R F X H R X H F X H R F X
+ – =
Locally, if I know p(r), can I better predict p(f)?
H R H F
H X
,, , log
,
x r f
x
p r fI R F X p p r f
p r p f
p I R F
x x
xx x
x x
x
![Page 19: Automated Medical Image Registration using Global and Conditional Mutual Information](https://reader034.vdocuments.us/reader034/viewer/2022052701/56813949550346895da0e671/html5/thumbnails/19.jpg)
Validation
![Page 20: Automated Medical Image Registration using Global and Conditional Mutual Information](https://reader034.vdocuments.us/reader034/viewer/2022052701/56813949550346895da0e671/html5/thumbnails/20.jpg)
In theory
R F
Global MI:whole imagelocal optimum
31
2
0
00~
AA
AH
3
2
00
0~
0~
A
AH
Conditional MI:central regionglobal optimum
![Page 21: Automated Medical Image Registration using Global and Conditional Mutual Information](https://reader034.vdocuments.us/reader034/viewer/2022052701/56813949550346895da0e671/html5/thumbnails/21.jpg)
Multimodal registration• 200 2D image pairs
– ‘CT’, ‘MR’– 256x256 pixels– I = 0, 200, 400, noise s
=50– Mesh spacing 32 voxels– 32 bins, PW, PV
• Initial transformation– m uniform, < 30 pixels
• Validation– Intensity difference– Warping index– ROI: 10% outside polygon
CT (
ori
gin
al)
MR
(warp
ed)
PV
, co
ndit
ional
MI
PW
,conditio
nal
MI
PV
, glo
bal M
I
PW
, glo
bal M
I
![Page 22: Automated Medical Image Registration using Global and Conditional Mutual Information](https://reader034.vdocuments.us/reader034/viewer/2022052701/56813949550346895da0e671/html5/thumbnails/22.jpg)
Multimodal registration
CT (
ori
gin
al)
MR
(warp
ed)
PV
, co
ndit
ional
MI
PW
,conditio
nal
MI
PV
, glo
bal M
I
PW
, glo
bal M
I
![Page 23: Automated Medical Image Registration using Global and Conditional Mutual Information](https://reader034.vdocuments.us/reader034/viewer/2022052701/56813949550346895da0e671/html5/thumbnails/23.jpg)
Bias field registration
• 200 2D image pairs– Lena image– 8 bit, 256x256 pixels– Floating: Bias field
• 2nd degree multiplicative• •
• Initial transformation– m uniform, < 30 pixels
• Validation– Intensity difference– Warping index
edycxbxyax 22
75.0,25.0,25.0,75.0
CT (
ori
gin
al)
MR
(warp
ed)
PV
, co
ndit
ional
MI
PW
,conditio
nal
MI
PV
, glo
bal M
I
PW
, glo
bal M
I
![Page 24: Automated Medical Image Registration using Global and Conditional Mutual Information](https://reader034.vdocuments.us/reader034/viewer/2022052701/56813949550346895da0e671/html5/thumbnails/24.jpg)
Bias field registration
CT (
ori
gin
al)
MR
(warp
ed)
PV
, co
ndit
ional
MI
PW
,conditio
nal
MI
PV
, glo
bal M
I
PW
, glo
bal M
I
![Page 25: Automated Medical Image Registration using Global and Conditional Mutual Information](https://reader034.vdocuments.us/reader034/viewer/2022052701/56813949550346895da0e671/html5/thumbnails/25.jpg)
Conclusion• Small structures
– Global joint entropy has two local optima• Bias towards minimal floating entropy
– Local joint entropy has single optimum• Combination of joint and marginal entropy
• Bias fields– Global MI
• Combines contributions from all over the image• Bias field: widening of tissue histogram peaks
– Local MI• Local histograms only• Less widening
![Page 26: Automated Medical Image Registration using Global and Conditional Mutual Information](https://reader034.vdocuments.us/reader034/viewer/2022052701/56813949550346895da0e671/html5/thumbnails/26.jpg)
Conclusion
• Mesh size?– Smaller mesh = fewer voxels/bin = less statistical
power– Compensated by overlapping B-spline windows (?)– 2D: 9216 voxels/bin (~ 3D: 7x7x7 voxels)
• Calculation time – 10x for 2D (200s), 20x for 3D (days)
• Other conditional similarity measures– Conditional Cross Correlation
![Page 27: Automated Medical Image Registration using Global and Conditional Mutual Information](https://reader034.vdocuments.us/reader034/viewer/2022052701/56813949550346895da0e671/html5/thumbnails/27.jpg)
Questions?