3d single and multimodal medical image registration using
TRANSCRIPT
![Page 1: 3D single and multimodal medical image registration using](https://reader034.vdocuments.us/reader034/viewer/2022042311/625b9ad9f869bf2ce00145a7/html5/thumbnails/1.jpg)
3D single and multimodal medical imageregistration using robust voxel similarity measuresand statistically constrained deformable models
Christoforos NIKOU
Laboratoire des Sciences de l’Image de l’Informatiqueet de la Télédétection (LSIIT)
Institut de Physique Biologique (IPB) – Faculté de Médecine
Centre National de la Recherche Scientifique (CNRS)Université Louis Pasteur-Strasbourg I (ULP)
![Page 2: 3D single and multimodal medical image registration using](https://reader034.vdocuments.us/reader034/viewer/2022042311/625b9ad9f869bf2ce00145a7/html5/thumbnails/2.jpg)
Research framework
• General framework– brain imaging– intra-subject– rigid registration– images
• Single modal (MR/MR,SPECT/SPECT)
• multimodal(MRI/SPECT)
Introduction
MRI SPECT
![Page 3: 3D single and multimodal medical image registration using](https://reader034.vdocuments.us/reader034/viewer/2022042311/625b9ad9f869bf2ce00145a7/html5/thumbnails/3.jpg)
Registration shortcomings
• Robustness to:– non gaussian noise;– lesion evolution;– non overlaping informations or structures;– incomplete acquisitions.
Introduction
![Page 4: 3D single and multimodal medical image registration using](https://reader034.vdocuments.us/reader034/viewer/2022042311/625b9ad9f869bf2ce00145a7/html5/thumbnails/4.jpg)
Example 1 : lesion evolution
Reference image Image to beregistered
Introduction
![Page 5: 3D single and multimodal medical image registration using](https://reader034.vdocuments.us/reader034/viewer/2022042311/625b9ad9f869bf2ce00145a7/html5/thumbnails/5.jpg)
Example 2 : non overlaping structures
Introduction
![Page 6: 3D single and multimodal medical image registration using](https://reader034.vdocuments.us/reader034/viewer/2022042311/625b9ad9f869bf2ce00145a7/html5/thumbnails/6.jpg)
Example 3 : incomplete acquisition
Introduction
![Page 7: 3D single and multimodal medical image registration using](https://reader034.vdocuments.us/reader034/viewer/2022042311/625b9ad9f869bf2ce00145a7/html5/thumbnails/7.jpg)
Methods
• Robust similarity functions– reject outliers
• Deformable modeling– Statistical constraints through training
Introduction
![Page 8: 3D single and multimodal medical image registration using](https://reader034.vdocuments.us/reader034/viewer/2022042311/625b9ad9f869bf2ce00145a7/html5/thumbnails/8.jpg)
Presentation outline
• State of the art• Robust similarity metric-based registration• Statistically constrained deformable model-
based registration• Conclusion
![Page 9: 3D single and multimodal medical image registration using](https://reader034.vdocuments.us/reader034/viewer/2022042311/625b9ad9f869bf2ce00145a7/html5/thumbnails/9.jpg)
Medical image registration (1)
• Transformation– rigid– affine– projective– deformable
• Image primitives– non image-based methods
• stereostatic frame• markers
– image-based methods• deformable models• contours• surfaces• anatomical landmarks• voxels
State of the art
![Page 10: 3D single and multimodal medical image registration using](https://reader034.vdocuments.us/reader034/viewer/2022042311/625b9ad9f869bf2ce00145a7/html5/thumbnails/10.jpg)
Medical image registration (2)
• Similarity measure– standard distances;– principal axes;– correlation;– histogram
• variance;• entropy.
• Image modality– single modal;– multimodal;– modality to model
(atlas).
State of the art
![Page 11: 3D single and multimodal medical image registration using](https://reader034.vdocuments.us/reader034/viewer/2022042311/625b9ad9f869bf2ce00145a7/html5/thumbnails/11.jpg)
Medical image registration (3)
• Validation– precision
• blind evaluation• simulations
– robustness– CPU time– clinical routine application
State of the art
![Page 12: 3D single and multimodal medical image registration using](https://reader034.vdocuments.us/reader034/viewer/2022042311/625b9ad9f869bf2ce00145a7/html5/thumbnails/12.jpg)
Robustness to outliers
• Single modal images– Median least squares [Alexander-96]– Sign changes [Herbin-89]
• Multimodal images– Mutual information [Collignon-94, Wells-96]
State of the art
![Page 13: 3D single and multimodal medical image registration using](https://reader034.vdocuments.us/reader034/viewer/2022042311/625b9ad9f869bf2ce00145a7/html5/thumbnails/13.jpg)
Presentation outline
• State of the art• Robust similarity metric-based registration• Statistically constrained deformable model-
based registration• Conclusion
![Page 14: 3D single and multimodal medical image registration using](https://reader034.vdocuments.us/reader034/viewer/2022042311/625b9ad9f869bf2ce00145a7/html5/thumbnails/14.jpg)
Rigid registration
I x I T x
E F I x I T x
t t t x y z s s sx y z x y zT
1 2
1 2
( ) ( ( ))
( ) ( ( ), ( ( )))
[ , , , � , � , �, , , ]
⇔
=
=
Θ
ΘΘ
Θ
Robust voxel similarity measures
![Page 15: 3D single and multimodal medical image registration using](https://reader034.vdocuments.us/reader034/viewer/2022042311/625b9ad9f869bf2ce00145a7/html5/thumbnails/15.jpg)
Standard similarity measures (1)
• Quadratic error [Hajnal-95, Alpert-96]• Correlation [Van den Elsen-95]• Sign changes [Venot-84]
• Ratio uniformity [Woods-92]• Inter-image uniformity [Woods-93]• Mutual information [Wells-96]
Single modal images
Multiomodalimages
Robust voxel similarity measures
![Page 16: 3D single and multimodal medical image registration using](https://reader034.vdocuments.us/reader034/viewer/2022042311/625b9ad9f869bf2ce00145a7/html5/thumbnails/16.jpg)
Standard similarity measures(2)
• Single modal images (quadratic error)
E I x I T xx
( ) [ ( ) ( ( ))]Θ Θ= −� 1 22
Robust voxel similarity measures
![Page 17: 3D single and multimodal medical image registration using](https://reader034.vdocuments.us/reader034/viewer/2022042311/625b9ad9f869bf2ce00145a7/html5/thumbnails/17.jpg)
Inter-image uniformity (1)
• Fundamental hypothesis:– correspondance between
uniform regions
• Application:– partitioning of the reference
image to its grey levels– Projection of the
partitioning to the floatingimage
– variance minimization in theprojected regions
Robust voxel similarity measures
![Page 18: 3D single and multimodal medical image registration using](https://reader034.vdocuments.us/reader034/viewer/2022042311/625b9ad9f869bf2ce00145a7/html5/thumbnails/18.jpg)
Inter-image uniformity(2)
• Multimodal images (inter-image uniformity)
E N
I T x
NI T x
g gg
G
gx I x g
g
gg x I x g
( ) ( )
( ) ( ( )) ( )
( ) ( ( ))
( )
( )
Θ Θ
Θ Θ
Θ
Θ
Θ
= ×
= −
=
=
=
=
�
�
�
σ
σ µ
µ
1
2
2
1
1
1
Robust voxel similarity measures
![Page 19: 3D single and multimodal medical image registration using](https://reader034.vdocuments.us/reader034/viewer/2022042311/625b9ad9f869bf2ce00145a7/html5/thumbnails/19.jpg)
How realistic is the uniformity assumption?
Robust voxel similarity measures
![Page 20: 3D single and multimodal medical image registration using](https://reader034.vdocuments.us/reader034/viewer/2022042311/625b9ad9f869bf2ce00145a7/html5/thumbnails/20.jpg)
Uniformity validation?
Ideal histogram Real histogram
Robust voxel similarity measures
![Page 21: 3D single and multimodal medical image registration using](https://reader034.vdocuments.us/reader034/viewer/2022042311/625b9ad9f869bf2ce00145a7/html5/thumbnails/21.jpg)
Robust similarity measures (1)
• Single modal images (robust quadratic error)
� Θ−=Θx
CxTIxIE ))),(()(()( 21ρ
Robust voxel similarity measures
![Page 22: 3D single and multimodal medical image registration using](https://reader034.vdocuments.us/reader034/viewer/2022042311/625b9ad9f869bf2ce00145a7/html5/thumbnails/22.jpg)
Robust similarity measures(2)
• Multimodal images (robust inter-image uniformity)
E N
I T x C
NI T x C
g gg
G
g gx I x g
gg x I x g
gg
( ) ~ ( )
~ ( ) ( ( ( )) ~ ( ), )
~ ( ) argmin ( ( ( )) , )
( )
( )
Θ Θ
Θ Θ
Θ
Θ
Θ
= ×
= −
= −
=
=
=
�
�
�
σ
σ ρ µ
µ ρ µµ
1
2
2
1
1
1
Robust voxel similarity measures
![Page 23: 3D single and multimodal medical image registration using](https://reader034.vdocuments.us/reader034/viewer/2022042311/625b9ad9f869bf2ce00145a7/html5/thumbnails/23.jpg)
The Geman-McClure robust estimator
ρ function ψ function
Robust voxel similarity measures
![Page 24: 3D single and multimodal medical image registration using](https://reader034.vdocuments.us/reader034/viewer/2022042311/625b9ad9f869bf2ce00145a7/html5/thumbnails/24.jpg)
Registration algorithm
• Multiresolusion strategy• Stochastic optimization
– Fast simulated annealing• Deterministic optimisation
– Iterated Conditionnal Modes (ICM)
Robust voxel similarity measures
![Page 25: 3D single and multimodal medical image registration using](https://reader034.vdocuments.us/reader034/viewer/2022042311/625b9ad9f869bf2ce00145a7/html5/thumbnails/25.jpg)
Experimental results
• MRI phantom• simulation (noise)• blind evaluation (comparative protocol)• Clinical applications cliniques (epilepsy,
MS)
Robust voxel similarity measures
![Page 26: 3D single and multimodal medical image registration using](https://reader034.vdocuments.us/reader034/viewer/2022042311/625b9ad9f869bf2ce00145a7/html5/thumbnails/26.jpg)
Approach Translation (vox) Rotation (deg)
LS 2,30 ± 1,75 4,71 ± 2,88
RLS 0,03 ± 0,07 0,41 ± 0,21
MRI/MRI : simulation
LS : robust least squaresRLS : robust least squares
Robust voxel similarity measures
![Page 27: 3D single and multimodal medical image registration using](https://reader034.vdocuments.us/reader034/viewer/2022042311/625b9ad9f869bf2ce00145a7/html5/thumbnails/27.jpg)
MRI/SPECT : simulation
Approach Translation (vox) Rotation (deg)
IU 3,85 ± 5,59 8,33 ± 4,51
MI 1,41 ± 0,74 0,94 ± 1,58
RIU 0,82 ± 0,53 0,21 ± 0,48
IU : inter-image uniformityMI : mutual informationRIU : robust inter-image uniformity
Robust voxel similarity measures
![Page 28: 3D single and multimodal medical image registration using](https://reader034.vdocuments.us/reader034/viewer/2022042311/625b9ad9f869bf2ce00145a7/html5/thumbnails/28.jpg)
2D MRI/MRI (1)
Reference image Image to beregistered
Robust voxel similarity measures
![Page 29: 3D single and multimodal medical image registration using](https://reader034.vdocuments.us/reader034/viewer/2022042311/625b9ad9f869bf2ce00145a7/html5/thumbnails/29.jpg)
2D MRI/MRI(2)
LS RLS
Robust voxel similarity measures
![Page 30: 3D single and multimodal medical image registration using](https://reader034.vdocuments.us/reader034/viewer/2022042311/625b9ad9f869bf2ce00145a7/html5/thumbnails/30.jpg)
3D MRI/MRI (1)
Before registration
Robust voxel similarity measures
![Page 31: 3D single and multimodal medical image registration using](https://reader034.vdocuments.us/reader034/viewer/2022042311/625b9ad9f869bf2ce00145a7/html5/thumbnails/31.jpg)
3D MRI/MRI (2)
Least squares
Robust voxel similarity measures
![Page 32: 3D single and multimodal medical image registration using](https://reader034.vdocuments.us/reader034/viewer/2022042311/625b9ad9f869bf2ce00145a7/html5/thumbnails/32.jpg)
3D MRI/MRI(3)
Robust leastsquares
Robust voxel similarity measures
![Page 33: 3D single and multimodal medical image registration using](https://reader034.vdocuments.us/reader034/viewer/2022042311/625b9ad9f869bf2ce00145a7/html5/thumbnails/33.jpg)
3D MRI/SPECT (1)
Inter-image uniformity
Robust voxel similarity measures
![Page 34: 3D single and multimodal medical image registration using](https://reader034.vdocuments.us/reader034/viewer/2022042311/625b9ad9f869bf2ce00145a7/html5/thumbnails/34.jpg)
3D MRI/SPECT(2)
Mutual information
Robust voxel similarity measures
![Page 35: 3D single and multimodal medical image registration using](https://reader034.vdocuments.us/reader034/viewer/2022042311/625b9ad9f869bf2ce00145a7/html5/thumbnails/35.jpg)
3D MRI/SPECT(3)
Robust inter-imageuniformity
Robust voxel similarity measures
![Page 36: 3D single and multimodal medical image registration using](https://reader034.vdocuments.us/reader034/viewer/2022042311/625b9ad9f869bf2ce00145a7/html5/thumbnails/36.jpg)
Blind evaluation (1)
• Vanderbilt University (Nashville, TN, USA)image database
• CT/MRI et PET/MRI• Registration error computed on VOIs
Robust voxel similarity measures
![Page 37: 3D single and multimodal medical image registration using](https://reader034.vdocuments.us/reader034/viewer/2022042311/625b9ad9f869bf2ce00145a7/html5/thumbnails/37.jpg)
Blind evaluation (2)
• Several MRI modalities(T1, T2, PD …)
• MRI/PET registrationwithout removing nonbrain structures
Robust voxel similarity measures
![Page 38: 3D single and multimodal medical image registration using](https://reader034.vdocuments.us/reader034/viewer/2022042311/625b9ad9f869bf2ce00145a7/html5/thumbnails/38.jpg)
Blind evaluation (3)
Erreur médiane (mm) Erreur maximale (mm)
Type RIU Others RIU Others MRI/CT 1,6 - 2,9 3,3 - 4,0 4,3 - 6,2 10,7 - 12,2
MRI/PET 1,9 - 4,3 3,3 - 3,6 5,2 - 9,0 7,4 - 9,7
Robust voxel similarity measures
![Page 39: 3D single and multimodal medical image registration using](https://reader034.vdocuments.us/reader034/viewer/2022042311/625b9ad9f869bf2ce00145a7/html5/thumbnails/39.jpg)
Blind evaluation (4)
Rang de classement MRI/CT
(15 groups) MRI/PET
(13 groups)
MRI modality Méd Max Méd Max
T1 6 3 12 4 DP 7 4 5 1 T2 3 4 4 1
T1 rect. 7 5 13 11 DP rect. 7 5 1 1 T2 rect. 6 4 5 2
Robust voxel similarity measures
![Page 40: 3D single and multimodal medical image registration using](https://reader034.vdocuments.us/reader034/viewer/2022042311/625b9ad9f869bf2ce00145a7/html5/thumbnails/40.jpg)
CPU time (3D)CPU time (3D)
HP C360 (image 1283)
LS RLS IU MI RIU
5 mn 7 mn 10 mn 10 mn 20 mn
IU : inter-image uniformityMI : mutual informationRIU : robust inter-image uniformity
LS : least squaresRLS : robust least squares
Robust voxel similarity measures
![Page 41: 3D single and multimodal medical image registration using](https://reader034.vdocuments.us/reader034/viewer/2022042311/625b9ad9f869bf2ce00145a7/html5/thumbnails/41.jpg)
Clinical application
Jan 1998 – May 1999 : 108 cases treatedictal SPECT
Inter-ictal SPECT
superimpoposition
registration
registration
Robust voxel similarity measures
![Page 42: 3D single and multimodal medical image registration using](https://reader034.vdocuments.us/reader034/viewer/2022042311/625b9ad9f869bf2ce00145a7/html5/thumbnails/42.jpg)
Partial conclusion
• fully automatic method• redundant information is considered• local minima is not a problem• brain extraction is overcome• blind evaluation• clinical routine• other applications
Robust voxel similarity measures
![Page 43: 3D single and multimodal medical image registration using](https://reader034.vdocuments.us/reader034/viewer/2022042311/625b9ad9f869bf2ce00145a7/html5/thumbnails/43.jpg)
Visible / IR Registration
Robust voxel similarity measures
![Page 44: 3D single and multimodal medical image registration using](https://reader034.vdocuments.us/reader034/viewer/2022042311/625b9ad9f869bf2ce00145a7/html5/thumbnails/44.jpg)
Visible / IR Registration
Robust voxel similarity measures
Image uniformity Robust image uniformity
![Page 45: 3D single and multimodal medical image registration using](https://reader034.vdocuments.us/reader034/viewer/2022042311/625b9ad9f869bf2ce00145a7/html5/thumbnails/45.jpg)
Perspectives
• connected components approach inRIU
• robust estimator parametersadaptation to image modality
• robust mutual information
Perspectives