mr-guided mixed reality for breast conserving surgical...
TRANSCRIPT
![Page 1: MR-Guided Mixed Reality for Breast Conserving Surgical ...med.stanford.edu/.../seminar_pdfs/2017/SCIT_seminar... · •Early stage breast cancer treatment is removal of tumor - lumpectomy](https://reader034.vdocuments.us/reader034/viewer/2022050109/5f46ac3f92d8f0660674580c/html5/thumbnails/1.jpg)
MR-Guided Mixed Reality for Breast Conserving Surgical
PlanningSuba Srinivasan ([email protected])
March 30th 2017 Mentors: Prof. Brian A. Hargreaves, Prof. Bruce L. Daniel
M E D I C I N E
![Page 2: MR-Guided Mixed Reality for Breast Conserving Surgical ...med.stanford.edu/.../seminar_pdfs/2017/SCIT_seminar... · •Early stage breast cancer treatment is removal of tumor - lumpectomy](https://reader034.vdocuments.us/reader034/viewer/2022050109/5f46ac3f92d8f0660674580c/html5/thumbnails/2.jpg)
• Early stage breast cancer treatment is removal of tumor - lumpectomy
Post wire-localization
• close margins • mastectomy • 3mm Ductal Carcinoma in situ (DCIS) at biopsy site
24% Wilke et al., 2014
MRI Guided Mixed Reality for Surgical Planning
normal tissue
Negative margin
Positive margin
edgecancer
cells
http://www.clrcut.com/normal tissueedge cancer
cells
2
![Page 3: MR-Guided Mixed Reality for Breast Conserving Surgical ...med.stanford.edu/.../seminar_pdfs/2017/SCIT_seminar... · •Early stage breast cancer treatment is removal of tumor - lumpectomy](https://reader034.vdocuments.us/reader034/viewer/2022050109/5f46ac3f92d8f0660674580c/html5/thumbnails/3.jpg)
http://www.cancer.gov/
Mam
mog
raph
y
http://weill.cornell.edu/mri/MRI/Chest/breast_mass_mri.htm
MRI
Current Breast Imaging Techniques
3
Surg
ery
![Page 4: MR-Guided Mixed Reality for Breast Conserving Surgical ...med.stanford.edu/.../seminar_pdfs/2017/SCIT_seminar... · •Early stage breast cancer treatment is removal of tumor - lumpectomy](https://reader034.vdocuments.us/reader034/viewer/2022050109/5f46ac3f92d8f0660674580c/html5/thumbnails/4.jpg)
Prone vs. Supine Breast MRI
4
![Page 5: MR-Guided Mixed Reality for Breast Conserving Surgical ...med.stanford.edu/.../seminar_pdfs/2017/SCIT_seminar... · •Early stage breast cancer treatment is removal of tumor - lumpectomy](https://reader034.vdocuments.us/reader034/viewer/2022050109/5f46ac3f92d8f0660674580c/html5/thumbnails/5.jpg)
Objective
http://www.cancer.gov/Mam
mog
raph
y
http://weill.cornell.edu/mri/MRI/Chest/breast_mass_mri.htm
MRI
Surg
ery
To enable surgeons to do more definitive surgeries by
• acquiring MR images in close to surgical position - supine instead of prone breast MRI
• Projecting these 3D MR images on to the patient for surgical planning
5
![Page 6: MR-Guided Mixed Reality for Breast Conserving Surgical ...med.stanford.edu/.../seminar_pdfs/2017/SCIT_seminar... · •Early stage breast cancer treatment is removal of tumor - lumpectomy](https://reader034.vdocuments.us/reader034/viewer/2022050109/5f46ac3f92d8f0660674580c/html5/thumbnails/6.jpg)
HoloLens - for Surgical Planning
6
![Page 7: MR-Guided Mixed Reality for Breast Conserving Surgical ...med.stanford.edu/.../seminar_pdfs/2017/SCIT_seminar... · •Early stage breast cancer treatment is removal of tumor - lumpectomy](https://reader034.vdocuments.us/reader034/viewer/2022050109/5f46ac3f92d8f0660674580c/html5/thumbnails/7.jpg)
Phantom MRI Dataset
https://wiki.cancerimagingarchive.net/display/Public/QIN+Breast+DCE-MRI
Patie
nt M
RI
3D Surface Rendering 2D Slice
7
![Page 8: MR-Guided Mixed Reality for Breast Conserving Surgical ...med.stanford.edu/.../seminar_pdfs/2017/SCIT_seminar... · •Early stage breast cancer treatment is removal of tumor - lumpectomy](https://reader034.vdocuments.us/reader034/viewer/2022050109/5f46ac3f92d8f0660674580c/html5/thumbnails/8.jpg)
Deformable RegistrationAf
ter
MRI mask
Phantom-mask
Befo
re
https://www.slicer.org/ 8
![Page 9: MR-Guided Mixed Reality for Breast Conserving Surgical ...med.stanford.edu/.../seminar_pdfs/2017/SCIT_seminar... · •Early stage breast cancer treatment is removal of tumor - lumpectomy](https://reader034.vdocuments.us/reader034/viewer/2022050109/5f46ac3f92d8f0660674580c/html5/thumbnails/9.jpg)
SegmentationWarped dataset Tumor
SkinChest
https://www.itksnap.org/ 9
![Page 10: MR-Guided Mixed Reality for Breast Conserving Surgical ...med.stanford.edu/.../seminar_pdfs/2017/SCIT_seminar... · •Early stage breast cancer treatment is removal of tumor - lumpectomy](https://reader034.vdocuments.us/reader034/viewer/2022050109/5f46ac3f92d8f0660674580c/html5/thumbnails/10.jpg)
Standard Viewing Planes
Serra
Vie
w S
agitt
al
Serra
Vie
w C
oron
al
Serra
Vie
w T
rans
vers
e
10
![Page 11: MR-Guided Mixed Reality for Breast Conserving Surgical ...med.stanford.edu/.../seminar_pdfs/2017/SCIT_seminar... · •Early stage breast cancer treatment is removal of tumor - lumpectomy](https://reader034.vdocuments.us/reader034/viewer/2022050109/5f46ac3f92d8f0660674580c/html5/thumbnails/11.jpg)
Scrolling through slices
11
![Page 12: MR-Guided Mixed Reality for Breast Conserving Surgical ...med.stanford.edu/.../seminar_pdfs/2017/SCIT_seminar... · •Early stage breast cancer treatment is removal of tumor - lumpectomy](https://reader034.vdocuments.us/reader034/viewer/2022050109/5f46ac3f92d8f0660674580c/html5/thumbnails/12.jpg)
Registration of Holograms to Patient
12
1. Manual Registration
2. Automatic Registration
• Moving the head/camera position to align the holograms to the patient
• Selecting the markers and adjusting the rotation using gestures
• Integration of OpenCV for automatic recognition of the markers in patient
• Alignment of the MRI markers to the recognized optical markers
![Page 13: MR-Guided Mixed Reality for Breast Conserving Surgical ...med.stanford.edu/.../seminar_pdfs/2017/SCIT_seminar... · •Early stage breast cancer treatment is removal of tumor - lumpectomy](https://reader034.vdocuments.us/reader034/viewer/2022050109/5f46ac3f92d8f0660674580c/html5/thumbnails/13.jpg)
Automatic Registration - Tag RecognitionExample AR Tags
http://docs.opencv.org/3.1.0/d5/dae/tutorial_aruco_detection.html
Step 2:
1. Remove the perspectiveFor each 4 sided polygon
2. Divide the matrix based on the marker size (4 x 4) or (5 x 5)3. Read the bits and match it to the input dictionary (orientation)
Step 3:Given 2D image corners, 3D object corners Output position, rotation
1. Find the contours in the video frame 2. Approximate the contours by a polygon
1. Remove those with corners ≠ 4
Step 1:
13
![Page 14: MR-Guided Mixed Reality for Breast Conserving Surgical ...med.stanford.edu/.../seminar_pdfs/2017/SCIT_seminar... · •Early stage breast cancer treatment is removal of tumor - lumpectomy](https://reader034.vdocuments.us/reader034/viewer/2022050109/5f46ac3f92d8f0660674580c/html5/thumbnails/14.jpg)
Phantom-MRI Registration
Phantom MRI
Tag 1’
Tag 2
Tag 3
Tag 1
Tag 2’
Tag 3’center’center
(Tag i’ - center’) × (Tag i - center)T
Covariance Matrix
Singular Value Decomposition
R (rotation) = UVT
t (translation) = center - R center’
Iterative Closest Point Algorithm
14
![Page 15: MR-Guided Mixed Reality for Breast Conserving Surgical ...med.stanford.edu/.../seminar_pdfs/2017/SCIT_seminar... · •Early stage breast cancer treatment is removal of tumor - lumpectomy](https://reader034.vdocuments.us/reader034/viewer/2022050109/5f46ac3f92d8f0660674580c/html5/thumbnails/15.jpg)
How well do we perceive the holograms?
If the user is asked to draw the hologram that they are visualizing • is the shape preserved?
• dimension?
![Page 16: MR-Guided Mixed Reality for Breast Conserving Surgical ...med.stanford.edu/.../seminar_pdfs/2017/SCIT_seminar... · •Early stage breast cancer treatment is removal of tumor - lumpectomy](https://reader034.vdocuments.us/reader034/viewer/2022050109/5f46ac3f92d8f0660674580c/html5/thumbnails/16.jpg)
Perceptual Accuracy - Set upStep 1: inter-pupillary distance measurement/ adjustment
rd2
rd1
![Page 17: MR-Guided Mixed Reality for Breast Conserving Surgical ...med.stanford.edu/.../seminar_pdfs/2017/SCIT_seminar... · •Early stage breast cancer treatment is removal of tumor - lumpectomy](https://reader034.vdocuments.us/reader034/viewer/2022050109/5f46ac3f92d8f0660674580c/html5/thumbnails/17.jpg)
Perceptual Accuracy - Set upStep 2: Tag tracking and adjustment of the tag position
![Page 18: MR-Guided Mixed Reality for Breast Conserving Surgical ...med.stanford.edu/.../seminar_pdfs/2017/SCIT_seminar... · •Early stage breast cancer treatment is removal of tumor - lumpectomy](https://reader034.vdocuments.us/reader034/viewer/2022050109/5f46ac3f92d8f0660674580c/html5/thumbnails/18.jpg)
Perceptual Accuracy - Setup
18
![Page 19: MR-Guided Mixed Reality for Breast Conserving Surgical ...med.stanford.edu/.../seminar_pdfs/2017/SCIT_seminar... · •Early stage breast cancer treatment is removal of tumor - lumpectomy](https://reader034.vdocuments.us/reader034/viewer/2022050109/5f46ac3f92d8f0660674580c/html5/thumbnails/19.jpg)
Perceptual Accuracy
a. Set-up b. Example overlay Image overlay of drawn shapes (green+white) and ground truth
(magenta+white) 19
![Page 20: MR-Guided Mixed Reality for Breast Conserving Surgical ...med.stanford.edu/.../seminar_pdfs/2017/SCIT_seminar... · •Early stage breast cancer treatment is removal of tumor - lumpectomy](https://reader034.vdocuments.us/reader034/viewer/2022050109/5f46ac3f92d8f0660674580c/html5/thumbnails/20.jpg)
Perceptual Accuracy - Results
DotsError in depth dimension (mean ± std. deviation)
[range]-1.0 ± 3.5 [-6.1 7.1] mm
Error in right-left direction (mean ± std. deviation) [range]
-0.2 ± 1.3 [-3.3 2.2] mmShapes
Error in depth dimension (mean ± std. deviation) [range]
-1.1±2.0 [-5.9 2.3] mmError in right-left direction (mean ± std. deviation)
[range]0.1±1.2 [-2.2 3.0] mm
Margin Tolerance [0.68 5.74] mmDice coefficient [0.56 0.95]
N=6 subjects
20