ct-video registration accuracy for virtual guidance of bronchoscopy 1 penn state university,...
Post on 19-Dec-2015
215 views
TRANSCRIPT
![Page 1: CT-Video Registration Accuracy for Virtual Guidance of Bronchoscopy 1 Penn State University, University Park, PA 16802 2 Lockheed-Martin, King of Prussia,](https://reader036.vdocuments.us/reader036/viewer/2022062714/56649d2b5503460f94a003f6/html5/thumbnails/1.jpg)
CT-Video Registration Accuracy for Virtual Guidance of Bronchoscopy
1Penn State University, University Park, PA 168022Lockheed-Martin, King of Prussia, PA
3University of Iowa, Iowa City, IA 52242
SPIE Medical Imaging 2004, San Diego, CA, 14-19 February 2004
James P. Helferty,James P. Helferty,1,21,2 Eric A. Hoffman, Eric A. Hoffman,33 Geoffrey McLennan,Geoffrey McLennan,33 andand William E. HigginsWilliam E. Higgins1,31,3
![Page 2: CT-Video Registration Accuracy for Virtual Guidance of Bronchoscopy 1 Penn State University, University Park, PA 16802 2 Lockheed-Martin, King of Prussia,](https://reader036.vdocuments.us/reader036/viewer/2022062714/56649d2b5503460f94a003f6/html5/thumbnails/2.jpg)
Matching Video and CT Rendering
ROI seen in CT but not in video
CT Scan of Chest
CT-Guided Bronchoscopy for
Lung Cancer Staging• Bronchoscopic biopsy critical for
staging.
• Physicians make errors when maneuvering bronchoscope to a biopsy site.
• Lymph nodes are hidden from endoscopic video, but visible in 3D CT analysis exploit CT using image guidance
• CT-guidance of bronchoscopy reduce errors, improve
biopsy success rate
VideoendoscopyInside airways
![Page 3: CT-Video Registration Accuracy for Virtual Guidance of Bronchoscopy 1 Penn State University, University Park, PA 16802 2 Lockheed-Martin, King of Prussia,](https://reader036.vdocuments.us/reader036/viewer/2022062714/56649d2b5503460f94a003f6/html5/thumbnails/3.jpg)
Image-Guided Bronchoscopy Systems
• McAdams et al. (AJR 1998) and Hopper et al. (Radiology 2001)
• Virtual bronchoscopy for lymph-node biopsy, but no live guidance.
• Solomon et al. (Chest 2000) – E/M sensor attached to scope
•limited planning, many potential errors, limited guidance
• Bricault et al. (IEEE-TMI 1998) – no device needed
• Registered videobronchoscopy to CT, but no live guidance.
• Mori et al. (SPIE Med. Imaging 2001, 2002) – no device needed
• Registered videobronchoscopy to CT and tracked video.
•Efforts not interactive: >20 sec to process each video frame.
Show potential, but recently proposed systems have limitations:
No device
needed
![Page 4: CT-Video Registration Accuracy for Virtual Guidance of Bronchoscopy 1 Penn State University, University Park, PA 16802 2 Lockheed-Martin, King of Prussia,](https://reader036.vdocuments.us/reader036/viewer/2022062714/56649d2b5503460f94a003f6/html5/thumbnails/4.jpg)
VideoStream
AVI FileAVI File
Endoscope
Scope Monitor
Scope ProcessorScope Processor
Light SourceLight Source
RGB,Sync,Video
Matrox Cable
Matrox PCI card
Main ThreadVideo Tracking
OpenGL Rendering
Worker ThreadMutual Information
Dual CPU System
Main ThreadVideo Tracking
OpenGL Rendering
Worker ThreadMutual Information
Dual CPU System
Video AGP card
VideoCapture
RenderedImage
Polygons, Viewpoint
Image
PC Enclosure
Software written in Visual C++.
Our Group’sImage-Guided Bronchoscopy System
![Page 5: CT-Video Registration Accuracy for Virtual Guidance of Bronchoscopy 1 Penn State University, University Park, PA 16802 2 Lockheed-Martin, King of Prussia,](https://reader036.vdocuments.us/reader036/viewer/2022062714/56649d2b5503460f94a003f6/html5/thumbnails/5.jpg)
Bronchoscope3D CT ScanDataSources
ImageProcessing
HTML Multimedia Case Study
SiteList
Segmented Airway Tree
Centerline Paths
Screen Snapshots
Recorded Movies
Physician Notes
Stage 1: 3D CT Assessment and Planning
Segment 3D Airway Tree Calculate Centerline Paths Define Target ROI biopsy sites
Stage 1: 3D CT Assessment and Planning
Segment 3D Airway Tree Calculate Centerline Paths Define Target ROI biopsy sites
Stage 2: Live Bronchoscopy
Capture Endoscopic Video Correct Video’s Barrel
Distortion Track/Register Video and Virtual
CT Map Target ROIs on Video
Stage 2: Live Bronchoscopy
Capture Endoscopic Video Correct Video’s Barrel
Distortion Track/Register Video and Virtual
CT Map Target ROIs on Video
See: Helferty et al., SPIE Med. Imaging 2001; Swift et al., Comp. Med. Imag. Graph. 2002.
System Processing Flow
![Page 6: CT-Video Registration Accuracy for Virtual Guidance of Bronchoscopy 1 Penn State University, University Park, PA 16802 2 Lockheed-Martin, King of Prussia,](https://reader036.vdocuments.us/reader036/viewer/2022062714/56649d2b5503460f94a003f6/html5/thumbnails/6.jpg)
Case h005_512_85. Root site = (253,217,0), seger = (RegGrow, no filter), ROI #2 considered (Blue)
Display during Stage-2 Bronchoscopy
![Page 7: CT-Video Registration Accuracy for Virtual Guidance of Bronchoscopy 1 Penn State University, University Park, PA 16802 2 Lockheed-Martin, King of Prussia,](https://reader036.vdocuments.us/reader036/viewer/2022062714/56649d2b5503460f94a003f6/html5/thumbnails/7.jpg)
Stage 2: CT-Guided Bronchoscopy Protocol
Live video from bronchoscope
Endoluminal 3D
CT rendering
1. Provide Virtual-World CT rendering ICT
2. Move bronchoscope “close” to ICT target view IV
3. Register Virtual World to target view IV
4. Go to Step 1 unless biopsy site reached
Key Step:
CT-Video Registration
![Page 8: CT-Video Registration Accuracy for Virtual Guidance of Bronchoscopy 1 Penn State University, University Park, PA 16802 2 Lockheed-Martin, King of Prussia,](https://reader036.vdocuments.us/reader036/viewer/2022062714/56649d2b5503460f94a003f6/html5/thumbnails/8.jpg)
CT-Video Registration Problem: Viewpoints
Optimal CT rendering
ICTTarget video IV =tIV
6-parameter viewpoint
3D position
3-angle direction
Standard cameradirection matrix
![Page 9: CT-Video Registration Accuracy for Virtual Guidance of Bronchoscopy 1 Penn State University, University Park, PA 16802 2 Lockheed-Martin, King of Prussia,](https://reader036.vdocuments.us/reader036/viewer/2022062714/56649d2b5503460f94a003f6/html5/thumbnails/9.jpg)
CT-Video Registration Problem: Optimization Problem
Normalized Mutual Information (NMI):
NMI Optimization:
i – starting point for
ICT
h(V), h(CT) – entropies
based on image
histograms (PDFs)
Ref: Studolme et al., Pattern Recognition, 1/99.
![Page 10: CT-Video Registration Accuracy for Virtual Guidance of Bronchoscopy 1 Penn State University, University Park, PA 16802 2 Lockheed-Martin, King of Prussia,](https://reader036.vdocuments.us/reader036/viewer/2022062714/56649d2b5503460f94a003f6/html5/thumbnails/10.jpg)
1. Steepest Ascent
2. Nelder-Meade Simplex
3. Simulated Annealing
CT-Video Registration Problem: Optimization Algorithms Tested
![Page 11: CT-Video Registration Accuracy for Virtual Guidance of Bronchoscopy 1 Penn State University, University Park, PA 16802 2 Lockheed-Martin, King of Prussia,](https://reader036.vdocuments.us/reader036/viewer/2022062714/56649d2b5503460f94a003f6/html5/thumbnails/11.jpg)
CT-Video Registration Problem: Error Measures for Tests
Position error
Angle error
Needle error
where:
needle position for bronchoscope ( )IV
“needle” position for optimal CT view ( )
ICT
no
po
![Page 12: CT-Video Registration Accuracy for Virtual Guidance of Bronchoscopy 1 Penn State University, University Park, PA 16802 2 Lockheed-Martin, King of Prussia,](https://reader036.vdocuments.us/reader036/viewer/2022062714/56649d2b5503460f94a003f6/html5/thumbnails/12.jpg)
Registration Protocol for Tests
1. Target video frame: View to optimize:
2. Registration process:
a. Fix 5 parameters of ICT’s viewpoint to IV’s true viewpoint:
-10 mm < X, Y, Z < 10 mm -20o < , , < 20o
b. Initialize ICT’s remaining parameter away from true value
c. Run NMI optimization until convergence
d. Measure errors
IV
ICT
errors for acceptable
registrations
![Page 13: CT-Video Registration Accuracy for Virtual Guidance of Bronchoscopy 1 Penn State University, University Park, PA 16802 2 Lockheed-Martin, King of Prussia,](https://reader036.vdocuments.us/reader036/viewer/2022062714/56649d2b5503460f94a003f6/html5/thumbnails/13.jpg)
Test #1: Performance of Optimization Algorithms (a) Eliminate video and CT source differences
(b) Measure registration error precisely
1. Target video frame: -- known fixed virtual CT view
2. View to optimize: -- based on SAME 3D CT image as
3. Test each optimization algorithms: stepwise, simplex, annealing
IV
IV
ICT
![Page 14: CT-Video Registration Accuracy for Virtual Guidance of Bronchoscopy 1 Penn State University, University Park, PA 16802 2 Lockheed-Martin, King of Prussia,](https://reader036.vdocuments.us/reader036/viewer/2022062714/56649d2b5503460f94a003f6/html5/thumbnails/14.jpg)
Test #1 -- Performance of Optimization Algorithms Example Registrations
(a) Test “video” view IV
(b) “Good” simplex result (X=8mm)
(c) “Poor” annealing result
(Y=10mm)
(d) “Poor” annealing result
( yaw = 20o)
ICT
ICT
ICT
ICT
![Page 15: CT-Video Registration Accuracy for Virtual Guidance of Bronchoscopy 1 Penn State University, University Park, PA 16802 2 Lockheed-Martin, King of Prussia,](https://reader036.vdocuments.us/reader036/viewer/2022062714/56649d2b5503460f94a003f6/html5/thumbnails/15.jpg)
Threshold for acceptable
angle error na
na = 5o
Test #1 -- Performance of Optimization Algorithms Example Error Plot (na)
Initial (yaw) varied.
Other 5 parameters
of ICT ‘s viewpoint
start at “true” values
![Page 16: CT-Video Registration Accuracy for Virtual Guidance of Bronchoscopy 1 Penn State University, University Park, PA 16802 2 Lockheed-Martin, King of Prussia,](https://reader036.vdocuments.us/reader036/viewer/2022062714/56649d2b5503460f94a003f6/html5/thumbnails/16.jpg)
Test #2: Impact of Airway Morphology -- 6 Test ROIs (a), (b) proximal and distal trachea
(c), (d) proximal and distal right main bronchus
(e), (f) proximal and distal left main bronchus
Run Simplex
Algorithm
ROI 3
![Page 17: CT-Video Registration Accuracy for Virtual Guidance of Bronchoscopy 1 Penn State University, University Park, PA 16802 2 Lockheed-Martin, King of Prussia,](https://reader036.vdocuments.us/reader036/viewer/2022062714/56649d2b5503460f94a003f6/html5/thumbnails/17.jpg)
Roll Z
Test #2: Impact of Airway Morphology – ROI 3
![Page 18: CT-Video Registration Accuracy for Virtual Guidance of Bronchoscopy 1 Penn State University, University Park, PA 16802 2 Lockheed-Martin, King of Prussia,](https://reader036.vdocuments.us/reader036/viewer/2022062714/56649d2b5503460f94a003f6/html5/thumbnails/18.jpg)
Test #2: Impact of Airway Morphology
Ranges of Starting Points that result in acceptable registrations
![Page 19: CT-Video Registration Accuracy for Virtual Guidance of Bronchoscopy 1 Penn State University, University Park, PA 16802 2 Lockheed-Martin, King of Prussia,](https://reader036.vdocuments.us/reader036/viewer/2022062714/56649d2b5503460f94a003f6/html5/thumbnails/19.jpg)
Test #3: Registering CT to Real Video* 6 Matching Test Pairs
Compare final registered result to
ICT
CTIV
CTIv
IV ROI
Grp 3
![Page 20: CT-Video Registration Accuracy for Virtual Guidance of Bronchoscopy 1 Penn State University, University Park, PA 16802 2 Lockheed-Martin, King of Prussia,](https://reader036.vdocuments.us/reader036/viewer/2022062714/56649d2b5503460f94a003f6/html5/thumbnails/20.jpg)
Test #3: Registering CT to Real Video* ROI-3 Pair
Z Roll
![Page 21: CT-Video Registration Accuracy for Virtual Guidance of Bronchoscopy 1 Penn State University, University Park, PA 16802 2 Lockheed-Martin, King of Prussia,](https://reader036.vdocuments.us/reader036/viewer/2022062714/56649d2b5503460f94a003f6/html5/thumbnails/21.jpg)
Test #3: Registering CT to Real Video* Summary over 6 ROI Pairs
Ranges of Starting Points that result in acceptable registrations
![Page 22: CT-Video Registration Accuracy for Virtual Guidance of Bronchoscopy 1 Penn State University, University Park, PA 16802 2 Lockheed-Martin, King of Prussia,](https://reader036.vdocuments.us/reader036/viewer/2022062714/56649d2b5503460f94a003f6/html5/thumbnails/22.jpg)
Test #4: Sensitivity to Different Lung Capacities* CT scan – done at full inspiration (TLC)* Bronchoscopy – done with chest nearly deflated (FRC)
1. Target “video” frame: = -- known fixed CT view (from FRC CT volume)
2. View to optimize: -- CT view from TLC CT volume
3. Run Simplex optimization algorithm:
Compare final result to previously matched result
IV
ICT
FRCICT
ICT
TLCICT
![Page 23: CT-Video Registration Accuracy for Virtual Guidance of Bronchoscopy 1 Penn State University, University Park, PA 16802 2 Lockheed-Martin, King of Prussia,](https://reader036.vdocuments.us/reader036/viewer/2022062714/56649d2b5503460f94a003f6/html5/thumbnails/23.jpg)
Test #4: Sensitivity to Different Lung Capacities* 3 TLC/FRC Matching Pairs (Pig data; volume controlled)
TLC FRC TLC FRC
TLC FRC
ROI
Pair 2
FRCIV = ICT
TLCICT
![Page 24: CT-Video Registration Accuracy for Virtual Guidance of Bronchoscopy 1 Penn State University, University Park, PA 16802 2 Lockheed-Martin, King of Prussia,](https://reader036.vdocuments.us/reader036/viewer/2022062714/56649d2b5503460f94a003f6/html5/thumbnails/24.jpg)
Test #4: Sensitivity to Different Lung Capacities* ROI Pair #2 (Pig data; volume controlled)
Z
![Page 25: CT-Video Registration Accuracy for Virtual Guidance of Bronchoscopy 1 Penn State University, University Park, PA 16802 2 Lockheed-Martin, King of Prussia,](https://reader036.vdocuments.us/reader036/viewer/2022062714/56649d2b5503460f94a003f6/html5/thumbnails/25.jpg)
Test #4: Sensitivity to Different Lung Capacities* 3 TLC/FRC Matching Pairs (Pig data; volume controlled)
Ranges of Starting Points that result in acceptable registrations
![Page 26: CT-Video Registration Accuracy for Virtual Guidance of Bronchoscopy 1 Penn State University, University Park, PA 16802 2 Lockheed-Martin, King of Prussia,](https://reader036.vdocuments.us/reader036/viewer/2022062714/56649d2b5503460f94a003f6/html5/thumbnails/26.jpg)
1. Method successful and runs in near real-time (5 sec per registration).
2. Good airway segmentation and video/CT “camera” calibration important.
3. Registration successful:
a. over a wide range of anatomy
b. Independent of lung volume
c. +/- 8-10 mm position deviations, +/-15-20o direction deviation
4. Head toward continuous video tracking and CT-video registration.
Helferty et al. SPIE Med. Imaging 2003
AcknowledgementsThis work was partially supported by NIH grants #CA074325 and CA091534, and by the Olympus Corporation.
Discussion
![Page 27: CT-Video Registration Accuracy for Virtual Guidance of Bronchoscopy 1 Penn State University, University Park, PA 16802 2 Lockheed-Martin, King of Prussia,](https://reader036.vdocuments.us/reader036/viewer/2022062714/56649d2b5503460f94a003f6/html5/thumbnails/27.jpg)
![Page 28: CT-Video Registration Accuracy for Virtual Guidance of Bronchoscopy 1 Penn State University, University Park, PA 16802 2 Lockheed-Martin, King of Prussia,](https://reader036.vdocuments.us/reader036/viewer/2022062714/56649d2b5503460f94a003f6/html5/thumbnails/28.jpg)
Steepest Ascent Algorithm
Also tested Nelder-Meade Simplex and Simulated Annealing
![Page 29: CT-Video Registration Accuracy for Virtual Guidance of Bronchoscopy 1 Penn State University, University Park, PA 16802 2 Lockheed-Martin, King of Prussia,](https://reader036.vdocuments.us/reader036/viewer/2022062714/56649d2b5503460f94a003f6/html5/thumbnails/29.jpg)
Test #2: Impact of Airway MorphologyConsider 6 Varied Airway Locations (ROIs)
1. Target video frame: -- a known fixed virtual CT view
2. View to optimize: -- based on SAME 3D CT image as
3. Run Simplex optimization algorithm.
IV
IV
ICT
![Page 30: CT-Video Registration Accuracy for Virtual Guidance of Bronchoscopy 1 Penn State University, University Park, PA 16802 2 Lockheed-Martin, King of Prussia,](https://reader036.vdocuments.us/reader036/viewer/2022062714/56649d2b5503460f94a003f6/html5/thumbnails/30.jpg)
Test #3: Registering CT to Real Video
1. Target video frame: -- known fixed video frame; have matching
2. View to optimize: -- from corresponding CT image
3. Run Simplex optimization algorithm:
a. Fix 5 parameters of ICT’s viewpoint to IV’s true viewpoint
b. Run optimization
c. Compare final registered result to
4. Test on three target “video/CT” matching pairs
IV
ICT
VICT
ICT
VICT
![Page 31: CT-Video Registration Accuracy for Virtual Guidance of Bronchoscopy 1 Penn State University, University Park, PA 16802 2 Lockheed-Martin, King of Prussia,](https://reader036.vdocuments.us/reader036/viewer/2022062714/56649d2b5503460f94a003f6/html5/thumbnails/31.jpg)
Test #4: Sensitivity to Different Lung Capacities* CT scan – done at full inspiration (TLC)* Bronchoscopy – done with chest nearly deflated (FRC)
1. Target “video” frame: = -- known fixed CT view (from FRC CT volume)
2. View to optimize: -- CT view from TLC CT volume
3. Run Simplex optimization algorithm:
a. Fix 5 parameters of ICT’s viewpoint to IV’s true viewpoint
b. Run optimization
c. Compare final result to previously matched result
4. Test on three “FRC/TLC” matching pairs
IV
ICT
ICT
TLCICT
FRCICT