taking endoscopy to a higher dimension · endoscopic 3-d information are precondition to calculate...
TRANSCRIPT
![Page 1: Taking endoscopy to a higher dimension · Endoscopic 3-D information are precondition to calculate intra-operative orientation! registrating with pre-operative MR/CT volumes avoid](https://reader035.vdocuments.us/reader035/viewer/2022070107/6021a682ececf974c224c069/html5/thumbnails/1.jpg)
Taking endoscopy to a higher dimensionComputer Aided 3-D NOTES
July 8, 2008
Dipl.-Ing. Kurt Höller1,M. Petrunina2, Dipl. Med.-Inf. J. Penne1, Prof. Dr.-Ing. J. Hornegger1,Dipl.-Ing. A. Schneider3, Dr. med. D. Wilhelm3, Prof. Dr. med. H. Feußner3
1Chair of Pattern Recognition (LME),Friedrich-Alexander-University Erlangen-Nuremberg
2Department of Medicine 1 (MED1),Friedrich-Alexander-University Erlangen-Nuremberg
3Workgroup for Minimal Invasive Surgery (MITI),Klinikum r. d. Isar, Technical University Munich
![Page 2: Taking endoscopy to a higher dimension · Endoscopic 3-D information are precondition to calculate intra-operative orientation! registrating with pre-operative MR/CT volumes avoid](https://reader035.vdocuments.us/reader035/viewer/2022070107/6021a682ececf974c224c069/html5/thumbnails/2.jpg)
2 / 36
Content
1 Introduction/Motivation2 NOTES
Idea of NOTESChallenges with NOTES
3 3-D endoscopy contributionsOrientationAugmented RealityEnhanced field of viewCollision Prevention
4 Summarize5 Outlook
K. Höller
![Page 3: Taking endoscopy to a higher dimension · Endoscopic 3-D information are precondition to calculate intra-operative orientation! registrating with pre-operative MR/CT volumes avoid](https://reader035.vdocuments.us/reader035/viewer/2022070107/6021a682ececf974c224c069/html5/thumbnails/3.jpg)
3 / 36
Overview
1 Introduction/Motivation2 NOTES
Idea of NOTESChallenges with NOTES
3 3-D endoscopy contributionsOrientationAugmented RealityEnhanced field of viewCollision Prevention
4 Summarize5 Outlook
K. Höller
![Page 4: Taking endoscopy to a higher dimension · Endoscopic 3-D information are precondition to calculate intra-operative orientation! registrating with pre-operative MR/CT volumes avoid](https://reader035.vdocuments.us/reader035/viewer/2022070107/6021a682ececf974c224c069/html5/thumbnails/4.jpg)
4 / 36
Endoscopic 3-D approachesState of the Art
3-D information can be achieved with
endoscopic ultrasound (EUS)magnetically anchored instrumentspassive optical approaches
stereo visionstructure from motionshape from shading
active optical approachespattern projectiontime-of-flight hybrid system
inertial sensors for gravity related orientation
K. Höller
![Page 5: Taking endoscopy to a higher dimension · Endoscopic 3-D information are precondition to calculate intra-operative orientation! registrating with pre-operative MR/CT volumes avoid](https://reader035.vdocuments.us/reader035/viewer/2022070107/6021a682ececf974c224c069/html5/thumbnails/5.jpg)
5 / 36
First prototype of a 3-D endoscopeBased on time-of-flight technology
first presented at
K. Höller
![Page 6: Taking endoscopy to a higher dimension · Endoscopic 3-D information are precondition to calculate intra-operative orientation! registrating with pre-operative MR/CT volumes avoid](https://reader035.vdocuments.us/reader035/viewer/2022070107/6021a682ececf974c224c069/html5/thumbnails/6.jpg)
6 / 36
Preliminary resultsLiver phantom with gall bladder
⇒ More details in the talk of Jochen Penne et al.
K. Höller
![Page 7: Taking endoscopy to a higher dimension · Endoscopic 3-D information are precondition to calculate intra-operative orientation! registrating with pre-operative MR/CT volumes avoid](https://reader035.vdocuments.us/reader035/viewer/2022070107/6021a682ececf974c224c069/html5/thumbnails/7.jpg)
6 / 36
Preliminary resultsLiver phantom with gall bladder
⇒ More details in the talk of Jochen Penne et al.K. Höller
![Page 8: Taking endoscopy to a higher dimension · Endoscopic 3-D information are precondition to calculate intra-operative orientation! registrating with pre-operative MR/CT volumes avoid](https://reader035.vdocuments.us/reader035/viewer/2022070107/6021a682ececf974c224c069/html5/thumbnails/8.jpg)
7 / 36
’Towards NOTES3D’Joint funding application at Deutsche Forschungsgemeinschaft (DFG)
Participating institutes:LME, Erlangen (Prof. J. Hornegger)MITI group, Munich (Prof. H. Feussner)CAMP, Munich (Prof. N. Navab)LGDV, Erlangen (Prof. G. Greiner)MED1, Erlangen (Prof. E.G. Hahn)
Submitted during
K. Höller
![Page 9: Taking endoscopy to a higher dimension · Endoscopic 3-D information are precondition to calculate intra-operative orientation! registrating with pre-operative MR/CT volumes avoid](https://reader035.vdocuments.us/reader035/viewer/2022070107/6021a682ececf974c224c069/html5/thumbnails/9.jpg)
8 / 36
Overview
1 Introduction/Motivation2 NOTES
Idea of NOTESChallenges with NOTES
3 3-D endoscopy contributionsOrientationAugmented RealityEnhanced field of viewCollision Prevention
4 Summarize5 Outlook
K. Höller
![Page 10: Taking endoscopy to a higher dimension · Endoscopic 3-D information are precondition to calculate intra-operative orientation! registrating with pre-operative MR/CT volumes avoid](https://reader035.vdocuments.us/reader035/viewer/2022070107/6021a682ececf974c224c069/html5/thumbnails/10.jpg)
9 / 36
Overview
1 Introduction/Motivation2 NOTES
Idea of NOTESChallenges with NOTES
3 3-D endoscopy contributionsOrientationAugmented RealityEnhanced field of viewCollision Prevention
4 Summarize5 Outlook
K. Höller
![Page 11: Taking endoscopy to a higher dimension · Endoscopic 3-D information are precondition to calculate intra-operative orientation! registrating with pre-operative MR/CT volumes avoid](https://reader035.vdocuments.us/reader035/viewer/2022070107/6021a682ececf974c224c069/html5/thumbnails/11.jpg)
10 / 36
Time LineFrom open surgery to NOTES
Surgery can be done as:
open surgery→ for hundreds of years
minimally invasive / laparoscopicsurgery→ since the late 80s
and through natural orifices→ "no longer if but when" (W. O. Richards, D. W. Rattner 2005)
⇒ July 22/23, 2005 white paper and foundation of Consortiumfor Assessment and Research (NOSCAR) on NOTES:Natural Orifice Translumenal Endoscopic Surgery
K. Höller
![Page 12: Taking endoscopy to a higher dimension · Endoscopic 3-D information are precondition to calculate intra-operative orientation! registrating with pre-operative MR/CT volumes avoid](https://reader035.vdocuments.us/reader035/viewer/2022070107/6021a682ececf974c224c069/html5/thumbnails/12.jpg)
11 / 36
Participating groups with NOTESGreat chance for technical innovations
Figure: Interdisciplinarity of NOTES
K. Höller
![Page 13: Taking endoscopy to a higher dimension · Endoscopic 3-D information are precondition to calculate intra-operative orientation! registrating with pre-operative MR/CT volumes avoid](https://reader035.vdocuments.us/reader035/viewer/2022070107/6021a682ececf974c224c069/html5/thumbnails/13.jpg)
12 / 36
Peroral transgastric routeNatural Orifice Translumenal Endoscopic Surgery
K. Höller
![Page 14: Taking endoscopy to a higher dimension · Endoscopic 3-D information are precondition to calculate intra-operative orientation! registrating with pre-operative MR/CT volumes avoid](https://reader035.vdocuments.us/reader035/viewer/2022070107/6021a682ececf974c224c069/html5/thumbnails/14.jpg)
13 / 36
Peranal transcolonic routeNatural Orifice Translumenal Endoscopic Surgery
K. Höller
![Page 15: Taking endoscopy to a higher dimension · Endoscopic 3-D information are precondition to calculate intra-operative orientation! registrating with pre-operative MR/CT volumes avoid](https://reader035.vdocuments.us/reader035/viewer/2022070107/6021a682ececf974c224c069/html5/thumbnails/15.jpg)
14 / 36
Transvaginal routeNatural Orifice Translumenal Endoscopic Surgery
K. Höller
![Page 16: Taking endoscopy to a higher dimension · Endoscopic 3-D information are precondition to calculate intra-operative orientation! registrating with pre-operative MR/CT volumes avoid](https://reader035.vdocuments.us/reader035/viewer/2022070107/6021a682ececf974c224c069/html5/thumbnails/16.jpg)
15 / 36
Peroral transesophageal routeNatural Orifice Translumenal Endoscopic Surgery
K. Höller
![Page 17: Taking endoscopy to a higher dimension · Endoscopic 3-D information are precondition to calculate intra-operative orientation! registrating with pre-operative MR/CT volumes avoid](https://reader035.vdocuments.us/reader035/viewer/2022070107/6021a682ececf974c224c069/html5/thumbnails/17.jpg)
16 / 36
NOTES Publications 2004-2007Fast growing community
Figure: NOTES Publications in SE (SAGES), GIE (ASGE), Endoscopy (ESGE), DDW
⇒ In 2008 more than twice the number of publications of 2007K. Höller
![Page 18: Taking endoscopy to a higher dimension · Endoscopic 3-D information are precondition to calculate intra-operative orientation! registrating with pre-operative MR/CT volumes avoid](https://reader035.vdocuments.us/reader035/viewer/2022070107/6021a682ececf974c224c069/html5/thumbnails/18.jpg)
17 / 36
Overview
1 Introduction/Motivation2 NOTES
Idea of NOTESChallenges with NOTES
3 3-D endoscopy contributionsOrientationAugmented RealityEnhanced field of viewCollision Prevention
4 Summarize5 Outlook
K. Höller
![Page 19: Taking endoscopy to a higher dimension · Endoscopic 3-D information are precondition to calculate intra-operative orientation! registrating with pre-operative MR/CT volumes avoid](https://reader035.vdocuments.us/reader035/viewer/2022070107/6021a682ececf974c224c069/html5/thumbnails/19.jpg)
18 / 36
Potential barriers to clinical practiceAccording to the NOTES white paper, New York 2005
Fundamental challenges to the safe introduction of NOTES
Access to peritoneal cavity
⇒ item we can support
Gastric or intestinal closurePrevention of infectionDevelopment of suturing and anastomotic (nonsuturing) devicesMaintaining spatial orientation
⇒ item we can support
Development of a multitasking platform
⇒ item we can support
Management of intraperitoneal complications and hemorrhagePhysiologic untoward eventsTraining other providers
K. Höller
![Page 20: Taking endoscopy to a higher dimension · Endoscopic 3-D information are precondition to calculate intra-operative orientation! registrating with pre-operative MR/CT volumes avoid](https://reader035.vdocuments.us/reader035/viewer/2022070107/6021a682ececf974c224c069/html5/thumbnails/20.jpg)
18 / 36
Potential barriers to clinical practiceAccording to the NOTES white paper, New York 2005
Fundamental challenges to the safe introduction of NOTES
Access to peritoneal cavity⇒ item we can supportGastric or intestinal closurePrevention of infectionDevelopment of suturing and anastomotic (nonsuturing) devicesMaintaining spatial orientation⇒ item we can supportDevelopment of a multitasking platform⇒ item we can supportManagement of intraperitoneal complications and hemorrhagePhysiologic untoward eventsTraining other providers
K. Höller
![Page 21: Taking endoscopy to a higher dimension · Endoscopic 3-D information are precondition to calculate intra-operative orientation! registrating with pre-operative MR/CT volumes avoid](https://reader035.vdocuments.us/reader035/viewer/2022070107/6021a682ececf974c224c069/html5/thumbnails/21.jpg)
19 / 36
Overview
1 Introduction/Motivation2 NOTES
Idea of NOTESChallenges with NOTES
3 3-D endoscopy contributionsOrientationAugmented RealityEnhanced field of viewCollision Prevention
4 Summarize5 Outlook
K. Höller
![Page 22: Taking endoscopy to a higher dimension · Endoscopic 3-D information are precondition to calculate intra-operative orientation! registrating with pre-operative MR/CT volumes avoid](https://reader035.vdocuments.us/reader035/viewer/2022070107/6021a682ececf974c224c069/html5/thumbnails/22.jpg)
20 / 36
Overview
1 Introduction/Motivation2 NOTES
Idea of NOTESChallenges with NOTES
3 3-D endoscopy contributionsOrientationAugmented RealityEnhanced field of viewCollision Prevention
4 Summarize5 Outlook
K. Höller
![Page 23: Taking endoscopy to a higher dimension · Endoscopic 3-D information are precondition to calculate intra-operative orientation! registrating with pre-operative MR/CT volumes avoid](https://reader035.vdocuments.us/reader035/viewer/2022070107/6021a682ececf974c224c069/html5/thumbnails/23.jpg)
21 / 36
Navigation support - OrientationFinding the entry point to the peritonial cavity
Challenge:
More information on position and orientation of the robotic deviceor the endoscope
Solution:
Nonrigid registration of intraoperative 3-D data with preoperativeCT or MR data is possibleCalculated transformation parameters can be used to represent,correct und visualize actual position and orientation
K. Höller
![Page 24: Taking endoscopy to a higher dimension · Endoscopic 3-D information are precondition to calculate intra-operative orientation! registrating with pre-operative MR/CT volumes avoid](https://reader035.vdocuments.us/reader035/viewer/2022070107/6021a682ececf974c224c069/html5/thumbnails/24.jpg)
22 / 36
Navigation support - OrientationFinding the entry point to the peritonial cavity
K. Höller
![Page 25: Taking endoscopy to a higher dimension · Endoscopic 3-D information are precondition to calculate intra-operative orientation! registrating with pre-operative MR/CT volumes avoid](https://reader035.vdocuments.us/reader035/viewer/2022070107/6021a682ececf974c224c069/html5/thumbnails/25.jpg)
23 / 36
Overview
1 Introduction/Motivation2 NOTES
Idea of NOTESChallenges with NOTES
3 3-D endoscopy contributionsOrientationAugmented RealityEnhanced field of viewCollision Prevention
4 Summarize5 Outlook
K. Höller
![Page 26: Taking endoscopy to a higher dimension · Endoscopic 3-D information are precondition to calculate intra-operative orientation! registrating with pre-operative MR/CT volumes avoid](https://reader035.vdocuments.us/reader035/viewer/2022070107/6021a682ececf974c224c069/html5/thumbnails/26.jpg)
24 / 36
Navigation support - Augmented RealityFinding the entry point to the peritonial cavity
Challenge:
Avoid injuries of hidden organs and vessels, e.g. while finding theentry point to the peritoneal cavityKnowledge of structures behind the visible wall is needed for asafe incision
Solution:
Registration with preoperative volumesSegmentation of objects of interest in the preoperative volumesAdaption of those objects by iteratively computed transformationparametersVisualization of hidden organs or vessels in intraoperativeendoscopic images by augmented reality
K. Höller
![Page 27: Taking endoscopy to a higher dimension · Endoscopic 3-D information are precondition to calculate intra-operative orientation! registrating with pre-operative MR/CT volumes avoid](https://reader035.vdocuments.us/reader035/viewer/2022070107/6021a682ececf974c224c069/html5/thumbnails/27.jpg)
25 / 36
Navigation support - Augmented RealityFinding the entry point to the peritonial cavity
K. Höller
![Page 28: Taking endoscopy to a higher dimension · Endoscopic 3-D information are precondition to calculate intra-operative orientation! registrating with pre-operative MR/CT volumes avoid](https://reader035.vdocuments.us/reader035/viewer/2022070107/6021a682ececf974c224c069/html5/thumbnails/28.jpg)
25 / 36
Navigation support - Augmented RealityFinding the entry point to the peritonial cavity
K. Höller
![Page 29: Taking endoscopy to a higher dimension · Endoscopic 3-D information are precondition to calculate intra-operative orientation! registrating with pre-operative MR/CT volumes avoid](https://reader035.vdocuments.us/reader035/viewer/2022070107/6021a682ececf974c224c069/html5/thumbnails/29.jpg)
26 / 36
Overview
1 Introduction/Motivation2 NOTES
Idea of NOTESChallenges with NOTES
3 3-D endoscopy contributionsOrientationAugmented RealityEnhanced field of viewCollision Prevention
4 Summarize5 Outlook
K. Höller
![Page 30: Taking endoscopy to a higher dimension · Endoscopic 3-D information are precondition to calculate intra-operative orientation! registrating with pre-operative MR/CT volumes avoid](https://reader035.vdocuments.us/reader035/viewer/2022070107/6021a682ececf974c224c069/html5/thumbnails/30.jpg)
27 / 36
Navigation support - Off-axis viewFinding the entry point to the peritonial cavity
Challenge:
Overcome boundaries of limited field of view like axis in-line viewand loss of spatial orientation
Solution:
3-D surface knowledge can be used to extend and virtually rotatethe field of viewWith a 3-D mosaicking technique, the field of view can beextended by reconstruction of the operation area.
K. Höller
![Page 31: Taking endoscopy to a higher dimension · Endoscopic 3-D information are precondition to calculate intra-operative orientation! registrating with pre-operative MR/CT volumes avoid](https://reader035.vdocuments.us/reader035/viewer/2022070107/6021a682ececf974c224c069/html5/thumbnails/31.jpg)
28 / 36
Navigation support - Off-axis viewFinding the entry point to the peritonial cavity
K. Höller
![Page 32: Taking endoscopy to a higher dimension · Endoscopic 3-D information are precondition to calculate intra-operative orientation! registrating with pre-operative MR/CT volumes avoid](https://reader035.vdocuments.us/reader035/viewer/2022070107/6021a682ececf974c224c069/html5/thumbnails/32.jpg)
29 / 36
Overview
1 Introduction/Motivation2 NOTES
Idea of NOTESChallenges with NOTES
3 3-D endoscopy contributionsOrientationAugmented RealityEnhanced field of viewCollision Prevention
4 Summarize5 Outlook
K. Höller
![Page 33: Taking endoscopy to a higher dimension · Endoscopic 3-D information are precondition to calculate intra-operative orientation! registrating with pre-operative MR/CT volumes avoid](https://reader035.vdocuments.us/reader035/viewer/2022070107/6021a682ececf974c224c069/html5/thumbnails/33.jpg)
30 / 36
Navigation support - Collision preventionFinding the entry point to the peritonial cavity
Challenge:
Provide a higher grade of safety for automatic tools and roboticdevicesEspecially important with multiple instruments through only oneflexible endoscope
Solution:
With real-time distance information efficient collision preventionwith tissue or other instruments can be enabledAuto-positioning depending on respiration or other patientmovements will be very helpful.
K. Höller
![Page 34: Taking endoscopy to a higher dimension · Endoscopic 3-D information are precondition to calculate intra-operative orientation! registrating with pre-operative MR/CT volumes avoid](https://reader035.vdocuments.us/reader035/viewer/2022070107/6021a682ececf974c224c069/html5/thumbnails/34.jpg)
31 / 36
Navigation support - Collision preventionFinding the entry point to the peritonial cavity
K. Höller
![Page 35: Taking endoscopy to a higher dimension · Endoscopic 3-D information are precondition to calculate intra-operative orientation! registrating with pre-operative MR/CT volumes avoid](https://reader035.vdocuments.us/reader035/viewer/2022070107/6021a682ececf974c224c069/html5/thumbnails/35.jpg)
32 / 36
Overview
1 Introduction/Motivation2 NOTES
Idea of NOTESChallenges with NOTES
3 3-D endoscopy contributionsOrientationAugmented RealityEnhanced field of viewCollision Prevention
4 Summarize5 Outlook
K. Höller
![Page 36: Taking endoscopy to a higher dimension · Endoscopic 3-D information are precondition to calculate intra-operative orientation! registrating with pre-operative MR/CT volumes avoid](https://reader035.vdocuments.us/reader035/viewer/2022070107/6021a682ececf974c224c069/html5/thumbnails/36.jpg)
33 / 36
Conclusion
Endoscopic 3-D information are precondition to
calculate intra-operative orientation→ registrating with pre-operative MR/CT volumes
avoid injuries of hidden organs and vessels→ making them visible by augmented reality
provide an enhanced field of view→ computing off-axis view or reconstructed area by stitching
to enable collision prevention, motion compensation andautomatic positioning of surgery tools→ using a real-time distance measurement
K. Höller
![Page 37: Taking endoscopy to a higher dimension · Endoscopic 3-D information are precondition to calculate intra-operative orientation! registrating with pre-operative MR/CT volumes avoid](https://reader035.vdocuments.us/reader035/viewer/2022070107/6021a682ececf974c224c069/html5/thumbnails/37.jpg)
34 / 36
Overview
1 Introduction/Motivation2 NOTES
Idea of NOTESChallenges with NOTES
3 3-D endoscopy contributionsOrientationAugmented RealityEnhanced field of viewCollision Prevention
4 Summarize5 Outlook
K. Höller
![Page 38: Taking endoscopy to a higher dimension · Endoscopic 3-D information are precondition to calculate intra-operative orientation! registrating with pre-operative MR/CT volumes avoid](https://reader035.vdocuments.us/reader035/viewer/2022070107/6021a682ececf974c224c069/html5/thumbnails/38.jpg)
35 / 36
OutlookSolution for loss of spatial orientation
real-time information of spatialorientation by measuring gravity
using MEMS-based inertialdevices:3-D accelerometers
K. Höller
![Page 39: Taking endoscopy to a higher dimension · Endoscopic 3-D information are precondition to calculate intra-operative orientation! registrating with pre-operative MR/CT volumes avoid](https://reader035.vdocuments.us/reader035/viewer/2022070107/6021a682ececf974c224c069/html5/thumbnails/39.jpg)
36 / 36
The End
Thank you for your attention!
Any further questions?
K. Höller