advanced interaction techniques for medical intervention and diagnosis
DESCRIPTION
Final thesis presentation at KU Leuven (Master of Science in Electrical Engineering)TRANSCRIPT
Advanced Interaction Techniques for Medical Intervention and Diagnosis
Chris DesomerLars Jacobs
Prof. Dr. Ir. P. Suetens Prof. Dr. Ir. T. Tuytelaars
Dr. Ir. S. De BuckDr. Ir. P. SlagmolenDr. Ir. J. GemmekeDr. Ir. M. Pedersoli
Supervisors:
Advisors:
Assessors:
07/04/2023 Slide 2 of 32
Outline• Context• Autostereoscopic visualisation• Gesture interface• Evaluation• Discussion & conclusion• Demo
Context
07/04/2023 Slide 4 of 32
Introduction (I)
07/04/2023 Slide 5 of 32
Introduction (II)
07/04/2023 Slide 6 of 32
Problem statement
07/04/2023 Slide 7 of 32
Purpose • Implementation of autostereoscopic visualisation
• Implementation of gesture interface
• Added value?
Autostereoscopic visualisation
07/04/2023 Slide 9 of 32
3D perception • Humans sees 3D if each eye gets another image
07/04/2023 Slide 10 of 32
Lenslet
07/04/2023 Slide 11 of 32
Multiview
07/04/2023 Slide 12 of 32
2D+Depth
07/04/2023 Slide 13 of 32
Real time aspects • Network optimisation
• Code optimisationo Block based instead of pixel basedo memcpy(), memset()
• Hardware optimisation
07/04/2023 Slide 14 of 32
4D visualisation • 3D + time
• Demo 4D
Gesture interface
07/04/2023 Slide 16 of 32
Hardware/framework gesture interface• Advanced interaction 3D hand/finger detection required• Kinect
o Far field detectiono Frameworks for 3D
hand/finger detection (e.g. Forth)
• Leap Motiono Near field detectiono Designed for 3D hand/finger
detection
07/04/2023 Slide 17 of 32
Comparison Kinect versus Leap MotionKinect Leap Motion
Price € 250 € 901
Position in the OR Outside sterile zone Inside sterile zone2
Interaction zone 57° horizontal Forth: 2m distance from sensor gives a
range of ≈ 2m
150° vertical 0,25 m distance from sensor gives a range of
≈ 2mResolution Kinect: 30 fps
Kinect + Forth: 20 fps 115 fps
Accuracy in ideal interaction zone 2cm 0,01mm
Conclusion: Leap Motion
1 Excl. price of sterilization 2 Sterilization can be done by a protective cover for ultrasound transducers
07/04/2023 Slide 18 of 32
Requirements Requirement Gesture Translation Closed hand movement
Scaling Pinch to zoom
Rotation Open hand rotation
Reset Swipe
On/off mechanism Virtual push button
Settings Static gesture
07/04/2023 Slide 19 of 32
Leap Motion
Gesture recognition
using heuristics
Action
Flow
Leap Motion
Gesture recognition using
correlation
Action
Data frame
Parameters
07/04/2023 20
Virtual push button
Evaluation
07/04/2023 Slide 22 of 32
Data models• Meshes
• Tracts
• Volume data o Filtering
07/04/2023 Slide 23 of 32
Data models• Meshes
• Tracts
• Volume data o Filtering
07/04/2023 Slide 24 of 32
Data models• Meshes
• Tracts
• Volume data o Filtering
Autostereoscopic visualisation
07/04/2023 Slide 26 of 32
Gesture interfaceGesture Intuitiveness DifficultyTranslation 3.4 3.0
Scaling 4.2 3.0
Rotation 3.5 2.9
Swipe 4.7 4.2
Airbutton 4.3 3.6
Static gestures 3.6 3.5
1: very unintuitive/very difficult5: very intuitive/very easy
07/04/2023 Slide 27 of 32
System Usability Scale (SUS)
50% gives at least a SUS score of 74 (≈70%)25% gives at least a SUS score of 79 (≈88%)
An average system has a SUS score of 68
Discussion & conclusion
07/04/2023 Slide 29 of 32
Discussion (I)• Autostereoscopic visualisation
+ Added value- Low resolution
• Gesture interface+ Dynamic 3D gesture set+ Intuitive- Training needed
07/04/2023 Slide 30 of 32
Discussion (II)• Forensic science
o Virtual crime sceneo No sterility
• Cardiovascular surgery o Combination = added valueo Catheter visualisation
07/04/2023 Slide 31 of 32
Conclusion• First general prototype
o Autostereoscopic visualisationo Gesture interface
• Nexto Application specific prototypeo Optimisations per medical domain
Demo
07/04/2023 Slide 34 of 32
References (I)• Article, R., & Ar, W. I. S. (2013). Augmented Reality in
Surgery, 139.• Dodgson, N. A. (2005). Autostereoscopic 3D displays.
Computer, 38(8), 31–36. doi:10.1109/MC.2005.252• Delude, C., Is Heart Surgery History?, 2007, (‘http://
protomag.com/assets/is-heart-surgery-history?page=4’)• Ector, B. J., Buck, S. De, Sc, M., Ph, D., Heidbuchel, H., &
Ph, D. (2008). Improved Efficiency in the EP Lab with syngo DynaCT Cardiac, (October), 26–31.
07/04/2023 Slide 35 of 32
References (II)• Oikonomidis, I., Kyriazis, N., & Argyros, A. (2011). Efficient
model-based 3D tracking of hand articulations using Kinect. Procedings of the British Machine Vision Conference 2011, 101.1–101.11. doi:10.5244/C.25.101
• Ruijters, D. (2008). Integrating autostereoscopic multi-view lenticular displays in minimally invasive angiography. Workshop proceedings, 87–94. Retrieved from https://lirias.kuleuven.be/handle/123456789/198103
• Sansoni, G., & Docchio, F. (2013). Biomedical 2D and 3D Imaging : State of Art and Future Perspectives, 3–19.
• Solutions, P. (n.d.). 3D Interface Specifications White Paper.
07/04/2023 Slide 36 of 32
2D+Depth
07/04/2023 Slide 37 of 32
Hardware optimisations Windows 8 laptop
Windows 7 desktop
iMac
CPU Intel Core i7 Q720 @ 1.60GHz
Intel Core i5 2500 @ 3.30GHz
Intel Core i5 4670 @ 3.40GHz
RAM memory 8Gb SO-DIMM DDR3 @ 533MHz
16Gb DDR3 @ 666MHz
32Gb DDR3 @ 1600MHz
GPU AMD Radeon HD6570M 1Gb RAM
AMD Radeon HD6450 1Gb RAM
NVIDIA GeForce GTX 780M 4GB RAM
Refresh rate ± 2.5 fps ± 9 fps ± 20 fps