Virtual reality simulation in Virtual reality simulation in SURGERYSURGERY

Dr sumer yadavDr sumer yadav

What is What is virtual realityvirtual reality ? ?

Virtual realityVirtual reality ( (VRVR) ) is a technology is a technology which allows a user which allows a user to interact with a to interact with a computer-simulated environment, environment, whether that whether that environment is a environment is a simulation of the simulation of the real world or an real world or an imaginary world. imaginary world.

simulation

network communication

full-body immersion telepresence

immersion

artificiality

interaction

Components of virtual reality

What is What is simulationsimulation ? ?

SimulationSimulation is the is the imitationimitation of some real of some real thing.thing.

UsesUses 1. natural systems or human systems in 1. natural systems or human systems in

order to gain insight into their functioning.order to gain insight into their functioning. 2. Testing2. Testing 3.Training3.Training 4. Education4. Education 5.Simulation of technology for 5.Simulation of technology for

performance optimizationperformance optimization 6. Video games6. Video games

What is What is Virtual surgeryVirtual surgery ? ?

virtual reality simulation of surgical virtual reality simulation of surgical procedures.procedures.

analog for the actual surgery where analog for the actual surgery where doctors can practice on a virtual doctors can practice on a virtual patient before performing the patient before performing the surgery. surgery.

Advantage to the Advantage to the surgeonssurgeons

1.Improved surgical planning1.Improved surgical planning

2. practice surgery process on 2. practice surgery process on 3D models 3D models

VIRTUAL SURGERY SIMULATION VIRTUAL SURGERY SIMULATION

2 STEPS

3D IMAGE SIMULATION TOUCH SIMULATION

3D IMAGE SIMULATION 3D IMAGE SIMULATION

generate a 3D generate a 3D model of the part model of the part of the body that of the body that undergoes surgery.undergoes surgery.

first construct a first construct a reference model of reference model of a normal person.a normal person.

anatomical images can anatomical images can be derived from a be derived from a series of patient’s series of patient’s Magnetic Resonance Magnetic Resonance Images (MRI), Images (MRI), Computed Computed Tomography (CT) or Tomography (CT) or video recording, which video recording, which are 2D images. These are 2D images. These images are segmented images are segmented using various using various segmentation methods segmentation methods

final model is obtained by final model is obtained by deforming the reference deforming the reference model according to the model according to the condition of organ that is condition of organ that is operated upon.operated upon.

The image is digitally The image is digitally mapped on to the mapped on to the polygonal meshpolygonal mesh representing whatever part representing whatever part of the body on organ is of the body on organ is being examined. Each being examined. Each vortex of the polygon is vortex of the polygon is assigned attributes like assigned attributes like colour and reflectivity from colour and reflectivity from the reference model. the reference model.

software algorithmssoftware algorithms that that can calculate the can calculate the whereabouts of the virtual whereabouts of the virtual instrument and determines instrument and determines whether it has collide with whether it has collide with a body part or anything a body part or anything else. else.

algorithms to solve algorithms to solve how it how it looks or behave when the looks or behave when the body part is cutbody part is cut. We need . We need models of how various models of how various tissues behave when cut, tissues behave when cut, prodded, punctured and so prodded, punctured and so on. on.

The parameters of this model can The parameters of this model can then be then be tweakedtweaked to match what a to match what a surgeon experiences during an surgeon experiences during an actual procedure. To create graphic actual procedure. To create graphic that move without flickering collision that move without flickering collision detection and tissue deformation detection and tissue deformation must be calculated at least 30 must be calculated at least 30 times/sec.times/sec.

whole-body voxel human whole-body voxel human modelsmodels

Whole-body Whole-body voxel human models depict human models depict the shape of the human the shape of the human body (tissues and body (tissues and organs) as an aggregate organs) as an aggregate of minute elements. of minute elements. Each minute block, or Each minute block, or voxel, is labeled with a voxel, is labeled with a tag representing the tag representing the name of a tissue or an name of a tissue or an organ such as muscle or organ such as muscle or fat, and assigned with fat, and assigned with the electrical, physical the electrical, physical properties and color of properties and color of the respective the respective tissue/organ.tissue/organ.

TOUCH SIMULATION TOUCH SIMULATION

hapticshaptics = the ability to simulate touch = the ability to simulate touch Physicians rely a great deal on their Physicians rely a great deal on their

sense of touch for everything from sense of touch for everything from routine diagnosis to complex, life routine diagnosis to complex, life saving surgical procedure. So haptics, saving surgical procedure. So haptics, or the ability to simulate touch, goes a or the ability to simulate touch, goes a long way to make virtual reality long way to make virtual reality simulators more life like. simulators more life like.

  While the brain can be tricked into While the brain can be tricked into seeing seamless motion by flipping seeing seamless motion by flipping through 30 or so images per second, through 30 or so images per second, touch signals need to be refreshed touch signals need to be refreshed up to once a millisecond up to once a millisecond

  The precise rate at which a The precise rate at which a computer must update a haptic computer must update a haptic interface varies depending on what interface varies depending on what type of virtual surface is encountered type of virtual surface is encountered

soft object require lower update rates soft object require lower update rates than harder objects than harder objects

Why??Why??

A low update rate may not prevent a A low update rate may not prevent a users surgical instrument from users surgical instrument from sinking sinking into theinto the virtual fleshvirtual flesh, but in soft tissues , but in soft tissues that sinking is what is expected. that sinking is what is expected.

Most difficult to simulate is two Most difficult to simulate is two floppy objects interacting with each floppy objects interacting with each other-such as colon and other-such as colon and sigmoidocope, the long bendable sigmoidocope, the long bendable probe used to view the colon-probe used to view the colon-because of multiple collision point. In because of multiple collision point. In addition, the mechanics of such addition, the mechanics of such interaction are complicated, because interaction are complicated, because each object may deform the other. each object may deform the other.

simulating touch sensation simulating touch sensation calculate the forces applied to cut, probed, calculate the forces applied to cut, probed,

puncture the various tissues. puncture the various tissues.

Also how they react or behave when cut, Also how they react or behave when cut, probed, punctured using surgical probed, punctured using surgical instruments. instruments.

First we have to make physical models of First we have to make physical models of various tissues. The major difficulty in various tissues. The major difficulty in modeling organs is the physical behavior as modeling organs is the physical behavior as they have all kinds of complexities-they are they have all kinds of complexities-they are anisotropic, non homogeneous and anisotropic, non homogeneous and nonlinear.nonlinear.

If the haptic device is to give a realistic If the haptic device is to give a realistic impression of say pressing the skin on a patient’s impression of say pressing the skin on a patient’s arm, the mechanical contributions of the skin, the arm, the mechanical contributions of the skin, the fatty tissue benefit, muscle and even bone must fatty tissue benefit, muscle and even bone must be summed up.be summed up.

The equations to solve such a complex problem The equations to solve such a complex problem are known, but so far the calculations cannot be are known, but so far the calculations cannot be made fast enough to update a display at 30Hz, let made fast enough to update a display at 30Hz, let alone update a haptic interface at 500-1000Hz.alone update a haptic interface at 500-1000Hz.

VIRTUAL SURGERY SIMULATOR

POWERFUL PC HAPTIC INTERFACER

Working principleWorking principle

force feedback force feedback loop. loop.

Advantage to the surgeonAdvantage to the surgeon Surgeons improve surgery plans and practice surgery process on Surgeons improve surgery plans and practice surgery process on

3D models. 3D models. Pre operative evaluation of results.Pre operative evaluation of results. Error correction - repeat the surgical procedure as many number Error correction - repeat the surgical procedure as many number

of times of times Better anatomical picture of procedure – wide angles availableBetter anatomical picture of procedure – wide angles available Familiarity with actual procedureFamiliarity with actual procedure

VIRTUAL REALITY APPLICATIONS IN SURGERY

TRAINING AND EDUCATION

SURGICAL PLANNING

IMAGE GUIDANCE

TELESURGERY

TRAINING AND EDUCATION TRAINING AND EDUCATION

Surgeon must be ready to manage potentially Surgeon must be ready to manage potentially life-threatening situations in dynamic, life-threatening situations in dynamic, unpredictable environments. unpredictable environments.

Traditional teaching

textbook images cadavers

DisadvantageDisadvantage

textbook images, limit textbook images, limit one’s perspective of one’s perspective of anatomical structures to anatomical structures to 2D plane 2D plane

cadavers; limited in supply cadavers; limited in supply and generally allowing and generally allowing

one-time use only.one-time use only.

Advantage of VR simulatorAdvantage of VR simulator

view the anatomy from a wide range of view the anatomy from a wide range of angles and “fly through” organs to angles and “fly through” organs to examine bodies from inside. The examine bodies from inside. The experience can be highly interactive experience can be highly interactive allowing students to strip away the various allowing students to strip away the various layers of tissues and muscles to examine layers of tissues and muscles to examine each organ separately. each organ separately.

Unlike cadavers, VR models enable the Unlike cadavers, VR models enable the user to perform a procedure countless user to perform a procedure countless times.times.

It is interactive. It is interactive. An instructor presence is not necessary An instructor presence is not necessary Changes can be made that demonstrate Changes can be made that demonstrate

variation in anatomy or disease state variation in anatomy or disease state Students could also try different Students could also try different

technique and loot at tissues from technique and loot at tissues from perspective that would be impossible perspective that would be impossible during real operations. during real operations.

SURGICAL PLANNING SURGICAL PLANNING

In traditional surgery planning, the surgeon In traditional surgery planning, the surgeon calculates various parameters and procedure for calculates various parameters and procedure for surgery from his earlier experience and surgery from his earlier experience and imagination imagination

The surgeon does not have an exact idea about The surgeon does not have an exact idea about the result of the surgery after it has been the result of the surgery after it has been performed. performed. So the result of the surgery depends So the result of the surgery depends mainly on human factorsmainly on human factors

This leads to lots of errors and even to the This leads to lots of errors and even to the risk of losing the life of the patients risk of losing the life of the patients

examplesexamples

minimally invasive neurosurgical minimally invasive neurosurgical procedures. procedures.

craniofacial surgery. craniofacial surgery. Liver resection on a 3D deformable Liver resection on a 3D deformable

liver model with aid of a virtual liver model with aid of a virtual laparoscopic tool laparoscopic tool

If we want something to come to an abrupt If we want something to come to an abrupt stop that is in the case of bone , etc it stop that is in the case of bone , etc it requires a higher update rates than requires a higher update rates than bumping into something a little squishy like bumping into something a little squishy like skin, liver etc. skin, liver etc.

But still, simulating squish is no easy task But still, simulating squish is no easy task either. The number of collision point either. The number of collision point between a virtual squishy object and a between a virtual squishy object and a virtual instrument is larger and more virtual instrument is larger and more variable than between a virtual rigid object variable than between a virtual rigid object and an instrument. and an instrument.

‘‘The virtual reality technology can The virtual reality technology can serve as serve as useful adjunctuseful adjunct to traditional to traditional surgical planning techniques. surgical planning techniques.

3D reconstruction of important 3D reconstruction of important anatomical structures anatomical structures

complex anatomical relationships in complex anatomical relationships in specific patientspecific patient prior to surgery and prior to surgery and also to examine and display the also to examine and display the microsurgical anatomy of various microsurgical anatomy of various internal operations. internal operations.

IMAGE GUIDANCE IMAGE GUIDANCE

computer-aided surgery. computer-aided surgery. integration of integration of

Advanced imaging

3D graphicsimage processing

application of computational application of computational algorithm and VR visualization algorithm and VR visualization

Diagnostic imaging

intraoperative surgical navigation

preoperative surgical planning

Navigation in surgery Navigation in surgery

stereotactic principles stereotactic principles ability to locate a given point using ability to locate a given point using

geometric reference geometric reference Used till now in neurosurgery and Used till now in neurosurgery and

Robotic Surgery Robotic Surgery

‘‘X-ray vision’ of surgeonX-ray vision’ of surgeon

intra operative mapping of 3D image intra operative mapping of 3D image overlays on live video overlays on live video

used in conjunction with an open MRI used in conjunction with an open MRI scan scan

updates of patient’s position and updates of patient’s position and intra operative shape changes in soft intra operative shape changes in soft tissues with in reasonable time tissues with in reasonable time frame.frame.

TELESURGERY TELESURGERY

allows surgeons to operate on people who allows surgeons to operate on people who are physically separated from themselves. are physically separated from themselves.

master-slave robotmaster-slave robot video cameras configured to provide a video cameras configured to provide a

stereoscopic view stereoscopic view surgeon relies on a 3D virtual representation surgeon relies on a 3D virtual representation

of the patient of the patient successful in humans undergoing endoscopic successful in humans undergoing endoscopic

Coronary Artery Bypass grafting Coronary Artery Bypass grafting basic vascular and urologic procedures in basic vascular and urologic procedures in

swinesswines

VIRTUAL REALITY APPLICATIONS IN SURGERY

TRAINING AND EDUCATION

SURGICAL PLANNING

IMAGE GUIDANCE

TELESURGERY

IMPORTANCE OF VIRTUAL IMPORTANCE OF VIRTUAL REALITY IN SURGICAL FIELD REALITY IN SURGICAL FIELD

Intelligent computer backup Intelligent computer backup minimizes the number of medical minimizes the number of medical ‘mistakes’ ‘mistakes’

More effective use of minimal-access More effective use of minimal-access surgical techniquesurgical technique

Better training in anatomy and Better training in anatomy and surgical skill, with reduced need for surgical skill, with reduced need for cadavers. cadavers.

current limitations of VR current limitations of VR implementation implementation

shortcomings in the shortcomings in the realism of the simulations. realism of the simulations.

cost and processing power cost and processing power of available hardware. of available hardware.

Need to improve human-Need to improve human-computer interfaces computer interfaces

problem of time delays in problem of time delays in the simulator’s response to the simulator’s response to the users movements the users movements

Simulator sicknessSimulator sickness eyestrain, nausea, loss of eyestrain, nausea, loss of balance and disorientation balance and disorientation

CONCLUSION CONCLUSION

A recent report released by Institute of Medicine in A recent report released by Institute of Medicine in Washington DC, estimates that Washington DC, estimates that medical errors may medical errors may cause 1,00,000 patient deaths each year in US alonecause 1,00,000 patient deaths each year in US alone

Proponent of virtual reality believes that Proponent of virtual reality believes that incorporation of this technology into medical training incorporation of this technology into medical training will bring this grim statistic downwill bring this grim statistic down. .

the benefits of VR systems in medicines have clearly the benefits of VR systems in medicines have clearly been established in several areas, including been established in several areas, including improved training, better access to services, and improved training, better access to services, and increase cost effectiveness and accuracy in increase cost effectiveness and accuracy in performing certain conventional surgical procedures. performing certain conventional surgical procedures.

Thank youThank you