phase recognition during surgical procedures using embedded and body-worn sensors
DESCRIPTION
Presentation of paper on the PerCom 2011 conference in Seattle.TRANSCRIPT
Jakob Bardram, Afsaneh Doryab, Rune M.Jensen, Poul M. Lange, Kristian L. G.Nielsen, and Soeren T. Petersen
•Motivation•Field studies•Sensor platform•The experiment•The results•Discussion and Conclusion
Outline
Motivation
• Existing work has mainly focused on daily routines of an individual person in home or outdoor
MOTIVATIONHospital work is collaborative which involves constant coordination and communication
MOTIVATIONAutomatic recognition of surgical procedure can be used for:
•Coordination, communication and planning
•Patient safety
•Context-aware information management and retrieval
Motivation
How can we automatically recognize the phases of a surgery using sensors?
Motivation
How can we automatically recognize the phases of a surgery using sensors?
What should be sensed?
Motivation
How can we automatically recognize the phases of a surgery using sensors?
What should be sensed?Which sensors provide more significant input ?
What clinicians do in a surgery?
How do they move around?
Field study
4 important activity zones
Who is involved in a surgery?
Field study
Which tools and instruments they use?Where they use them?
Field study
Field study
• 97% of tasks involved at least one physical instrument and 78% involved several instruments
• Direct relation between physical instruments and physical tasks
Field study
• 97% of tasks involved at least one physical instrument and 78% involved several instruments
• Direct relation between physical instruments and physical tasks
Temporal and sequential procedure
•Based on our detailed study, these parameters seemed important to track:
•Based on our detailed study, these parameters seemed important to track:
•The location of clinicians and the patient
•Based on our detailed study, these parameters seemed important to track:
•The location of clinicians and the patient
•The location of objects on different tables
•Based on our detailed study, these parameters seemed important to track:
•The location of clinicians and the patient
•The location of objects on different tables
•The use of objects and instruments by clinicians
Sensor platform - Hardware
Ubisense server
Platform server
Arduino hub
ArduinoArm-wrist object tracking
UbisensePerson-tracking
AlienShort-range- object tracking
TCP Serial
TCP
TCP
USB
XBee
IcodeShort-range- object tracking
•Synchronize input from different sensors•store raw observations (one instance per second)
Sensor platform- Software
Sensor platform – Feature vector
Sensor Platform – Feature vector
The Experiment
The Experiment
The Experiment
The Experiment
Analysis
Test the feasibility of the sensor platform
Analysis
Test the feasibility of the sensor platformHow accurate phase recognition could be using sensed
data
Analysis
Test the feasibility of the sensor platformHow accurate phase recognition could be using sensed
dataIdentify the impact of each sensor in achieving high
accuracy
4 simulated operations – (4 datasets)Leave-one-out cross validationUsing decision trees for classification
Analysis - Phase Recognition
Analysis - Decision tree
Analysis – Initial Results
Analysis – Feature Set
Laryngoscope in anesthesia table zone
Anesthesia nurse in anesthesia machine zone
Tube in anesthesia table zone
… time
0 1 0 … t1
0 1 1 … t2
0 1 1 … t3
. . . . .
. . . . .
1 1 0 … tn
Features
Feature Instances at time t1-tn
Results – Phase RecognitionLaryngoscope in anesthesia table zone
Anesthesia nurse in anesthesia machine zone
Tube in anesthesia table zone
… time
0 1 0 … t1
0 1 1 … t2
0 1 1 … t3
. . . . .
. . . . .
1 1 0 … tn
Features
Feature Instances at time t1-tn
Laryngoscope in anesthesia table zone
Laryngoscope in anesthesia table zone- acc
Anesthesia nurse in anesthesia machine zone
Anesthesia nurse in anesthesia machine zone- acc
Tube in anesthesia table zone
Tube in anesthesia table zone- acc
… time
0 0 1 1 0 0 … t1
0 0 1 2 1 1 … t2
0 0 1 3 1 2 … t3
. . . . . .
. . . . . .
1 10 1 13 0 2 … tn
Feature Processing
Laryngoscope in anesthesia table
zone
Anesthesia nurse in anesthesia machine zone
Tube in anesthesia table zone
… time
0 1 0 … t1
0 1 1 … t2
0 1 1 … t3
. . . . .
. . . . .
1 1 0 … tn
Feature Processing
Laryngoscope in anesthesia table zone
laryngoscope in anesthesia table zone- accumulated
time
0 0 t1
1 1 t2
1 2 t3
0 2 t4
0 2 t5
1 3 t6
Laryngoscope in anesthesia table
zone
Anesthesia nurse in anesthesia machine zone
Tube in anesthesia table zone
… time
0 1 0 … t1
0 1 1 … t2
0 1 1 … t3
. . . . .
. . . . .
1 1 0 … tn
Results – Phase Recognition
With historical features
Without historical features
Sensor Significance
Sensor Significance
Ubisense only
Ubisense only
Tables+
wristband
Sensor Significance
Sensor Significance
Ubisense only
Tables+
Wristband
Wristbandsonly
Location important or not?Using palm based sensors in the OR?Using sensors or images?Classical machine learning enough?
Discussion
Conclusion
• Possible to achieve relatively high classification accuracy in phase recognition during surgical procedures using machine learning techniques
Conclusion
• Possible to achieve relatively high classification accuracy in phase recognition during surgical procedures using machine learning techniques
• The experiment hepled to analyze the weight and hence the importance of different sensors
Conclusion
• Possible to achieve relatively high classification accuracy in phase recognition during surgical procedures using machine learning techniques
• The experiment hepled to analyze the weight and hence the importance of different sensors
• Our study gives important input for further research in design of suitable sensors for the OR
Thank you