COLLABORATIVE ROBOTIC SYSTEM FOR

HAND-ASSISTED LAPAROSCOPIC SURGERY

MEDICAL ROBOTICS

medicalrobotics.uma.es

Speaker: Dr. Enrique Bauzano Núñez

Authors: M.C. López Casado (mclopezc@uma.es)E. Bauzano (ebauzano@uma.es)I. Rivas Blanco (irivas@uma.es)V.F. Muñoz (vfmm@uma.es)J.C. Fraile (jcfraile@eii.uva.es)

Some Examples of Surgical Robots

MEDICAL ROBOTICS

medicalrobotics.uma.es

IEEE Spectrum, 2016. Source: https://www.youtube.com/watch?v=2fnv_3qn3Yc

Collaborative Robotic System for Hand-Assisted Laparoscopic Surgery 1 / 13

Classification of Surgical Robots by Autonomy

Collaborative Robotic System for Hand-Assisted Laparoscopic Surgery 2 / 13

GRUPO DE ROBÓTICA MÉDICA

www.roboticamedica.uma.es

• Remote Surgery

• Improvement of

surgeon’s abilities

• …

• Direct Surgeon’s orders

• Visual Guidance

• Virtual Fixturing

• …

• Biopsies

• Palpating,

Cutting and Gluing

• Hair-Graft

• Tumor radiation

• …

MIS Techniques – Laparoscopic Surgery

GRUPO DE ROBÓTICA MÉDICA

www.roboticamedica.uma.es

Collaborative Robotic System for Hand-Assisted Laparoscopic Surgery 3 / 13

MIS Techniques – Single Incision Laparoscopic Surgery

GRUPO DE ROBÓTICA MÉDICA

www.roboticamedica.uma.es

Collaborative Robotic System for Hand-Assisted Laparoscopic Surgery 4 / 13

MIS Techniques – Hand Assisted Laparoscopic Surgery

GRUPO DE ROBÓTICA MÉDICA

www.roboticamedica.uma.es

Collaborative Robotic System for Hand-Assisted Laparoscopic Surgery 5 / 13

How can we design a collaborative surgical robot?

GRUPO DE ROBÓTICA MÉDICA

www.roboticamedica.uma.es

Collaborative Robotic System for Hand-Assisted Laparoscopic Surgery 6 / 13

Enviro

nm

ent

Surgeon PatientRobot

Assistant

Enviro

nm

ent

Robot-W

orld

In

tegra

tion

Surgeon PatientRobot

Assistant

HMIPerception

System

Action

System

How can we design a collaborative surgical robot?

GRUPO DE ROBÓTICA MÉDICA

www.roboticamedica.uma.es

High-Priority Orders:

- Voice

- Haptics

- GUI…

Collaborative Robotic System for Hand-Assisted Laparoscopic Surgery 7 / 13

Sensors Processing:

- Image Processing

- Tools Location

- Hand Tracking…

Tools actions:

- Auto-Guided Tools

- Grasp actuation

- Tools Tracing…

How can we design a collaborative surgical robot?

GRUPO DE ROBÓTICA MÉDICA

www.roboticamedica.uma.es

Enviro

nm

ent

Robot-W

orld

In

tegra

tion

Cognitio

n

Long-Term Memory

Surgeon PatientRobot

Assistant

HMIPerception

System

Action

System

Procedural

Memory

Semantic

Memory

Decision

Procedure

Reinforcement

Learning

Working Memory

Collaborative Robotic System for Hand-Assisted Laparoscopic Surgery 8 / 13

How can we design a collaborative surgical robot?

GRUPO DE ROBÓTICA MÉDICA

www.roboticamedica.uma.es

Collaborative Robotic System for Hand-Assisted Laparoscopic Surgery 9 / 13

Cognitio

n

Long-Term Memory

Procedural

Memory

Semantic

Memory

Decision

Procedure

Reinforcement

Learning

Working Memory

DATABASE

Protocol States & Transitions,

Related Gestures/Actuations…

CODE

List of If–Then, Training algorithms…

REWARD–PENALTY Score Parametrized

expressions

Select which

If–Then statement has to be triggered

Current System State

Cognitio

n

Long-Term Memory

Procedural

Memory

Semantic

Memory

Decision

Procedure

Reinforcement

Learning

Working Memory

Process of the cognition layer

GRUPO DE ROBÓTICA MÉDICA

www.roboticamedica.uma.es

Enviro

nm

ent

Robot-W

orld

In

tegra

tion

Surgeon PatientRobot

Assistant

HMIPerception

System

Action

System

Collaborative Robotic System for Hand-Assisted Laparoscopic Surgery 10 / 13

1. Scissors tool appears on the image

2. IF (Scissors on image) THEN State =

State + 1

3. State + 1 defines camera target as “scissors Tool Tìp”

4. Actuation Algorithm: Camera follows

“Scissors Tool Tip”

5. Move the camera to center “Scissors

Tool Tip”

6. Surgeon tells if the action was correct

7. If required, updates the trained camera parameters

on memory

Cognitive Robotic Assistant concept for HALS

Gesture Recognition System Auto-Guided System

GRUPO DE ROBÓTICA MÉDICA

www.roboticamedica.uma.es

Collaborative Robotic System for Hand-Assisted Laparoscopic Surgery 11 / 13

Cognitive Robotic Assistant concept for HALS

Collaborative Robotic System for Hand-Assisted Laparoscopic Surgery 12 / 13

GRUPO DE ROBÓTICA MÉDICA

www.roboticamedica.uma.es

First approach of cognitive mini-camera assistant

GRUPO DE ROBÓTICA MÉDICA

www.roboticamedica.uma.es

Collaborative Robotic System for Hand-Assisted Laparoscopic Surgery 13 / 13

More information - Website

GRUPO DE ROBÓTICA MÉDICA

www.roboticamedica.uma.es

Collaborative Robotic System for Hand-Assisted Laparoscopic Surgery