tr&d proposal telesurgical robot simulator

8/8/2019 TR&D Proposal TeleSurgical Robot Simulator

http://slidepdf.com/reader/full/trd-proposal-telesurgical-robot-simulator 1/76



Application for ICT-Related Development and Research Grant Page II

Read carefully before filling the form.

1. Please do not alter the layout of the application form. Information

must be filled in the spaces provided, under set format.

2. Guidance notes in various fields should not be deleted.

3. Required information should be duly filled in the specified fields.

4. Required heads/fields which are not relevant to the project

should be marked N/A (Not Applicable) or left blank and should

not be deleted.

5. Specifications, justifications, purposes must be provided against

each item in the Budget file.

6. Please do not change the formulas in the budget sheets.



Application for ICT-Related Development and Research Grant Page III

List of Abbreviations and Acronyms

EE External EvaluatorsICT Information and Communication TechnologiesIPR Intellectual Property Rights

JPD Joint Project Director (Co-Director)PD Project DirectorPI Principal Investigator (Organization)

"Principal Investigator" means the person, company, partnership,undertaking, concern, association of persons, body of individuals,consortium or joint venture which receives funding from the Company to execute a research and dev elopment project.”

R&D Research and DevelopmentMIS Minimally Invasive SurgeryUI User InterfaceUAV Unmanned Air VehicleUGV Unmanned Ground Vehicle

List of Abbreviations and Acronyms Used by PI/PD in the Proposal

(Please add abbreviations and acronyms in the table below, if any.)



Application for ICT-Related Development and Research Grant Page IV

Application for ICT-Related Development and Research Grant

Guidelines and Forms

Introduction

National ICT R&D Fund was created in January 2007 by Ministry of IT with the vision totransform Pakistan's economy into a knowledge-based economy by promoting efficient,sustainable, and effective ICT (IT and Telecommunications) initiatives through synergicdevelopment of industrial and academic resources. Collaborative efforts betweenacademia, research institutions, and industry are greatly encouraged to ensure that localeconomy can reap the monetary benefits of investment in research. This organizationhas significant funds available for proposals that are geared towards creating ICT relatedtechnologies.

Research grants will be awarded for high-level and promising ICT-related developmentand research projects by individuals or groups from academia and/or industry activelyinvolved in the development and research individually or collaboratively. These projectsshould be based on either a universally known technology or a new technologydeveloped by the applicant and should be aimed at achieving economically viablesystems, products, or processes beneficial to the nation.

The grant will normally be provided for a period of up to two years – renewable everyyear based on the performance. The grant will cover the honoraria of the project and co-project directors, salaries of researchers, stipends for student research assistants, andsupporting staff. It will also cover travel(s) within and outside the country for project-related activities and for project-related scientific conferences. The grant may be used topurchase very specific unavoidable equipment kept to the bare minimum, consumablematerials, and other items needed for the project. The grant can also be used to coverthe training expenses of the personnel associated with the research project.

Submission Procedure

Duly filled applications in all respects along with any documents should be emailed [email protected]. A hard copy should also be submitted by registered post or byfax at our mailing address given below. On receipt of the applications the proposals willbe evaluated internally as well as externally as laid down in our policy documents. ThePD may need to revise the proposal in light of the evaluator‘s recommendations.

There is no deadline for submission of the application; however the application shouldbe submitted at least three months prior to commencing the project.

Joint Funding

The project proposal may be jointly funded by ICT R&D Fund and other fundingagencies/industry. The efforts to obtain joint funding will be at the discretion of thePrincipal Investigator (PI) – Organization to which Project Director belongs. However anysuch information must be provided to ICT R&D Fund. The funds released will beprovided to the PI.



Application for ICT-Related Development and Research Grant Page V

Agreement

A written agreement will be made between National ICT R&D Fund and PI. The PI willundertake to administer the grant according to the agreement and to provide laboratoryspace, and other facilities necessary for the project. The equipment purchased with ICTR&D Fund for the approved project will remain the property of ICT R&D Fund. The

laptops will be returned to ICT R&D Fund after completion of the project. The grantee isrequired to submit a final narrative and financial report within one month of thecompletion of the project. The IPR issues will be sorted according the policy in vogue.

For further information, please contact:

Solicitation and Evaluation Department,National ICT R&D Fund,6th Floor, HBL Towers,Jinnah Avenue, Blue Area, IslamabadTel.: (+92-51) 921 5360 - 65Fax: (+92-51) 921 5366

Website: www.ictrdf.org.pk

http://www.ictrdf.org.pk/






Application for ICT-Related Development and Research Grant Page VI

Table of ContentsDescription Page #

1. Project Identification ............................................................................................................... 1A. Reference Number: ............................................................................................................. 1B. Project Title:........................................................................................................................ 1

C. Project Director (PD): ......................................................................................................... 1D. Organization (The Principal Investigator (PI)): .................................................................. 1E. Other Organizations Involved in the Project: ..................................................................... 2F. Key Words: ......................................................................................................................... 2G. Research and Development Theme: ................................................................................... 3H. Project Status: ..................................................................................................................... 3I. Project Duration: ................................................................................................................. 3J. Executive Summary: ........................................................................................................... 3

2. Scope, Introduction and Background of the Project ........................................................... 5A. Scope of the Project: ........................................................................................................... 5B. Introduction: ........................................................................................................................ 6B1. Literature Review: ............................................................................................................ 11B2. Current State of the Art: ................................................................................................... 21C. Challenges: ........................................................................................................................ 21D. Motivation and Need: ....................................................................................................... 22

3. Objectives of the Project ................................................................................................... 25A. Specific Objectives Being Addressed by the Project: ....................................................... 25

4. Research Approach ............................................................................................................... 27A. Development / Research Methodology:............................................................................ 27B. Project Team: .................................................................................................................... 45C. Team Structure: ................................................................................................................. 46D. Project Activities:.............................................................................................................. 48E. Key Milestones and Deliverables: .................................................................................... 53

5. Benefits of the Project ........................................................................................................... 55A. Direct Customers / Beneficiaries of the Project: ............................................................... 55B. Outputs Expected from the Project: .................................................................................. 55C. Organizational / HRD Outcomes Expected: ..................................................................... 55D. Technology Transfer / Diffusion Approach: ..................................................................... 55

6. Risk Analysis ........................................................................................................................ 56A. Risks of the Project: .......................................................................................................... 56A1. Comments: ....................................................................................................................... 56

7. Contractual Matters ............................................................................................................... 57A. Contractual Obligations under this Project: ...................................................................... 57B. Ownership of Intellectual Property Rights: ...................................................................... 57C. Competent Authority of the Principal Investigator (Organization). ................................. 57

8. Project Schedule / Milestone Chart....................................................................................... 589. Proposed Budget ................................................................................................................... 59 Bibliography ................................................................................................................................. 60 Annexure A – Curriculum Vitae ....................................................Error! Bookmark not defined. Annexure B - Details of Hardware and Software Requirements……………………………….. 68Annexure C - Details of Work Packages, Milestones and Deliverables ……………………….. 76



Application for ICT-Related Development and Research Grant Page VII

References …………………………………………………………………………………….... 74Note: To update the table of contents, right click in the table and select ‗update field ‘ and then select

‗Update Entire Table‘.



Application for ICT-Related Development and Research Grant Page 1

Application for ICT-Related Development and Research Grant

1. Project Identification

A. Reference Number:

(for office use only)

B. Project Title:

Development of a Tele-Surgical Training Robot and Simulator

C. Project Director (PD):

Name: Dr. Osman Hasan

Designation: Assistant Professor

Organization: NUST School of Electrical Engineering & Computer Science(SEECS)

Mobile # : Tel. # : 92-51-9085-2137

Email: [email protected]

(A letter from the competent authority regarding project director’s time commitment for the proposed research project must be provided.)

C1. Joint Project Director (JPD):

Name: Dr. Asif Zafar

Designation: Professor of Surgery, Rawalpindi Medical College

Organization:Project Director Minimal Invasive Surgery & Telemedicine & e-Health Training Center, Holy Family Hospital, Rawalpindi

Mobile # : +92-300-5104404 Tel. # :+92-51-9290321 – Ext 2132

Email: [email protected]

C2. Contact Person: (If different from PD.)

Name:

Designation:

Organization:

Mobile # : Tel. # :

Email:

D. Organization (The Principal Investigator (PI)):




(Please indicate the name, address, telephone and fax of the Organization acting as Principal Investigator. The Project Director should also belong to this organization.)

Name:NUST School of Electrical Engineering and Computer Science(SEECS)

Address: Sector H#12, Islamabad

Registration #: (Please attach certified copy)

National Tax #:(Please attach certified copy)

Tel. # : +92-51-90852400 Fax # :

Website: www.seecs.edu.pk

E. Other Organizations Involved in the Project:

(Please identify all affiliated organizations collaborating in the project, and describe their

role/contribution to the project.)

E1. Industrial Organizations:

# Organization Name Role / Contribution 1.

2.

E2. Academic Organizations:

# Organization Name Role / Contribution 1. NUST School of Electrical Engineering &

Computer Science

Research & Development

2. Holy Family Hospital Research & Testing

E3. Funding Organizations:

# Organization Name Role / Contribution 1. National ICT R&D Funding

2.

E4. Other Organizations:

# Organization Name Role / Contribution

1.2.

F. Key Words:

(Please provide a maximum of 5 key words that describe the project. The key words will be incorporated in our database.)




Robotics, Surgical Robot, Simulator, Tele-Surgical Robot, Telemedicine

G. Research and Development Theme:

(If the proposal belongs to a theme specified by NICT R&D Fund, please identify the Research

Theme.)

<type here>

H. Project Status:

(Please mark )

New Modification to previous Project

Extension of existing project

H1. Project URL:(The project URL should be provided. This URL should be hosted by the project executing agency. Sufficient details such as executive summary, objectives are expected on the website.Once the proposal is approved, the website should also provide final copy of the proposal and deliverables/progress.)

I. Project Duration:

Expected Starting Date: December 2009

Planned Duration in months: 36 months

J. Executive Summary: Surgical robots have gained popularity recently due to the introduction of robots that canperform minimal invasive procedures as well as surgeries from remote locations. Theprimary goal of this project is to develop a low cost Tele-surgical robot and simulator fortraining of physicians. The basic requirement for such a project in Pakistan is to develop anenvironment where robots can be utilized in the field of medicine to perform minimallyinvasive surgeries. Surgical robots have become common in developed countries like USAand parts of EU because of their efficiency, reliability and enabling quick recovery of patients.Such robots are usually not easy to use in the beginning as they require extensive hands ontraining before being used on actual human patients. Therefore training systems andsimulators are required that can provide the surgeons with enough expertise before any reallife surgical operation. In this project we focus on the development of a training system which

consists of a Surgical Training Robot and a Simulator. This training system will initially servethe means to develop an environment where surgeons will become experienced with the useof robots for surgeries. As skills are acquired in this field by surgeons through training in MISand by ICT experts through designing and developing robots and software in this fieldeventually we will create human resource and support systems to perform minimal invasivesurgery procedures from remote locations in Pakistan.

Most training systems like ProMIS [6] available in the market cost thousands of dollars,




making it practically impossible for common use of such training systems in medical schoolsand teaching hospitals. Therefore, the main objective of this project is to develop a costeffective set of robots that can be initially for a period of several years used for training andeventually deployed where the infrastructure for performing complex surgeries is limited tohospitals in big cities only. These facilities for Tele-surgery will have to be augmented withpost operative care facilities. As the Tele-surgical training robot and simulator are put into

practice, they will train and produce experts in the area of robotic MIS procedures. In future,this will allow them to utilize these skills to perform Tele-surgeries, where they would becontrolling surgical robots from a remote location and performing MIS procedures. In a fewyears this will evolve into a system that will allow these trained surgeons to provide MISsurgical facilities to remote areas of Pakistan, without any risk of safety, saving infrastructureand development costs.

The proposed robot will be a combination of hardware and software systems, utilizing skillsand techniques in areas of robotics, image processing, computer game development andcontrol systems. The simulator will be developed using open source technology used ingame development, while the hardware of the training robot will be developed using partsthat are easily available off the shelf. This will be an integration of different technologies in

the area of robotics and software development. One of its advantages will be its user friendlyapplication software and it‘s easy to use hardware controls. Both these features shall help intrainees getting used to the system more easily and in quick time rather than handlingdifficult and hard to remember controls.

The simulator will run in two modes; Offline mode and online mode. In offline mode it willonly act as a simulator by displaying a virtual environment with 3-d images of organs andoperational procedures. Online mode will allow the user to control an actual robot while thereal imagery will be displayed from the robotic image acquisition device. Online mode couldbe used with the robot in close proximity or at a remote location. Furthermore, the onlinemode will allow the users to actually use the robotic arms on a mannequin (fake body) andpractice different procedures.

The surgical robot will provide the training skills for delicate control that is required to performsurgical operations such as laparoscopy. The robotic arms will allow the surgeons to becomeexperts at performing surgeries and control robotic arms in areas where a human handcannot reach, thereby reducing the risks of post-op infection and slow recovery. It will alsoprovide far better results and advantages than the traditional methods as stated by changMoo kang et al [1]. In the future this robot could be deployed at most remote locations inPakistan, which means that every remote location would have the facility of complex surgicaloperations performed by a trained and experienced surgeon. Since safety is one of the mostimportant issues, we plan to deploy this training system at the Telemedicine and E-healthtraining center Holy Family Hospital, Rawalpindi, where it will be tested and evaluated over acertain period of time. Once it will be verified that the surgical robot is capable of training

surgeons for robotic MIS procedures, we will move on to the next phase, where we willdevelop a fully functional Tele-Surgical robot capable of performing MIS procedurescontrolled from a remote location.

A cost effective and locally developed Tele-surgical training robot can bring about arevolution in the health care sector in not only Pakistan but most developing countries.Currently the robot in use in most developed countries is the da Vinci Surgical Robot [2],which costs about Rs. 160 million. This is the cost of a single robotic system. The extremelyhigh cost makes it almost impossible for developing countries to buy and implement this




technology in large numbers. The technology used by da Vinci is sophisticated andexpensive. We plan to develop a training robot using simpler technology as an initial step.Once the initial phase is complete and tested, a more sophisticated robot can be developedwith features similar to that of the da Vinci Robot that can be actually used for MISoperations. One of our main goals is to develop a training robot that is affordable by mostdeveloping countries. For this purpose we plan to use open source software tools for

developing the simulator and the robot software; and we plan to develop an indigenouslybuilt robot with inexpensive off the shelf parts that are commonly available.

The main value proposition of this proposal is that by creating synergy between knowledgeof expert surgeons, expert computer engineers and low cost hardware components createdfor video gaming industry extremely valuable training systems for minimal invasive surgerycan be created in the short term. This expertise can be further nurtured to develop highquality minimal invasive surgery systems in Pakistan at low cost and deploy these systemsthroughout Pakistan. Thus creating an ecosystem that improves health care while generatingsignificant R&D activities, foster innovation, create job opportunities and lead to wealthgeneration.

2. Scope, Introduction and Background of the Project

A. Scope of the Project:

Our goal for this project is to develop a training system comprising of a simulator and asurgical robot. The main idea is to develop a training system that can be used by surgeonsfor training of minimally invasive surgical procedures. There are two major objectives for thisproject. First is to train doctors to perform minimally invasive procedures with the use ofrobots and become experts in this area. Second is to test this robot and collect feedback asa preamble for the development of a Tele-Surgical Robot that can be used by thesesurgeons, in the future, to perform surgical operations from remote locations. Hence this

project is just the first of many phases that will continue to develop and improve on thisconcept while improving efficiency, accuracy, safety and precision.

This project will bring in expertise of different fields and group them together for thedevelopment and use of this project. Its development scope will include people in the field ofmedicine, robotics, image processing and control systems. In Pakistan there are no facilitiesavailable for the practitioners in the medicine filed to train themselves that involve the use ofrobots for procedures like MIS. We intend to address this problem by developing a trainingsystem that will allow the training as well as evaluation of surgeons on this system beforethey can go on to perform robotic MIS procedure in the operating theatre. Furthermore, thelow cost of the system will allow for it to be used in most medicine schools and teachinghospitals.

Currently similar robots for training MIS procedures are very expensive costing thousands ofdollars making them infeasible for countries like Pakistan. A low cost simulator and trainingrobot will allow the development of quality human resource and set the precedent fordevelopment of robotics in the health sector in Pakistan, encouraging indigenousdevelopment of training robots as well as robots in other fields.




B. Introduction:

In today‘s world robots have become common in many fields. Some examples of robotsbeing utilized in different fields are industrial robots, home cleaning (vacuum) robots, elderlycare robots, entertainment robots, military robots like UAVs and UGVs etc. Similarly for thefield of medicine there are surgical and health care robots. Surgical robots like da Vinci [2]

have improved the quality of healthcare by providing surgeons the ability to perform quickand efficient MIS procedures. In this project, our aim is to enhance the quality of healthcarein Pakistan by developing a Tele-surgical training robot and Simulator.

A Tele-Surgical robot enables a surgeon to perform surgeries or minimally invasive surgicalprocedures by controlling a robot from a remote location. There are several techniques whicha surgeon must master before he can successfully use a robot for an MIS procedure. Firstly,all the laparoscopic equipment that is used in an MIS operation is handled indirectly by thesurgeon. The surgeon uses controls that direct the movement of laparoscopic equipmentwhich is mounted or connected to three to four robotic arms. Secondly in a Tele-Surgicalrobot the robotic arms are connected to the control application over a network link. Thereforethere is a certain time lag before the movements of the surgeon‘s controls are transferred to

the robotic arms. Other than these there are other factors as well. For example, indirectcontrol of laparoscopic equipment involves high risk and safety issues. Network andelectrical failure are some of the things that can go wrong. A surgeon might not be able tomaster hand eye coordination, which is necessary in such procedures. Finally, a surgeonmight not have the necessary experience to perform a robotic MIS operation. This projectaims to solve all these problems by developing a training system which will allow thesurgeons to conduct practice runs and train themselves for all unforeseen and foreseencircumstances during a robotic MIS procedure. Figure 2.1 illustrates the working of a Tele-Surgical Robot.

Fig 2.1 Operation of Tele-Surgical Robotic System




The training system consists of two parts. A simulator and a surgical robot system. Thesimulator will be used for undergoing training of different scenarios. These scenarios willinclude performing different laparoscopic operations. The simulator will provide a virtualenvironment where the trainer will guide a surgeon to control the robotic arms of the virtualrobot model within the simulator to perform certain operations. These operations couldinclude cutting or removing of cancerous parts or isolating a certain organ from cancerous

tissue. Once the development is undertaken several training operations will be added basedon input from expert surgeons in the field of laparoscopy.

The second part of the trainer will be a surgical robot system. This will be an actual robotconsisting of robotic arms with laparoscopic equipment attached to it as end-effectors. Thesurgical robot trainer will give the trainee a ―real-world‖ experience of handling a surgicalrobot. The trainee would use this trainer robot to perform the same procedures performed inthe simulator on a mannequin (fake body). Robotic arms will be inserted into the body of themannequin and artificial organs would be manipulated by the trainee using the robotic arms.In this way, the training system will provide the trainee surgeon with two modes of learning.One using a virtual environment and one using a real-world environment, hence improvingthe level of expertise and adding to his experience of robot handling.

Robotic Surgery has several advantages over traditional methods. The following chartcompares the advantages of surgical robots over traditional methods [1].

A Tele-Surgical training robot and Simulator will help in producing surgeons with additionalskills of robotic surgery. The simulator could also be used to train surgeons for traditionalMIS laparoscopic procedures. A combination of different technologies for the development ofthis project will bring together people from different backgrounds. People from the fields ofengineering and medicine will combine their skills to develop this training robot, which in turnwill allow both sets of people to attain skills in the fields of others. Hence this project willcreate a synergy between different fields allowing people to enhance their horizons to many




fields and apply them to further create products that are unique and innovative.

The Concept

The concept of the proposed system is based on a two phase learning method. We will

develop a training system that will train surgeons using virtual reality and graphical methodsas well as using an actual robotic system capable of performing surgical procedures. Asimulator is used mostly when training on an actual system is not possible, or if it is possiblethen only a limited number of times. A typical example for use of simulators is by the peopleat NASA (National Astronautics and Space Agency) to train its astronauts for space flight.Militaries also use simulators to train pilots for helicopters and airplanes, saving huge fuelcosts.

Our concept for the simulator is similar to that mentioned above. In our scenario, it isimpossible and unethical to train surgeons on patients. A simple mistake can be lifethreatening. Therefore a simulator is necessary that can allow surgeons to train without thefear of committing life threatening mistakes. Surgeon‘s can commit unlimited mistakes andtrain for as long as required. A simulator in this case not only saves time and money but also

saves the surgeons from getting into potential life threatening situations.

Just as a pilot can never feel the actual physical features unless he flies a plane in reality,similarly a surgeon can never be perfectly trained unless he controls a surgical robotphysically. Keeping this scenario in mind our training system also consists of a training robotthat will allow the trainee surgeons to train on mannequins and get a feel for the physicalcontrol features of the robot. A combination of the two will train a surgeon for the bestpossible circumstances in a real life operation.

The Technology

In the field of robotics there are no standards for hardware or software development. One of

the development environments for robotics is Microsoft Robotic Studio which requires alicense. In this project our main focus is to develop a robotic system with minimal costs,making it cheap to purchase and deploy in large numbers. Therefore we will focus on opensource development tools and libraries to accomplish the different development stages of therobotic system.

The technology that we will use in the development of this robot will be the key to its ‘ lowcost. Using open source development tools and libraries along with cheap off the shelfhardware and electronics we plan to develop a reliable and efficient robot that will be easilyreconfigurable for tasks other than serving as a Tele-surgical robot. There is no special kindof hardware required for this project, except that the robotic arms will use off the shelflaparoscopic equipment that is specially developed for Minimal Invasive Surgeries. In casethe application of the client is different than a surgical robot, then the laparoscopic robotic

arms would be replaced by those that are required. Hence what we would get is a highly re-configurable remotely controlled robot that could be used for multiple purposes.

There are several stages of development of the many components in this proposed project.The three main components are the Simulator, Haptic UI controls and the Training Robot.We could have only developed a robot, but providing a simulator with it, gives manyadvantages over other available systems. The simulator will allow a user to get trainedbefore attempting any real-life operation. It will also help in simulating errors, failures and anyunforeseen circumstances that the robot might encounter. Whereas the training robot will




allow the users to experience the real world feel of working with a surgical robot.

Technology for the Robot and Simulator

As we mentioned one of our main goals is to develop a low cost robot, for which we intend to

use open source libraries and software development tools. Let us mention a few of the opensource libraries that will be used the development of this robot.

RoboIO Library

Roboard Library. This board will be used in the development of the SurgicalRobot System. It is provided with its own open source library for developmenthence saving any software API costs. Most code for interfacing the board withexternal hardware is handled using the available library therefore saving time fordeveloping interface drivers.

OpenGL

An Open Graphics Library package with plenty of documentation and examplesavailable online.

OpenCV

An Open source Computer Vision library providing functions such as imageloading, filters and many other computer vision methods.

VXL

Another open source Visual ―something‖ library with a large number of imageprocessing, mathematical and matrix operations along with several otherfunctionalities provided. All open source and available for commercialdevelopment without any licensing.

As far as the hardware is concerned, a custom robotic arm frame will be developed based onDynamixel‘s AX-12 actuator specially designed for high torque robotic arms. Even though wedon‘t need high torque, but these actuators also provide precision movement andtemperature feedback incase the motors become hot due to high load. For the grippers orthe end-effectors we will use commercially available laparoscopic equipment. The reason forusing off the shelf laparoscopic equipment is a very straight forward one as such equipmentis specially developed for MIS operations, and we would not need to develop any specialhardware as our motive is to perform MIS operations.

We will be using a custom designed embedded system as the heart of the robot, which willbe responsible for interfacing the robotic arms. The brains of the robot will be the Surgeon‘s

Console PC Application software, which will do all the calculations and any image processingthat might be required. We will also develop haptic (feeling feedback) user interface controls.These custom developed controls will give the operator of the robot a feel for the controlmovements and force applied.

The simulator will be developed using OpenGL using C++. The same input controls will beused to control the virtual robotic arms in the simulator as the actual ones. The simulator willbe programmed to provide different training scenarios for the surgeons to train on. It will be atool used for training of surgeons for MIS procedures before trying their hands on an actual




subject using the robot.

Application Scenario

In this section we describe scenarios where this project is applicable. This scenario will show

how this project is useful in a number of fields and how it will create an impact on peoplefrom different fields and backgrounds.

The training robot and simulator being cost effective can be deployed at medicalcolleges and teaching hospitals. It will allow the medical students to get familiar withthe latest in technology as well as get hands on training on simulators for future use.

The simulator and robot can be modified and used for other purposes as well. Forexample this training system could be used by engineering students as a case studyfor research. They could modify it and use it for other purposes, like controlling arobot remotely using this technology or for developing remote monitoring systems;the possibilities are limitless.

The same technology can be used to develop simulators for other systems as wellsuch as surgical robots for typical surgeries other than minimally invasive surgeries.

Computer scientists can use image processing techniques to enhance the reality inthe simulator, by integrating real video with the simulation (augmented reality).

In this project we are only focusing on laparoscopic surgery training using MIS procedures.At later stages we intend to extend the capabilities of this robot to perform other functions aswell. There are plenty of ideas and application scenarios that can be implemented on thisrobot in not only the field of medicine but other fields as well.

Implications and advantages of this project

This project has far reaching implications and great advantages. Once successfullyimplemented and tested, it can be commercially produced and deployed in most rural areasof Pakistan. Some of the future implications and advantages of this project are

Improving the lives of people by providing health care facility in their own localities

Improving the health care facilities in rural and urban areas of Pakistan

Training quality human resource in robotic MIS procedures

Providing MIS procedures by expert surgeons in all remote areas of Pakistan

Saving millions of rupees in infrastructure costs

Earning foreign exchange for the national exchequer by exporting this product toother countries

Setting a trend for the development of robotics in the field of medicine for otherpeople to follow




Developing a robot that can also be used in areas other than surgical operations, e.g.remote handling of hazardous material

This project has tremendous potential to be a successful product in many fields. We intend tostart with this project and move on to develop other products based on this core technology.

Our motivation is for the use of robotics in every field possible, saving costs, lives andhelping the advancement of Pakistan in the engineering and academic sectors.

B1. Literature Review:

(Detailed summary of what all has been done internationally in the proposed area quoting references and bibliography.)

The field of surgical robots is a relatively new field as compared to other robotic fields. Someof the robots that are being used right now in countries like the US have only beendeveloped in the last 5 to 10 years. As a result this is a very new field with unlimitedpotential. In this section we review some of the surgical robots that have been developedand are being used in actual operations.

1.1 Existing Surgical Robotic Systems

I) da Vinci Surgical Robot

The da Vinci Surgical System developed by Intuitive Surgical [2] is the mostadvanced surgical system available. It consists of an ergonomically designedsurgeon‘s console, a patient-side cart with four interactive robotic arms, the high-performance InSite ® Vision System and proprietary EndoWrist ® Instruments. In daVinci the surgeon‘s hand movements are scaled, filtered and seamlessly translatedinto precise movements of the EndoWrist Instruments, which is a technology

developed by Intuitive Surgical.

Fig 2.2 The da Vinci Surgical Robot System

Components of the da Vinci Surgical System

Surgeon Console




Patient-side Cart

EndoWrist Instruments

Vision System

The da Vinci Surgical System is the only commercially available technology thatcan provide the surgeon with the intuitive control, range of motion, fine tissue

manipulation capability and 3-D visualization characteristic of open surgery, whilesimultaneously allowing the surgeon to work through tiny incisions typical ofminimally invasive surgery.

Fig 2.3. An example of one of robotic surgery‘s greatest benefits—―key hole‖ incisions, asseen in this prostatectomy illustration. The smaller incision means faster healing, reducedbody trauma, infection risk and shorter hospital stays

II) Mako Surgical Robot System

Mako [3] developed the Robotic Arm Interactive Orthopedic System or RIO. RIO™assists the surgeon in achieving natural knee kinematics and optimal results at alevel of precision previously unattainable with conventional instrumentation.

Some of the features that this system provides are

Accurately plan implant size, orientation and alignment utilizing CT-derived 3-Dmodeling

Enabling the pre-resection capture of patient-specific kinematic tracking through fullflexion and extension

Real-time intra-operative adjustments for correct knee kinematics and soft-tissuebalance

Minimally invasive and bone sparing, with minimal tissue trauma for faster recovery




Fig 2.4. Mako Surgical System

Mako Surgical Robot System is different from da Vinci as it is limited to knee surgeries, butthe common points between MAKO and da Vinci are that both allow surgeons extra handsand increase their efficiency at performing Minimally Invasive Surgeries. Minimally InvasiveSurgical operations are far more efficient in early patient recovery and reduce chances ofpost operative infections as the body of the patient is not open during surgery. A robot furtherfacilitates the patient by allowing the surgeon to perform surgery within the body withoutdamaging any other areas and with better mobility.

III) NeuroArm

Fig 2.5. NeuroArm

NeuroArm has been developed by Garnette Sutherland [4], Professor of Neurosurgery,University of Calgary and Calgary Health Region, and his group. It is a MRI-compatible

http://www.neuroarm.org/people.php

http://www.neuroarm.org/people.php




image-guided computer-assisted device specifically designed for neurosurgery. It performsboth microsurgery and biopsy-stereotaxy applications. The system includes a workstation, asystem control cabinet, and two remote manipulators mounted on a mobile base. For biopsy-stereotaxy, either the left or right arm is transferred to an extension board that attaches to theOR table and the procedure is able to be performed inside the MRI bore.

NeuroArm includes two MR compatible manipulators with end-effectors that interface withmicrosurgical tools. It includes filters to eliminate unwanted tremors. End-effectors areequipped with three-dimensional (3D) force-sensor, providing the sense of touch. Thesurgeon seated at the workstation controls the robot using force feedback hand controllers.The workstation recreates the sight and sensation of microsurgery by displaying the surgicalsite and 3D MRI displays, with superimposed tools.

The system includes a workstation, a system control cabinet, and two remote slave-manipulators on a moveable base platform. The anthropomorphic arms have 7 degrees offreedom, are MR compatible and designed to hold a variety of surgical tools. The endeffector is designed to hold a variety of tools within a standardized interface, and allows fortwo separate operator controlled manipulations: tool roll and tool actuation.

IV) RP-7 Robot

The InTouch [5] RP-7, where RP stands for Remote Presence, is a mobile robotic platformthat enables the physician to be remotely present. Through the integration of keytechnologies, the Remote Presence Robotic System can remove time and distance barriersand effectively extend the physician's reach to manage patient care. The Robot's panoramicVirtually There visualization system, combined with the Holonomic Drive System, theSenseArray System 360, and the easy-to-use control interface, affords physicians, patientsand hospital staff a safe and effective interactive experience.Devices such as electronic stethoscopes, otoscopes and ultrasound can be connected to theExpansion Bay of the Robot, to transmit medical data to the remote physician.




Fig 2.6. System Architecture of RP7

1.2 Existing Simulators

1.2.1 LapSim Simulator

LapSim surgical simulator is a Laparoscopic Surgical Simulator that has extensive trainingscenarios for minimally invasive procedures. It provides basic and advanced trainingtechniques for keyhole surgery including suturing and anastomosis, cholecystectomy etc.

The simulator utilizes advanced 3D technology, including interactive live video, to provide thestudent with a realistic virtual working environment. Practice sessions can vary in graphiccomplexity as well as in the level of difficulty. Courses can easily be created or modified bythe teacher to fit a student´s specific needs, or the requirements of a training curriculum.

This simulator is available and is in use by Tele-medicine and E-health training center, HolyFamily Hospital Rawalpindi, for training of surgeons in minimally invasive procedures.

1.2.2 ProMIS Surgical Simulator

ProMIS is a surgical simulator developed by haptica [6] for training surgeons to performMinimally Invasive Surgeries. The simulator can be used with a physical mannequin toactually perform a virtual operation. The simulator also allows for evaluation surgeons beingtrained. Multiple simulators can be connected to compare their results of groups of studentsat a time.




Fig 2.7. ProMIS Simulator

ProMIS uniquely enables users to interact with virtual and physical models in the same unitwhile providing accurate, comprehensive feedback on performance. In validation studies, ithas been consistently shown to perform better than pure virtual reality simulators

1.2.3 Laparoscopy Virtual Reality System

Developed by immersion [16] this simulator has been developed for training in laparoscopicand minimally invasive surgical procedures. The simulator has hardware that includesstandard laparoscopic instruments to be used with different training scenarios. Trainingscenarios include essential skills like camera navigation, cutting, clipping, needle driving,knot tying and general surgery skills like Cholecystectomy.




Fig 2.8 Laparoscopy VR Simulator

1.3 Available Open Source Software technology

In this section we list some of the development that has been undertaken in the field of opensource simulator development. We specifically list open source projects, as our project alsofocuses on development using open source software technology. The advantage of theavailable open source technology for our project is that, we can use this existing technologyby using code that is already available under general public license, and build on it todevelop our own surgical simulator.

1.3.1 SPRING Surgical Simulator

SPRING is a real-time surgical simulation system with soft-tissue modeling and multi-user,multi-instrument, network haptic [17]. Spring can be used for training surgeons, buildingsurgery skills, surgical rehearsal etc. It is an extensible open source platform for developingscenarios and simulations. SPRING is especially intended for training in minimal accesssurgery in which indirect viewing and manipulation play key roles.

SPRING is written in highly portable standard C++, and can be compiled under Windows2000 and XP, LINUX, and UNIX (SUN Solaris, SGI Irix, Macintosh OS X). SPRING'sarchitecture includes support for multi-platform and multiprocessor systems, automaticallysensing the number of available processors. Specific interfaces for 3-D digitizers, hapticfeedback systems, and other inputs are supported via interface servers, which connect tospecialized input/output devices. These servers then communicate with the SPRING

simulation program via TCP/IP messages send through local or wide-area networks. Thisseparation of interaction device and computation allows several important capabilities:

use of interfaces that are not available on all computer systems, separation of CPU-intensive numerical interactions from I/O devices, coordination and cooperation of multiple users across the internet. Access to remote resources such as powerful processors




1.3.2 GiPSi (General Physical Simulation Interface)

GiPSi [18] is an open source/open architecture framework for developing organ level surgicalsimulations. The goal of GiPSi is to provide an open architecture software development

framework to facilitate free exchange of models and algorithms, and development ofinteroperable medical simulations, with special focus on the requirements of virtualenvironment-based medical training simulations.

GiPSi is a general software development framework for developing organ level surgicalsimulations, rather than a complete simulation engine. The framework consists of theSimulation Object API, which also includes the object interfacing API, the Visualization APIand the Haptics API. The implemented Modeling Tools and Computational Tools form aninitial set of GiPSi compliant libraries to support development of GiPSi based simulations.

1.3.3 Irrlicht Engine

The Irrlicht Engine [19] is an open source high performance realtime 3D engine written andusable in C++ and also available for .NET languages. It is completely cross-platform, usingD3D, OpenGL and its own software renderer, and has all of the state-of-the-art featureswhich can be found in commercial 3d engines. It can be used for developing 3Denvironments necessary for game development or simulator development.

This engine has all the features available for developing a simulator package, which includesa physics engine, rendering engine, collision detection and interaction with input devices likemouse, keyboard and joysticks. Being open source this engine is ideal for developingsurgical simulators, by using existing code and extending it to suit our developmentrequirements.

1.4 Available Hardware technology

There are several hardware components that are required when developing a robot. Thesemay include motors, microprocessors, microcontrollers, sensors, power supplies etc. For thefabrication of the robot, at an initial stage material like aluminum metal sheets can be used todevelop a prototype. Since this is a research based project, we want to use tools andhardware that is very low cost and durable. In this section we list some of the availabletechnology that can be used for the development of our training robot.

1.4.1 Motors

There are several kinds of motors available that can be used for developing the surgicalrobot system. Motors like servos, dc motors can be used to develop the robotic arms linkstructures. Usually hobby servos are used by robotocists for developing custom robots. Inthis project we want to focus on accuracy and low cost. Therefore we would like to useservos that would provide high torque and are cost effective. Some of the types of motorsthat are available are




DC Motors (Brushed/ brushless) Servos Stepper Motors

1.4.2 Microcontrollers

Depending on the type and complexity of a robot, it can either be based on low endmicrocontrollers or high end microprocessors. Some of the boards that are available andbeing used for development are

i) AVR Microcontrollers

Atmel‘s [20] AVRs are RISC architecture microcontrollers with separate program anddata stored in different memory spaces. AVRs are generally categorized in threegroups. TinyAVR, megaAVR and XMEGA. These microcontrollers can support clockspeeds of up to 32Mhz. AVR microcontrollers are used for robotics development and

for developing real-time critical applications.

ii) PIC Microcontrollers

PIC microcontrollers are also very popular for development of real time applicationsand embedded systems. They are popular with developers and hobbyists alike due totheir low cost, wide availability, large user base, extensive collection of applicationnotes, availability of low cost or free development tools, and serial programming (andre-programming with flash memory) capability. They are not expensive and are veryuseful when developing custom small and medium size electronic modules.

iii) 8051 Microcontrollers

8051 series of microcontrollers is also one of the most used series of microcontrollers.It is also used extensively in embedded systems and robotics projects. It has beenreplaced by AVR microcontrollers.

iv) Axon Microcontroller

Axon Microcontroller [13] runs an AVR microcontroller with 55 I/O ports and isspecially designed for robotic applications. It is mostly used by hobbyists and roboticsdevelopers.

1.4.3 Microprocessor boards

Usually microprocessors are expensive and have more computing power thanmicrocontrollers. Therefore they are usually used when high processing power is required.With clock speeds ranging from 200Mhz to 1.2 GHz microprocessor boards are very usefulfor developing high end robots running real time operating systems.




At the moment a 1.5 million dollar robot is unaffordable by any hospital in Pakistan. Acheaper solution will allow these training robots to be deployed in large numbers.

Once these training robots are deployed at rural and urban training centers, surgeonscan acquire robotic and laparoscopic surgical skills in their hometowns instead ofmoving to urban areas. Once these surgeons are trained they can work in their own

rural areas, solving the problem of unavailability skilled doctors in rural areas.

Patients would save traveling and living costs, since they would not require anytraveling to urban areas for laparoscopic surgical procedures.

Commercialization of this technology will result in considerable earn of foreignexchange for the country. Third world and developing countries that cannot affordexpensive surgical robotic systems will always be open to cost effective alternatives,allowing us to capitalize on this huge market.

Conclusion

There are several robots available commercially but very few of them integrate the two mostnecessary requirements, i.e. A robot with its own simulator. We intend to provide both withour Surgical Robot System. Furthermore most surgical robots and commercial simulatorsmentioned in this section cost thousands of dollars. We intend to develop a low cost systemthat will not only be a great product for third world and poor countries but will also be a greatproduct commercially.

B2. Current State of the Art:

(Please describe the current state of the art specific to this research topic.)

In the previous section, the current state of the art in Surgical Robots has already beenmentioned. Most of these robots are very expensive and are out of the reach of people aswell as governments in the third world countries. This project is a combination of the currentstate of the art technologies of ProMIS simulator and the da Vinci Surgical Robotic System.The simulator will allow training of surgeons before they can actually use the robot, it will alsohelp in evaluation of their skills. The Tele-Surgical Robot will allow remote surgicaloperations practically removing any infrastructure costs that are required to setup a locationfor MIS operations using traditional methods.

C. Challenges:

(Please describe the challenges, specific to this research topic, currently being faced internationally.)

There are several challenges that need to be met in order to develop a successful SurgicalTraining Robot and a Simulator for use by people in the field of medicine. Some of them areA robot and simulator system should

be safe and easy to usehave cheap maintenance and running costs




save cost for training of medical studentsreduce training time for studentsallow for training of many procedures, not only oneallow the students to train under different scenariosevaluate the performance of students

D. Motivation and Need:

(Please describe the motivation and need for this work.)

Pakistan is a developing country, currently involved in tackling a large number of problems atdifferent levels. On one end is the fight against miscreants planning against Pakistan, and onthe other is the fight to improve the living standards of the people by providing qualityeducation, healthcare and jobs. Our motivation for this project stems from all thesechallenges. A successful Tele-Surgical Training Robotic System is a great way forengineering and medicine students to learn about the latest in technology and engineering.With this project they will publish research papers in relevant conferences and journalsthereby gaining international exposure. It will improve the healthcare system in Pakistan bytraining surgeons to perform robotic surgeries in the future. It will also create many jobs, asbeing low cost, such a product has great potential for world markets.

The following statistics show how robotic surgery has improved over the traditional ways oflaparoscopic operations and the motivation for developing such a system. The data has beentaken from the Mt. Sinai School of Medicine, New York, USA [7]. It must be mentioned herethat Mt. Sinai School of Medicine uses robotic surgery.

Blood Loss Comparison

Laparoscopic and robotic surgeons make keyhole incisions, which are much smaller, muchless invasive and nearly bloodless. The equivalent of a few tablespoons of blood is lostduring robotic prostatectomies. When a large incision is made, blood loss is significant andtransfusions are sometimes required.




Fig. 2.8. Blood Loss Comparison

Complication Rates

Complication rates are far higher in open than robotic prostatectomies. Because far moretissue is exposed during open surgery, infection rates are higher.

Fig. 2.9 Complication Rates




Time in OR

There‘s been a significant decrease in operating room time with the use of robotictechnology.

Fig. 2.10 Operation Room Time

Postoperative Stay

Robotic prostatectomy patients are released from the hospital after one day in nearly 100%of cases.




Fig. 2.11 Length of Postoperative Stay

3. Objectives of the Project

A. Specific Objectives Being Addressed by the Project:

(Please describe the measurable objectives of the project and define the expected results. Use results- oriented wording with verbs such as ‘to develop..’, ‘to implement..’, ‘to research..’, ‘todetermine..‘, ‘to identify..’ The objectives should not be statements and should not include

explanations and benefits. Fill only those objectives that are applicable to the proposed project .)

A1. Research Objectives: (if any)

To develop a cost-effective Tele-surgical training robot and simulator that can be used totrain surgeons for robotic MIS procedures. Once developed and tested this will befurther researched and developed in the future for use and deployment in rural andremote health centers or hospitals of Pakistan. In doing so we will be publishingresearch papers in international conferences / journals highlighting our research.The following is a list of research areas that we will focus on during the project

Design Research of Robotic ManipulatorsDesigning robotic arms around standard laparoscopic equipmentDesigning the base, robotic mechanical structure suitable for operation

theatre use and robotic arms suitable for use in hospitalsLinking standard laparoscopic equipment with servo motors by transferring

torque using strings Design Research of Simulator Design

Simulation and modeling of internal organs of a human bodySimulation of blood flow inside the human body




Soft tissue modeling Design Research of Haptic User Interface Controls

Designing user interface controls that can provide feeling feedback to userssimilar to actual laparoscopic surgical procedures

Using input from haptic devices for generating real-time feedbackDesigning the mechanical structure of input controls that are similar to

industry standard laparoscopic equipment

A2. Academic Objectives: (if any)

To develop a simulator that can train medicine students to perform minimally invasivesurgeries using a robot. And to create a bridge between industry and academia byinvolving engineering and medicine students to work on different aspects of the project.This project is collaboration between Telemedicine and E-health Training Center, HolyFamily Hospital and NUST (National University of Sciences & Technology) School of

Electrical Engineering & Computer Science (SEECS). One of the objectives would be topublish research papers in international conferences and journals giving the opportunityfor Pakistani students to gain international exposure in the area of robotics research anddevelopment.

A3. Industrial Objectives: (if any)

To develop an indigenous low cost Tele-surgical training robot and simulator, affordableby developing and poor countries for cost effective training; And promoting indigenousdevelopment of robotics and bio-medical equipment.

A4. Human Resource Development Objectives: (if any)

As a result of this project we intend to setup a high-tech startup company that willdevelop this product and commercialize it. We will also focus on developing otherprojects based on the same technology. For example, we would develop robots in thecivilian sector as well as for military applications. We will utilize the engineering skillsthat are available in Pakistan as well as the immense talent pool of human resourcesavailable for utilization in every field.

A5. Other Objectives: (if any)




4. Research Approach

A. Development / Research Methodology:

(Please describe the technical details and justification of your development and research plan.Identify specialized equipment, facilities and infrastructure which are required for the project and their utilization plan.)

4.1. IntroductionThe basic idea of the project is to promote quality healthcare by providing training for minimalinvasive surgical procedure to surgeons. This is possible by developing a training systemthat includes a Simulator and a Surgical Robot System which are used for training ofsurgeons for real world robotic MIS procedures. The initial phase of the project involvesdeveloping a training system that can be used for minimal invasive procedures using roboticcontrol. In the future the capabilities of the system will be extended so that it becomes morethan a trainer and can be used in actual Tele-Surgical operations in rural and remote areasof Pakistan.

Fig. 4.1 The overall working of a Tele-Surgical Robot System

Figure 4.1 depicts the two training phases of the system. A simulator which trains the usersin a virtual environment, while a robotic system that will train the users by performing

procedures on a mannequin by controlling movements of the robotic arms.

For the development of this project we will follow the methodology of research in design,development and integration rather than developing everything from scratch. Our trainingsystem will be mostly software controlled; therefore the core technology developedindigenously will be the software and the mechanical design of the robot. The most importantarea of the development would be design research. As every module of the system whethermechanical or software would be designed from scratch, a lot of brainstorming and designresearch is required, which will use major part of the time duration of this project.




Most of the electronic parts will be used off the shelf, but since we will focus on developmentusing a modular approach, any part that becomes unavailable over a period of time could beeasily replaced by a similar part. Moreover at the current stage we will use some embeddeddevelopment boards by purchasing them; but at later stages we would develop most of theboards in Pakistan as well. Once the overall goal of the project is achieved by developing

and testing a Surgical Robot and Simulator Training system, we will move on to develop acompletely indigenously developed system with most modules made in Pakistan.

4.2 Development Methodology

We have divided the development of our project into three parallel phases of development.Within each phase there are multiple stages. The three major phases are

1. Development of the Surgical Robot2. Development of the Simulator3. Development of the Haptic User Interface

All phases of development are crucial for the completion and success of the whole project;therefore we have subdivided the three phases into multiple stages, each stage containingseveral tasks. Please refer to annexure C for a detailed description of the development plan.

Fig. 4.2 Workflow of the Project

SimulatorDevelopment

Surgical RobotDevelopment

Haptic UserInterface

Development

Planning, Analysisand Design Year 1

Development andIntegration

Year 2

Testing &Deployment Year 3




4.3. Research Methodology

As mentioned in section 4.1, a major part of this project is design research. We are focusingon the development of a whole training system, which has not been developed by anyone inthe world. There are separate modules available by different companies, e.g LapSim by [23]and daVinci by [2], but if we talk about a complete training system with a surgical robot alongwith a training simulator which also includes Tele-operative control, then there is no systemlike this available anywhere in the world. Furthermore, simulators that are availablecommercially are only for training of MIS procedures. There are no training systems availablefor training of surgeons for robotic surgery and Tele-robotic surgery. As mentioned in thearticle [25], there is a great need for a training system that can not only provide training forMIS procedures but also for robotic surgical procedures. We have divided our designresearch into the following design problems and tasks

1. Design of the Robotic Arms2. Design of control mechanism of manipulators (end-effectors)

3. Design of Haptic (feeling feedback) User Interface Controls4. Design of simulator for training of MIS and robotic surgical procedures

1) Design of Robotics Arms

A lot of design research will be required to find the right design of the robotic arms.Usually when a new product is designed, existing products can be researched andimproved upon. In our case this scenario is not possible, as most of the surgicalrobots that are commercially available are very expensive, and their designs aretrade secrets of their respective companies. Therefore before considering thedevelopment a lot of variables will have to be taken into the design research. Forexample, the following factors would need to be considered in the design of the

robotic arms

the best configuration of robotic arm that could be easily mounted over thepatient‘s bed,

ease of movement and degrees of freedom of the robotic arms number of joints, directly effecting the degrees of freedom how to design the robotic arms around the design of existing laparoscopic

equipment

As we want the robotic arms to work with the industry standard laparoscopicequipment, we would be using existing industry standard laparoscopic instruments(refer to annexure B for pictures and details). This will save us from re-inventing the

wheel and make our robotic system compatible with existing industry standards.

2) Design of control mechanism of manipulators

As already mentioned, we will design our robotic arms around existing industrystandard laparoscopic equipment. For this reason we would require research indeveloping an efficient and precise control mechanism for controlling the mechanismof the manipulators. One of the major design problems for this is to transfer the




control torque of a motor to the end-effector, as motors cannot be mounted directlyon the laparoscopic equipment. One design that can be used is control by usingwires and gears. Many designs would need to be researched before the mostefficient design can be found and implemented.

3) Design of Haptic User Interface (UI) Controls

One of the major things that are missing from existing simulators and surgical robotsis haptic or feeling feedback. When surgeons use existing simulators they cannot getany feedback from their input controls. This causes the surgeons to apply more orless force when working with real laparoscopic instruments as beginners, which canresult in fatal life threatening situations.

The haptic user interface module of our training system will solve this problem byproviding the surgeon different kinds of force feedback, depending on the input forcethat is applied by a surgeon. This will allow the surgeons to train for real lifelaparoscopic surgery. To develop force feedback input controls is once again is a bigresearch design problem. Existing designs will need to be studied. One of the

designs that we would study and use as a guideline are the fore feedback control joysticks that are available commercially. Different output signals can be generatedbased on the different input signals generated by the surgeon‘s hand movements.Every signal would be unique signifying a certain level of force that has been applied.

4) Simulator design for training of MIS and robotic surgical procedures

Another major part of design research would be the design and development of asimulator. There are hundreds of factors that need to be considered in the design ofa simulator. A number of these factors would be researched by the surgical team,while other factors relating to the actual performance of the simulator would bestudied by the developers. Some of these factors are

How to model and simulate human internal body tissue How to simulate blood flow How to model and simulate the interactions of external instruments and

internal body organs How to display the view that is usually available form an endoscopic camera

in MIS procedures etc

4.4. System Architecture

The project will be developed using a modular architecture approach. This allows different

modules to be attached or detached to the system depending on the application area of therobot and needs of the users. The following figure illustrates the modular architecture of thesystem. The arrows show the interaction between different modules of the system.




Modular Architecture of the System

Fig. 4.3 Modular Architecture of the Surgical Robot System

As shown in figure 4.3, every module is independently responsible for its own function.Although some modules are necessary but some can be removed. For example if someusers do not require the simulator then they can remove that module. Similarly if a wirelesslink is required then the interface link can be changed to a wireless interface link.

4.5 Simulator Architecture

The simulator will allow the users to train using real scenarios by looking at a virtual

environment and controlling the robot within this environment to get a feel for, and gain real-time experience of, working with the robot in real life. The robot controls within the virtualenvironment should provide the user with the same feeling and feedback as the realenvironment. This is crucial for the success of the system, as this simulator will be used toteach the students to perform real laparoscopic and robotic operations on patients.

The development of the simulator has been divided into the following stages

1. Modeling of the environment using OpenGL [9] 2. Modeling of the internal human tissue and body organs using 3D studio Max 3. Developing the robot control application module in the simulator application package 4. Development of real-life scenarios for the training of medical students for laparoscopy

or other surgical operations

4.5.1 Software Methodology for development of the Simulator

a) Using the Agile Software Development methodologyFor the development of the Simulator we will use the Agility Software Developmentmethodology. This method requires the building of a prototype application then taking it

Surgical Robot Training System

and Simulator

Mechanical

Module

User Interface

Module SoftwareModule

Network InterfaceModule

H/W Control Module

SimulatorModule




through several development stages, each time improving it after acquiring feedbackfrom the customer or user. This method is more suited to our way of development as it isused for quick development of the software and requires small teams working ondevelopment and design.

Fig. 4.4 Software Methodology for the development of Simulator

b) Developing the virtual environmentThe virtual environment will be developed using OpenGL, since using an Open source

library does not require any commercial license. Visual C++ will be used for theprogramming of the environment, while controls and APIs will be developed to interfacethe control hardware to be used with the virtual environment of the simulator. Thefollowing diagram explains how the controlling hardware will interact with the simulator

Agile Software Methodology for the Simulator

InitialProtot e

ContinuousDevelopment

Cycle

Final WorkingA lication

ClientFeedback




Fig. 4.5 Basic Working of a Simulator

c) Development of the GUI and Controls

The graphical user interface of the simulator will allow easy interaction with the simulator.Furthermore it will provide all the controls and the training tests that a user will be able toload and practice on. For the reality effect of the simulator sensory feedback will begenerated on the user controls as in the real control of robot.

d) Controlling the robot in the virtual environment using Haptic UI controls

The robot will be controlled by the same haptic input controls in real-time and in thesimulator as well. This will help the user in getting used to handling the controls of therobot. The robot control application will work in two modes; Offline and Online. Offlinewill allow the user to use the same controls with the simulator while online mode willallow him/her to use the controls to control the real surgical robot.

e) Interface driver between robot and control applicationOnce the simulator‘s environment will be developed code will need to be written toconnect the robot controlling hardware to the simulator. This will be done using a

standard USB port. The haptic user interface controls will be used to control the robotdirectly as well as over a wireless link. These controls would also be used to interact withthe simulator and control the robot within the virtual environment.

4.6 Architecture of Surgical Robot System (SRS)

We divide the development of SRS consists of the following stages

Development of Software Components




Development of Mechanical (Hardware) Components

4.6.1 Software ComponentsThe software architecture for the SRS has been divided into different layers dependingon the functionality of a particular part of the software.

Fig. 4.6 Layered Architecture

i) Top LayerThe top layer is the main application software layer. This is the most intelligent part ofthe robot as all the complex calculations and processing is done at this level. Anyintelligence that is added to the robot is processed at the top layer. For example theshaking of surgeons‘ hands is removed by applying filtering techniques at this layer. Thecontrol application operates at the top layer being the brains of the robot.

Control Application SoftwareThis is the main control application for robot command and control. It will be developed

using Microsoft‘s Visual Studio Development tools. The application will interact with thecontrol hardware or interface between the robot and the Control PC. The following figureillustrates the role of each layer of the software for the SRS.

Fig. 4.7 Computational Complexity of the layered architecture

Lower

Layer

Top Layer

Interface

Layer




ii) Interface Layer

a. Hardware Interface between modulesThe interface will be the driver that will connect the PC Control application to the robotcontrol hardware application. Using the modular approach for development, there will beseveral modules developed separately. Each module will be able to communicate withother modules. This communication is either wired or wireless. For wired communicationinterface circuits will be developed that will create an interface between differentmodules and allow communication.

b. Network Interface between Robot and Control ModulesThe network interface will allow communication between different modules of the systemover a network link. The simulator will be connected to the surgical robot system by anetwork link over a wired or wireless connection.

It will act as a communication interface between the application software and the

embedded software defining all the communication protocols and making sure of real-time control of the robotic system by the user. The communication link used at thehardware layer which will enable the communication of the robotic control applicationwith the robot from a remote location is discussed in section 4.8.

iii) Lower or Hardware Layer

Robotic Controller Driver (Embedded System)The lower layer will be an embedded system that interacts with the PC controlapplication and the robot to control its hardware. It can be considered as a translatingdevice that translates the commands given by the user for the robot. The interfacebetween the embedded robot controller and the PC control application can be Wired or

Wireless depending on the clients‘ requirement and the design of the system. A wirelessinterface increases the cost and complexity of the system whereas a wired system iseasier to implement and design, also being cheaper. To be programmed in C, thehardware controller for the robotic arm will be developed using a microprocessordevelopment board, as detailed in the chapter 3.

4.6.2 Hardware Components

There are several hardware components that need to be integrated and developed forthe development of the robotic arms of the Surgical Robot.

i) Robotic Arms

Depending on the design of the robot we will design between 2 to 4 robotic arms. Twoof the arms shall be the surgeons left and right arms while the third one will carry theendoscopic camera. The fourth arm will be an optional one allowing the surgeon withmore degrees of freedom to work with. The design of the robotic arms shall consist ofthe following components:




Joints with ActuatorsDynamixel [10] AX-12+ serially controlled actuators with high torque and accuracy

End EffectorsStandard Surgical Instruments will be attached to the end effectors of the roboticarms. Design of gripper hugely depends on the requirements of the client. It can

either be a grabber or a cutter. In the da Vinci robot there is a cutter or grabber thatis used for several functions. The following figures give examples of some of thelaparoscopic instruments that will be used as end effectors. [11]

Fig. 4.8 Surgical Instruments for end-effectors

Sensory FeedbackThe robotic arms and the gripper should consist of sensors that should supply theuser of the system with feedback. This feedback can be in the form of ―forcefeedback‖ or any other kind of reading that should inform the user about the forcethat is being applied at the end-effectors. We will design robotic arms by taking intoaccount the real-life experiences of surgeons doing laparoscopic surgicalprocedures. The surgical team will provide their feedback and experience to thehaptic user interface development team in designing sensory feedback that will beas close to the real feeling as possible.

Power Supply SystemPower supply for the robot and its motors has to be onboard the system. Powersupply for the Interface Application and the Robotic Controller Driver will bedesigned to meet all the requirements of the system

Mechanical Structure of Robotic ArmThe mechanical structure of the robotic arm shall be designed and developedindigenously by our team. We will use light weight aluminum or plastic for the shellof the robotic arm. The end-effectors shall be connected to the serially controlled




actuators through a wire and pulley system. Wire control is necessary aslaparoscopic instruments are very thin and are inserted through holes in a humanbody. Therefore the force transferred from a motor to the end-effectors can only beby a wire or a rigid string connected from the motor to the end-effectors.

The mechanical structure of the robotic arms will be based on the base, the elbow

design and the arm of the robot. The end-effector or manipulator would beconnected to the end of the robotic arm. The number of links or joints that arerequired would be decided during the design research of the robotic arm. This wouldalso affect the degrees of freedom of the robotic arm.

ii) Surgeon‘s Console for Interaction with the Robot

A standard PC will be used for the console that will be used by a surgeon to interactwith the robot as well as the simulator of the Surgical Robot System. The console willbe developed with an ergonomic design so that it can be used by the surgeon withcomfort and for long times. An LCD monitor will be encased in a cubicle typemechanical structure that will allow the surgeon to access all controls of the system

in an easy and ergonomic way.

iii) Haptic Input Controls

As described previously, haptic controls will be developed that will provide users withfeeling feedback. This feedback can be in the form of different types of signalsgenerated by the controls. For example if the surgeon or user tries to move the inputcontrols too fast, then the controls will resist that motion by generating force oppositeto the direction of the movement by the user. In other cases when a surgeon appliestoo much force then the feedback will generate a long pulse of motion in the controlsindicating that too much force is being applied by the user, and if the user uses theinput controls delicately then a small intensity force will be generated by the feedback

system. Therefore once the surgeon becomes used to these input controls, he willknow at once if he is using the controls with perfect control or if he is committingmistakes and applying undue force.

As an example of the hardware components we can consider the following illustrationof a typical surgical system.

Fig. 4.9 A typical Surgical Robot System [2]




4.7 Haptic User Interface

One of the most important parts of the system will be the input controls. In our survey ofexisting simulators, the LapSim [23] surgical simulator is the most widely used simulator forMIS training procedures. One of its‘ major drawbacks is lack of feedback in its input controls.

We intend to address this problem by developing haptic input controls that allow the user tofeel the force that is being exerted by the laparoscopic instruments on the human tissue orbody organs.

A surgeon will be controlling the robot and the simulator by these input controls. Therefore itis necessary that these controls are easy to handle, ergonomic in design so that they can beused for long hours at a time, are accurate in their movements and rugged so that they havea long lifetime. An example of input controls of the lapsim [23] simulator is given in thefollowing figure.

Fig. 4.10 Input controls of LapSim Surgical Simulator [23]

Throughout this project we shall focus on modular approach of development. This meansthat every module that we develop can be added and removed from the system withoutneeding to modify anything. For the same purpose we have targeted our design of the hapticinput controls to be compatible with the existing LapSim surgical simulation software.

4.7.1 Developing Interface Software

The Haptic UI controls will be intelligent controls with a sophisticated embeddedsoftware working in the background. This embedded software will be the primary

interface between the Haptic controls and other modules of the system. Themechatronics team responsible for the mechanical design of the UI controls will also beresponsible for developing the interface software of the input controls.

4.7.2 Developing the Input Controls Hardware

In this phase of development customized input controls would be developed for use bythe surgeons. These controls will be designed after studying the controls of the da VinciSurgical robot and other robots that are available commercially.




We will make efforts to make the input controls compatible with most commerciallyavailable simulators. As most of these simulators have input control hardware that isinterfaced through a USB port, we will make an effort to make these controlscompatible with these software and use USB port for interfacing.

4.7.3 Interfacing the end-effectors

In minimal access surgery also known as keyhole surgery, industry standardequipment is used the world over, which makes the surgeons used to this kind ofequipment. Figure 4.8 lists some of the industry standard equipment that is used bysurgeons. We will interface this equipment with our robotic arms of the surgical robot.This will allow the robotic arms to be safe while operation, at the same time giving theusers a feel for real life equipment. Since our purpose is to develop a training system, itis necessary that our robot is compatible with industry standard laparoscopyequipment.

4.8 Telecommunication Link

One of the objectives of this project is to provide quality healthcare to the masses inPakistan. This is done by providing MIS facilities to people at all rural and urban locations inPakistan. The telecom link is necessary to provide this facility in the Tele-Surgical Robot. Wewill use existing telecom infrastructure available in Pakistan for this purpose. Some of thetelecom links that can be used for this purpose are

PTCL‘s EVO wireless connection [8] PTCL‘s DSL service available in a number of cities in Pakistan [24] Other Edge connections available by different telecom companies

The network interface module will control the communication protocol while using one of the

above links at the hardware layer for data delivery. There are other options available forenabling the telecom link between our robot and its control application. Holy FamilyHospital‘s Telemedicine & eHealth training center has already facilities available for connecting remote locations over WiMax and satellite links. We will use thosecommunication links for testing the remote operation of our robot.

4.9. System Requirements

This section identifies the different software, hardware and infrastructure requirements thatare needed for the implementation of this project.

4.9.1 Hardware requirements

Requirements for Surgical Robot System

Roboard [12] The heart of the robot, this multiprocessor will control all the movements of the

robotic arms of the surgical robot. It will also be responsible for all sensoryfeedback from the robot. The board can be used to run a Linux kernel as well as




Rawalpindi and a Professor at the Rawalpindi Medical College [15].

4.10.1 Test plan for Simulator

Initial level TestingSince we are using the agile software development methodology, the testing of the simulator

system shall be carried out under multiple cycles. The first prototype will be tested by usersand feedback generated will be used to improve on the prototype, developing into a betterand complete application fulfilling all requirements.

Mid-level TestingOnce the final version of the simulator has been developed it will again be tested by usersand feedback will be used to further improve on the final version.

Final TestingFinal testing of the Simulator will be done once the development of the whole projectincluding the Surgical Robot System (SRS) is complete. The Simulator will be used inconjunction with the Surgical Robot System for final phase of testing.

4.10.2 Test plan for Surgical Robot System

The testing for the Surgical Robot System is a very important requirement. The robot willneed to be tested extensively before any training can be carried out using this robot. Wehave developed a comprehensive test plan for the testing and quality control of the SurgicalRobot System.

1. Testing of Hardware and ElectronicsThe electronics and the embedded system of the robot will be tested under differentconditions. Some of the testing criteria are described below

i. Testing of electronics by varying voltage in the power supplyii. Testing the electronics under varying temperaturesiii. Testing the electronics and the embedded software for long hours of

operation

2. Operational Testing of the Robotic ArmsOnce the electronics and the embedded system have been extensively tested, theoperation of the robotic arms and the end-effectors will be tested extensively. For thispurpose we will perform operational testing on a mannequin with similar skin materialas humans for the operation of the surgical instruments of the end-effectors. Severalsurgeons will be invited to test the operation of the system. Their feedback will be

used to improve on the overall operational experience of the training Surgical RobotSystem. The surgical team will play a crucial role in testing and evaluating theperformance of the robotic arms and the end-effectors.

3. Application Software TestingThe robot control application will go through a quality control process during thedevelopment phase. For this purpose a team of quality control professionals will alsobe constituted for this project, which will not only test the operation of the robotcontrol application but will also be responsible for the testing of the embedded




system of the robot controller. All the data generated will be used to improve on thenext versions of the application that are developed in the future.

4. Reliability TestingThe Surgical Robot Training System is a mission critical device. It requires extensivereliability testing. For this purpose several use cases will be developed for testing the

robot control application in conjunction with the embedded robot controller. Failurescenarios will be created and tested on the software application and hardware of therobot.

5. Network TestingThe system will be tested over wired and wireless links for performance andefficiency. Any time lag that occurs due to network operation would be reduced to theminimum possible value. The testing will be carried out using differenttelecommunication technologies that are available in Pakistan and the best one willbe chosen for actual operation of the system.

4.10.3 Evaluation and Test plan for local and remote locations

The final phase for the development of the system will be testing and evaluation atTelemedicine and E-health Training Center, Holy Family Hospital, Rawalpindi, under thesupervision of the Joint Project Director. There will be three phases of testing for the system

1. Test Phase -1

Testing of the Controls of the robot by surgeons performing typical MIS operations onthe ProMIS simulator Mannequin. This testing will be carried out with the Surgeonconsole and Surgical Robot System in close proximity and wired link.

2. Test Phase - 2

Second phase of testing will involve the same test procedures performed over awireless link within the same building. A high speed LAN or WAN link will be utilizedfor this purpose. The video feedback quality and the time to delay of movement fromcontrols to robot will be noted down, and will be improved upon, based on thefeedback from this test and evaluation phase.

3. Test Phase – 3

For the final phase of testing the Surgical Robot System will be installed at a remotelocation and tests will be performed to evaluate the video quality and the time delay

or lag between transmission of control signals to movement of robotic arms will beevaluated. These tests will be conducted over either a WiMax link or a VSat link to beprovided by the Telemedicine and E-health Training Center, Holy family Hospital.

4.11 System Interface and Flow Description

The proposed Surgical Robot Training System and Simulator will have different interfaces.The following section explains the data flow of the robot control application and the SurgicalRobot System along with the Simulator; and also the different interfaces of the system.




4.11.1 System Interfaces

The system will have two major interfaces for the users.

Interface with the Simulator

Input interface to the simulator will be using robot input controls. It will be the same inputinterface for the Simulator as well as the Surgical Robot System.

Interface with the Surgical Robot SystemThe interface with the Surgical Robot System will also be using the robot input controls.These controls will interface with the Robot Control Application which in turn will use theinterface driver layer to connect with the Surgical Robot System.

4.11.2 Data Flow of the System

The flowchart in this section is self explanatory. It shows the flow of data of the wholesystem. User interacts with the robot using a group of Robot controls attached to the robot

control application. These are also the same controls used to interact with the simulatorprogram. The Robot Control Application is the main heart of the whole system. It makes allthe decision of how, when and to whom the different input and output signals to deliver andin what form. For example the trembling motion of a surgeon‘s hands will be filtered out bythe robot control application before sending the movement signals to the interface device.Any accidental sudden movement by the surgeon shall also be filtered out. An intelligentsafety mechanism within the application software will make sure that no untoward movementhappens while the robot is working on a patient. Alarms will be generated if such amovement occurs and every movement will be logged so that the surgeon can view all themovements made by the robotic arms during an operation. An evaluation of the surgeon‘sactivity will also be possible based on the log data generated.




Fig. 4.10 Data Flow of the System

4.12 Utilization Plan

The robotic system once developed will go through an extensive test phase at Holy FamilyHospital, Rawalpindi. As the Telemedicine training center at this hospital has the

infrastructure available for connecting remote locations over satellite and high speedbroadband networks, the remote control of the system and its feedback will be monitoredand tested. Once this phase is complete, the system will move onto the utilization phase.The following are the different utilization phases for the robotic training system

Phase-1: Utilization at Holy Family Hospital for training of surgeons in laparoscopic androbotic surgery

Sensory Feedback

Sensory Feedback (Video & other

sensors)

Haptic UI

Robot Controls

Robot Control

Application

Interface link to Robot

Movement of

Robotic Arms

Simulator Application

Output in VirtualEnvironment

Embedded Robot

Controller




Phase-2: Deployment at remote locations already connected with Holy Family Hospital forremote training of surgeons and control of robotic training system

Phase-3: Testing of the system in the operation theater at Holy Family Hospital with anexpert surgeon at hand at all times to monitor the performance of the system, in case

anything goes wrong.

Phase-4: Commercialization of the technology with improved safety and health regulations inorder for the robotic system to be deployed at remote locations for actual surgery.

4.13 Maintenance Plan

A dedicated team of engineers will be trained and hired by NUST for providing maintenanceand support services to the users of the system. The completion of this project will result in aprototype being developed and tested for further improvement and commercialization of thetechnology.

Furthermore, as part of our plan to commercialize this technology we intend to start atechnology start-up company that will work on refining this technology and improving theoverall concept for marketing the product locally and internationally. This company will alsobe responsible for providing technical support and training to users of the robotic trainingsystem, making it sustainable over a period of time.

B. Project Team:

(Please attach the curriculum vitae (CV) of PD and JPD(s). Also attach the CVs of key research/

development personnel if available. Please follow the format included in Annexure A.The numbers in the table below must tally with the HR Cost sheet in the Budget file. )

Title / Position Number

Project Coordinator 1

Team Leads(Software / Mechatronics / Surgical)

3

Professional Researchers / Developers(Surgical Team)

1

Post Graduate Trainees(Surgical Team)

2

Professional Researchers / Developers(Software Design Team) 2Researchers / Development Assistants(Grad / Undergrad students)(Software Design Team)

2+1

Professional Researchers / Developers(Mechatronics Team)

2

Researcher / Development Assistants(Grad / Undergrad students)

2+1




(Mechatronics Team)

Support Staff

Contract Staff (please specify)

Others (please specify)

Add more rows if requiredC. Team Structure:

(Please define the team structure (organogram) and role/key responsibilities of each member. If in collaboration with another partner, the division of manpower at various locations of partners be provided.)

Title/Position(of each member)

Role/KeyResponsibilities

MinimumQualification

Required

Expertise /BackgroundRequired

MinimumExperienceRequired(years)

Project Coordinator (1) Coordination betweenSoftware, Mechatronicsand Surgical TeamsOverall design of the

Simulator, RobotControl Application,Embedded RobotController andManagement ofengineering andtechnical side of theproject

MS ElectricalEngineering andComputer Science

3

Team Lead (Software Software Design for BS/MS Computer Science/ 4

PD JPD

Project Coordinator

Team Lead (Software) Team Lead(Mechatronics)

Team Lead (Surgical)

Simulator ApplicationTeam (1)

Network ModuleDevelopment Team (1)

Haptic UI Design Team (1) Robotic ManipulatorsTeam (1)

Professional Researcher (1)




Design) (1) Simulator and RobotControl App

Expertise insoftwaremanagement anddevelopment usingWindows / LinuxArchitecture

Team Lead(Mechatronics) (1)

Mechanical and RoboticDesign

BS/MS Electronics/ Mechatronics,Experience indevelopment ofembeddedapplications androbotic design

4

Team Lead (SurgicalTeam) (1)

Location, Holy FamilyHospital

Developing Scenariosand providing support inthe developing of theSimulator. Testing ofSimulator and Robot

MBBS, FCPS Extensiveexperience ofLaparoscopic andRobotic Surgeryand usage ofSurgical Simulators

5

ProfessionalResearcher (SurgicalTeam) (1)

Assisting the Team leadin studying existingdesigns. Working withthe software andmechatronics teams intesting and evaluationof the system.

MBBS, FCPS Experience ofLaparoscopic andRobotic Surgeryand usage ofSurgical Simulators

4

Post Graduate Trainees(2)

Location, Holy FamilyHospital

Assisting in thedevelopment of theTraining Simulator andEmulator, Testing andevaluating the designand interface of thetraining system

1

Sr. Software Engineers(2)

Design andDevelopment ofSimulator and NetworkModule

BS/MS Computer Science,with experience ofOpenGL andOpenCV libraries,Visual C++/.NetLinux/Windows

4

UndergraduateTrainees (2)

Research and literaturestudy. Developing smallmodules for the system

Undergraduate students

Computer Science,C/C++

0

Graduate Trainees (1) Assisting in Design andDevelopment of


MS students Computer Science,expertise in C/C++,

Java, OpenGL

0

Sr. MechatronicsEngineers (2)

Design of RoboticManipulators and end-effectors along withInterface andEmbedded Software

BS / MS Expertise in designof robotic andcontrol systems

4

UndergraduateTrainees (2)

Assist in Design ofRobotic Arm andSurgeon

BS Students Some experiencein mechanicaldesign of robots

0




Console ,EmbeddedSoftware and haptic UIdesign

Graduate Trainees (1) Assisting the team indesign of robotic armsand mechanical

Structures

BS Mechanical Design 0

D. Project Activities:

(Please list and describe the main project activities, including those associated with the transfer of the research results to customers/beneficiaries. The timing and duration of research activities are to be shown in the Gantt chart in Section 8.)

Work Package1

Team Mobilization (3 months)

ActivitiesThe first phase of project will involve hiring of team members and mobilization ofteams. All teams will be divided into sub-teams by their team leads and assigned tasksto perform. Workload will be distributed among these teams. The first phase will alsoinclude procurement of development tools and hardware required initially like PCs,Servers, printers etc.

Deliverables

The deliverable for the first work phase will be the hiring report, team structure and taskassignment report

Work Package 2Literature Study and Utilization Plan(3 months)

Activities

This activity will include an extensive survey of existing technology by the three teams.Each Team will survey existing technology; identifying key elements that can be usedin the design of this project and make a comparison of all technologies studied.Simulator team will study existing open source simulators and open source graphicslibraries that can be used in simulator development. Network Interface team will surveywhat technological infrastructure is available in Pakistan that can be used for tele-oepration of the project. It will further study which libraries are required for socket andnetwork programming of the module. The detailed utilization plan of the project will be

developed. This will include the implementation details of different phases of theutilization plan described in section 4.12. During this stage an initial tentative designdocument will also be prepared which will details the design of the whole system. Thisdocument will be updated and revised as the project progresses through the differentstages of implementation.

Deliverables

Literature Review document of each team




Detailed Utilization Plan

Tentative Design Document of the System

Work Package 3

Analysis (3 months)

Activities

In the second phase, analysis will be carried out by the three development teams ofSoftware, Mechatronics and Surgical. The mechatronics team is further divided intoRobotic Manipulator Development Team and Haptic User Interface DevelopmentTeam. Similarly Software team is further divided into Simulator development team andNetwork Interface development team. An SRS document will be generated by eachteam describing the software requirements of simulator application, network interfacemodule, robotic manipulators, haptic UI controls.

Deliverables

SRS document by each development team will be delivered.

Work Package 4

Design (3 months)

Activities

Work package 3 will be the most crucial phase as this will be the design phase of theproject. All teams will conduct research on the design of their respective modules. Thesimulator will create the initial design of the Simulator, while the Haptic UI team will

design the user interface hardware and software and the Robotic Manipulators teamwill complete the design of the architecture of surgical robot and robotic arms. At theend of 12 months a comprehensive design document of the whole system will also beproduced which will include the design specifications for the different modules of thesystem.

Deliverables

A design document will be delivered by each team on the initial design of each module.

A comprehensive and detailed design document of the whole system will also bedelivered.

Work Package 5

Development Stage 1 (3 months)

This is the second most crucial phase after the design phase. This is will transform thedesign of each module into a reality. Each module will be developed through severalcycles in this work package. Each team will work in parallel ensuring the completion ofthe initial phase of development of each module. The development stage is divided into3 stages. Each stage is of 3 months. After completion of every stage a document or




demonstration will be delivered depending on the stage of development.

Activities

Simulator Development

The simulator core development will be done at this stage. The physics engine will becoded along with the environment of the simulator. The first step is developing the maincore i.e the engine of the simulator. This will be done using OpenGL library. Thisactivity requires significant development and coding.

Network Interface Development

The network interface module will also be developed at this stage by the networkinterface team. This software module will be developed as a separate module, so that itcan be integrated later in the simulator and the robot manipulator control application.

Haptic UI hardware development

The first prototype of the User Interface hardware will be fabricated and its embeddedsoftware controller developed at this stage.

Robotic Manipulators Development

The mechanical design of the robotic manipulators will be fabricated and motors will bemounted on the joints. This robotic arm will have basic control movements at this stage.

Deliverables

Prototypes of the haptic User Interface hardware and surgical robot manipulators andphysics engine of the simulator application will be the deliverables of this phase.

Work Package 6


Activities

Simulator DevelopmentSoft tissue modeling along with development and modeling of internal body organs willbe done at this stage. Blood flow will be designed and integrated with the physicsengine of the simulator.


Interface software will be written for the interface module, that will allow the input




controls to be used with simulator application as well as robotic manipulators.


The mechanical design of the end-effectors will be fabricated and integrated with therobotic manipulators. The main embedded robotic arm manipulation software will bewritten. A network interface module will be developed for the robotic manipulators to becontrolled over a network link.

Deliverables

Development Reports, Simulator Demo (secondary phase), prototypes of haptic UIcontrols and Surgical Robot end-effectors

Work Package 7


Activities

Simulator Development

The GUI and front end application will be designed. Haptic UI controls will be interfacedwith the simulator application. Since the simulator application runs in two modes; onlinemode and offline mode. In online mode it actually behaves as a robotic controlapplication, while in offline mode it behaves as a simulator application. Therefore it isnecessary that the haptic UI controls are integrated with both modes of the application.

The network interface module will be integrated with the simulator application andtested for connectivity as well.


Haptic UI controls will be interfaced with the Simulator application and tested.


Software will be developed for the control of robotic manipulators and end-effectors.The software will also be refined and tuned for the delicate and fine movements of theend-effectors. This input for refinement shall be provided by the surgical team.

DeliverablesDevelopment Reports, Demonstration of Simulator (third phase), Haptic UI controls andSurgical Robot manipulators and end-effectors movement.

Work Package 8

Internal Modular Integration (3 months)




Activities

All the internal modules developed separately will be integrated at this stage.


The application will be integrated with the physics engine module, soft tissue simulationmodule, GUI module and network interface module.

Robotic Manipulators

The robotic arms and end-effectors will be interfaced with the main embedded roboticcontrol application. Robotic manipulators will also be connected with the main simulatorapplication.

Deliverables

Working individual modules of the system will be demonstrated. These demos willinclude Simulator Application and Haptic UI controls with Robotic Manipulators.

Work Package 9

Integration of System Modules (3 months)

Activities

This is another crucial and important phase of the project. Once all the modules havebeen developed and tested independently they will be integrated to form the wholetraining system. All the problems that occur in the integration phase will be addressed

with the system going through another phase of iterations.Deliverables

An development report will be generated which will describe all the features that thesystem is able to perform. An evaluation report will also be delivered by the surgicalteam which will test all the crucial features of the system.

Work Package 10

Integration Testing (3 months)

Activities

In this work package extensive testing will be carried out to evaluate the performanceof the simulator as well as the surgical robot system, in conjunction with the haptic UIcontrols. Integration testing requires all modules to be tested to be working togetherwithout any bugs and errors, therefore all the bugs will be removed in this phase. Allthe development teams will be involved in this phase along with the surgical team.

Deliverables

An evaluation report will be delivered that will contain the integration test andevaluation analysis.




Work Package 11

Network Testing (3 months)

Activities

Once the whole system has been thoroughly tested and evaluated, the next phase willbe to re-evaluate all the capabilities of the system over a network link. The system willbe deployed at a remote location and controlled by surgeons of Holy Family Hospitalusing a network link. Any issues related to the network will be solved at this stage.

Deliverables

An evaluation report will be delivered that will list the performance comparison of thesystem over a wired link versus a wireless link.

A complete demonstration will also be delivered displaying the overall application of thesystem under Tele-operation.

Work Package 12

Deployment (3 months)

Activities

The final phase will include final level of testing, deployment, tutorials, user manualsand release of all documentation.

Deliverables

Tutorials, videos, user manuals of the system.

E. Key Milestones and Deliverables:

(Please list and describe the principal milestones and associated deliverables of the project. Akey milestone is reached when a significant phase in the project is concluded, e.g. selection and simulation of algorithms, completion of architectural design and design documents,commissioning of equipment, completion of test, etc.) The timing of milestones is also to be shown in the Gantt chart in Section 8. Quarterly deliverables are preferred .

The information given in this table will be the basis of monitoring and release of funds by theNational ICT R&D Fund.

No. Elapsed time from start (in months) of the project

Milestone Deliverables

1. 3 Team Mobilization Hiring Report




Team Structure and TaskAssignment Document

2. 6 Literature Survey Literature Survey document ofeach development team

Detailed Utilization PlanTentative Design Document of

the system

3. 9 Analysis Complete Software RequirementSpecification (SRS) documentof each development team

4. 12 Design Stage Complete Design documents by eachdevelopment team

Detailed Design Document ofthe System

5. 15 Prototype Development (1st Stage)

Development Reports (Stage1) for Simulator Application

Prototype 1.0 of Haptic UIControls

Prototype 1.0 of mechanicalstructure of RoboticManipulators

6. 18 Prototype Development (2nd Stage)

Development Report (Stage 2)of Simulator Application

Prototype 2.0 of Haptic UIControls

Prototype 1.0 of SurgicalRobotic end-effectors

7. 21 Modular Development complete(3rd Stage)

Demonstrations of SimulatorApplication,

Haptic UI controls

Robotic ManipulatorsMovement

8. 24 Completion of ModularIntegration

Demonstration of individualmodules of the system

9. 27 System Integration Complete Integration report anddemonstration of the systemwith all modules integrated




10. 30 Integration Testing Complete Evaluation Report

11. 33 Network Testing Complete Network performanceevaluation report anddemonstration

12. 36 Deployment and DocumentationRelease

Project introduction andorientation seminar / workshop

(Please add more rows if required.)

5. Benefits of the Project

A. Direct Customers / Beneficiaries of the Project:

(Please identify clearly the potential customers/beneficiaries of the research results and provide details of their relevance, e.g. size, economic contribution, etc.)

The direct beneficiary of this project is the Telemedicine and E-Health Training Center,Holy Family Hospital, Rawalpindi. Initially the project is being developed for them butthis project has tremendous commercial potential for markets in Asia, Africa andEurope as well.

B. Outputs Expected from the Project:

The output expected from the project is a low-cost simulator and surgical training robotfor training of medical staff in Minimal Invasive Procedures. One of the outcomes of thisproject will be to create a team from different fields and make them experts in using ICTfor developing new products in the field of medicine and robotics.

We will also involve experts in the field of industrial management to advise us regardingthe commercialization of this technology, once a working prototype is complete. Wehave already identified people from the industry that can help us in this process. Someindustry linkages are available between the industry, SEECS and Holy FamilyHospital‘s eHealth and Tele-medicine Training Center, that can be exploited and usedfor this purpose.

C. Organizational / HRD Outcomes Expected:

<type here>

D. Technology Transfer / Diffusion Approach:

(Please describe how the outputs of the project will be transferred to the direct beneficiaries/customers. Please also state if the project outputs are sustainable, i.e. if they can be utilized without further external assistance.)

The robot will be deployed at the Telemedicine and E-health Training Center, HolyFamily Hospital, Rawalpindi. In order for the successful deployment and use of thistechnology video tutorials will be developed, training seminars conducted and usermanuals and web-help will be made available on the project website. Technical support




will be provided to the users. This project once completed can be commerciallymarketed as it has great potential. One of the biggest advantages of this project will beits low cost development and hence over all low market price. Similar systems costthousands of dollars, whereas this system will be very cheap to buy, deploy, maintainand use. It has the potential to sustain itself in the long run.

During the development of this project we will exploit are links with the industry andtake advice on how to commercial this technology once the first working prototype after36 months is complete. For this purpose we have already identified groups working inthe high tech industry in the US, that are willing to help and guide us in this regard. Forthis purpose we will employ the services of persons with proven experience of bringingprototypes to the level of marketable products.

6. Risk Analysis

A. Risks of the Project:

(Please describe the factors that may cause delays in, or prevent implementation of, the project as proposed above; estimate the degree of risk.)

(Please mark where applicable) Low Medium High

Technical risk

Timing risk

Budget risk

A1. Comments:

The technical risk is low in this project, since most of the design has been consideredafter careful brainstorming sessions. Since we are using a modular architecture for the

system, even at some stage if a particular module is not available or has a problem itcan be easily replaced. Off the shelf parts make this project simpler to design andintegrate and with our expertise, the overall technical risk is low.

The timing risk is medium since we have a target of 36 months for completion. In caseof electronics development certain areas take more time to develop than certain others.Therefore there can be unexpected delays. Furthermore, a lot of design research andbrainstorming is required for the mechanical design of the system; hence it is essentialthat enough time is given to the design of each module.

Budget is always a risk. With many of the modules being purchased from outsidePakistan, most of them are priced in US dollars. If the price of dollar fluctuates too

much, than there can be a problem with the equipment costs, although care has beentaken to foresee beforehand any such circumstances and avoid them.




8. Project Schedule / Milestone Chart

(Project schedule using MS-Project (or similar tools) with all tasks, deliverables, milestones, cost estimates, payment schedules clearly indicated are preferred.)




9. Proposed Budget

Please use the embedded Excel Worksheet for providing budget details.

Double click the icon to open the worksheet.

C:\Documents andSettings\Sohail\Deskt




Bibliography




Annexure B – Details of hardware and software requirements

This section contains the details of some of the hardware requirements for the project. Theequipment identified and the prices quoted are for reference only. Actual equipment andprices might be different from those identified. The approximate cost is included with thedescription of each module to be purchased.

1. Roboard Microprocessor

Roboard Microprocessor Board $300

Why is it required?The Roboard is a dedicated development SBC (Single Board Computer) that isrequired for controlling all the mechanical devices of the robot. The actuators for therobotic arm, the feedback from the robotic arm and all the sensors will be interfacedwith this board. This board will run the embedded control application of the robot.

What is the approximate cost?The actual cost of the board is $300. Shipping charges amount to $75 and with tax thetotal approximate cost of the board comes to about $400.




2. Axon Microcontroller

Axon Microcontroller Board $100

Why is it required?The Axon mircrocontroller provides more than 55 I/O (input/output) ports for interfacingdifferent motors including servos and sensors. This will allow the Roboard to interfacewith all the motors and sensors, providing a protection layer to the Roboard, thereforein case of any hardware failure the brain of the robot would still be able to functionwhile this cheap controller will be easily replaceable.

What is the approximate cost?The actual cost of the board is $100. Shipping charges amount to $75 and with tax thetotal approximate cost of the board comes to about $200.

3. Dynamixel Robot Actuator

Dynamixel Actuator $44

Why is it required?The robot will consist of three to four robotic arms. Each robotic arm will have severaldegrees of freedom (several axes movements). Each axis will require its own motor.We have chosen Dynamixel‘s Robot Actuator since it provides sensory feedback for every actuator, which means that at any given time, if any axis of the robotic arm is not




functioning then evasive action can be taken by the robot and warn the surgeon of anypotential danger during operations.

What is the approximate cost?The actual cost of the actuator is $44. Shipping charges amount to $70 and with taxthe total approximate cost of the board comes to about $130. In total about 10 to 16 of

these motors will be required in total. For 16 of these actuators the shipping cost wouldbe about $100, therefore total cost will be $850.

4. Surgeon Console

LCD Monitor with PC Why is it required?

A PC will be required to run the main control application of the robot. This will also beused to run the Simulator. It will be interfaced with the input controls and the hardwareinterface driver of the Surgical Robot.

What is the approximate cost?A usual PC with enough computing power for the current project would approximatelycost about Rs 40,000.

5. End-effectors

Laparoscopic Instruments Why is it required?

The end-effectors or the grippers of the robotic arms will be designed to fit laparoscopicinstruments that are available in the market. This will allow for near perfect control forthe surgeons as they will already be used to these equipments.




References

1. The first Korean Experience of Telemanipulative Robot-Assisted LaparoscopicCholecystectomy Using the da Vinci System, Chang Moo Kang, Hoon Sang Chi,

Woo Jin Hyeung, Kyung Sik Kim, Jin Sub Choi, Woo Jung Lee & Byung Ro Kim,Yonsei Medical Journal, Vol 48. No.3, pp 540-545, 2007 2. Intuitive Surgical

www.intuitivesurgical.com 3. Mako Surgical

www.makosurgical.com 4. NeuroArm

www.neuroarm.org 5. InTouch Health

www.intouchhealth.com 6. ProMIS

www.haptica.com 7. Mt. Sinai School of Medicine, New Yorkhttp://www.roboticoncology.com/data/

8. PTCL Wireless Broadbandhttp://ptcl.com.pk/contentb.php?NID=199

9. OpenGLwww.opengl.org

10. Dynamixel Actuatorswww.robotis.com

11. Laparoscopic Instruments for End-effectorshttp://www.stryker.com/enus/products/Endoscopy/Laparoscopy/Laparoscopes/index.htm

12. Roboardwww.roboard.com/RB-100.htm

13. Axon Microcontrollerhttp://www.societyofrobots.com/axon/

14. OpenCV (Computer Vision Library)http://opencv.willowgarage.com/wiki/

15.Pakistan‘s Telemedicine Project www.telemedicine.pk

16. Laparoscopy Virtual Reality Systemhttp://www.immersion.com

17. SPRING: A General Framework for Collaborative, Real-time Surgical Simulation,Kevin Montgomery, Cynthia Bruyns, Joel Brown, Stephen Sorkin, FredericMazzella, Guillame Thonier, Arnaud Tellier, Benjamin Lermin, Aneel Menon,Medicine Meets Virtual Reality (MMVR), Pages 23-26, 2002

18. GiPSi: An Open Source/Open Architecture Software Development Frameworkfor Surgical Simulation, M. C. Cavusoglu, T. G. Goktekin, F. Tendick, and S. S.Sastry. In Proceedings of Medicine Meets Virtual Reality XII (MMVR 2004),Newport Beach, CA, January 14-17, 2004, pp.46-48.

19. Irrlicht Engine

http://www.intuitivesurgical.com/


http://www.makosurgical.com/


http://www.neuroarm.org/


http://www.intouchhealth.com/


http://www.haptica.com/


http://www.roboticoncology.com/data/


http://www.opengl.org/


http://www.robotis.com/


http://www.stryker.com/enus/products/Endoscopy/Laparoscopy/Laparoscopes/index.htm



http://www.roboard.com/RB-100.htm


http://www.societyofrobots.com/axon/


http://opencv.willowgarage.com/wiki/


http://www.telemedicine.pk/


http://www.immersion.com/




















Annexure C – Details of Work Packages, Milestones, DeliverablesYear 1: Analysis and DesignYear 2: DevelopmentYear 3: Testing & Deployment

WP1 Work Package 1 (HR, Equipment and Initial Design Study) 3 months

- Mobilization of Team (Hiring and Team Selection)- Selection and procurement of initial development tools and hardware

MS 1 : Hiring / Team Mobilization CompleteD1 : Hiring Report / Project commencement report

WP2 Work Package 2 (Literature Study and Utilization Plan) 3 months

- Literature Review (Simulator Team) Studying available open source simulators and libraries

- Literature Review (Haptic UI Team) Surveying designs for haptic user interfaces for Input controls

- Literature Review (Surgical Robot Team) Studying existing surgical robot designs

- Utilization Plan Study Developing a detailed utilization plan for the utilization of the developed technology

- Developing a tentative design of the systemMS 2 : Literature Survey CompleteD2.1 : Literature Review Document of each teamD2.2 : Detailed Utilization Plan D2.2 : Tentative Design Document of the System

WP3 Work Package 3 (Analysis) 3 months

- Software Requirement Specification Document (Simulator and Surgical Teams) SRS for Simulator SRS for GUI and Network Module Will be developed by Surgical and Simulator Design Teams

- Software Requirement Specification Document (Haptic UI Team)

SRS for input controls, will include requirements for core software module of inputcontrols and interface module for interfacing with other parts of the system

Will be developed by haptic controls and surgical teams- Software Requirement Specification Document for Surgical Robot (Surgical Robot

Team) SRS for Real Time OS running on the main control board of the robot. This will also

include FPGA requirements for development as well. This document will include the design of software that is required for the robotic

manipulators Will be developed by Manipulator design team in collaboration with all other teams

MS 3 : Analysis CompleteD3 : SRS Documents of each team

WP4 Work Package 4 (Design) 3 months

- Software Design Document (Simulator Development Team) Overall design and architecture of simulator will be included in the design

document. It will also include details of all modules to be developed in thesimulator

A separate design document will be developed by the network module developmentteam. This will list the design of the network communication model of the system

- Mechanical Design Document (Haptic UI Team) Design of the input controls will be included in this document. It will also include

design of interface circuitry required for interfacing with other parts of the system(simulator and robotic manipulators)

- Mechanical Design Document (Surgical Robot Team)




Overall mechanical design of the robotic manipulators will be included in thisdocument. CAD diagrams of the structure will be included as well as identificationof parts will be done that will be used in the development

MS 4 : Initial Design CompleteD4.1 : Design documents of each teamD4.2 : Detailed Design Document of the System

WP5 Work Package 5 (Development Stage 1) 3 months

- Simulator Development Physics Engine Development (Core Design) Network Interface module development

- Haptic UI hardware development Prototype 1.0 User Interface Hardware with embedded control software

- Robotic Manipulators development The robotic arms will be fabricated and the first prototype will be developed. This

hardware will have basic control movements.MS 5 : Prototype Development CompleteD5 : Development Reports, Simulation Software (Initial Phase), Prototypes of Haptic

UI Controls and Surgical Robotic Arms


- Simulator Development Soft Tissue Modeling will be done to simulate the internal body organs and blood

flow.- Haptic UI hardware

Interface module will be developed that will allow the input controls to be used withthe simulator application as well as the robotic manipulators.

- Robotic Manipulators development The end-effectors will be developed and integrated with the robotic arms. Network interface module to be integrated with embedded control software of

robotic manipulatorsMS 6 : Prototype Development CompleteD6 : Development Reports, Simulation Software (Secondary Phase), Prototypes of

Haptic UI Controls and Surgical Robotic End-Effectors


- Simulator Development Front End Application Design and haptic input control interface module

development Integration and testing of network interface module with front end application

- Haptic UI hardware Haptic User Interface controls will be integrated with simulator application and

tested- Robotic Manipulators

Software for control of robotic arms and end-effectors will be written for controllingthe fine movements of the end-effectors

The movements of the end-effectors will be refined based on input from surgicaldesign team

MS 7 : Prototype Development CompleteD7 : Development Reports and Demonstrations of Simulator Application, Haptic UI

controls and Robotic Manipulators movement

WP8 Work Package 8 (Internal Modular Integration) 3 months

- Simulator Application Integration of physics engine module, soft tissue simulation module, GUI module

and network interface module- Robotic Manipulators



Integration of different robotic arms with the main embedded software robot

controller application Integration of network software module with embedded software robot controller

applicationMS 8 : Completion of Modular IntegrationD8 : Demonstration of Individual Modules of the System (Simulator Application and

Haptic UI Controls with Robotic Manipulator Control)

WP9 Work Package 9 (Integration of System Modules) 3 months

- Integration of Modules Integration of Simulator Application with Haptic Interface Module Integration of Robotic Manipulators with Haptic Interface Module Integration of Simulator Application with Robotic Manipulators

MS 9 : Integration CompleteD9 : Integration Report and Demonstration of the system with all modules

integrated

WP10 Work Package 10 (Integration Testing) 3 months

- Integration Testing of Modules Integration testing of Simulator Application with haptic interface module Integration testing of Robot Control Application with haptic interface module Integration testing of Simulator Application with offline and online control (with

Robotic Manipulators)MS 10 : Integration Testing CompleteD10 : Integration Test Report

WP11 Work Package 11 (Network Testing) 3 months

- Network Testing of all modules Testing of Robotic Manipulators over a network Testing of Haptic Controls over a network link Testing of Robotic Manipulators Control over wired and wireless links

MS 11 : Network Testing CompleteD11 : Network Test and Evaluation Report and Demonstration at Holy Family

Hospital

WP12 Work Package 12 (Deployment and Documentation Release) 3 months

- Final Testing and Deployment- Training Sessions and Workshops

MS 12 : Deployment and Documentation ReleaseD12 : Workshop and Introductory Seminar