IMAGINE CUP 2012

KINECT FUN LABS CHALLENGE

ROUND 1 PROJECT PLAN TEMPLATE

Kinect Fun Labs Challenge Project Plan for [Alternate Reality]

Instructions

This is the Kinect Fun Labs Challenge Round 1 Project Plan Template. This is your Team’s Round 1 Entry

Requirement. It is designed to guide you to include the required components of your Team’s project. Please

use the questions in each section to align your submission with the judging requirements. Steps:

1. Insert your team name above

2. Answer the questions below. Please be thorough.

3. This Application Summary must not exceed 15,000 characters including spaces.

4. The character count starts below the line identified here*.

5. This document must be submitted in the English Language.

6. This document must be named as follows:

Kinect_Fun_Labs_Challenge_Round_1_Project_Plan_[Team Name]. DOC, .DOCX or .PDF,

.RTF or .TXT.

Submit your Team’s Round 1 entry by utilizing the submission form on the entry panel of the Kinect Fun

Labs Challenge page at imaginecup.com no later than then closing date of Round 1 (6 March, 2012, 11:59

GMT).

Questions

1. What problem are you solving as it relates to the Imagine Cup Theme?

Describe the real world problem you are working to solve (not the application itself – that information goes

below). Who will benefit from having this problem solved? How will they benefit? Will your solution impact a

large number of people very broadly, or a smaller number of people very deeply?

2. Name and Description of your Application:

What is the name of your Application or Creation? Describe your Application in detail.

3. Originality & Innovation:

How unique and original is your idea? Is the technology itself new and innovative, or is it the application of

existing NUI technology that is compelling? Were you inspired by an existing application of the Kinect?

4. Pre-existing source code:

If pre-existing source code files or third-party binary libraries are to be incorporated into the Application or

Creation (such as physics and game engines or control toolkits), then this pre-existing source code must be

clearly identified below and must not infringe on any third party rights, and must be used in accordance

with all applicable licensing and use terms. This includes images, music and source code. Tell us what open

source you are using, who owns it and what the applicable license information is.

5. Unique NUI Features:

Describe the Natural User Interface (NUI) features of your application. You must use features that are readily

available in any version of the official Microsoft Kinect for Windows SDK. At this time, you should have a

good idea of any supporting back-end infrastructure or cloud resources that your application requires. If your

application uses any cloud-based or other remote services, please describe the back-end infrastructure in

detail. Submissions that demonstrate technical innovation beyond these baseline features will be highly

regarded.

6. Usability:

User interface applications need to be as intuitive as possible. Is the application easy to use, or does it require

extensive training or trial-and-error? Does the application provide hints or other indicators to show the user

how to interact?

7. Presentation & Polish:

If you were to make a video about your project, what key points would you highlight? What scenarios would

you show the viewer to clarify the purpose and impact of your project?

*Character count starts below this line. Please remember that any application summary that includes more

than 15,000 characters, including spaces, will be disqualified. We strongly recommend that you confirm

your character count prior to submission.

1.) Description of Problem and Beneficiaries:

Acute Myocardial Infarction is one of the most widely cited causes of death among

millions in the world.

Statistics for the same are available at the end of this section. The statistics are far worse

for developing countries like India, Pakistan, China, and Thailand.

By using our gadget to improve the user interface of a cardiac cath-lab, we try to

reduce these mortality rates indirectly by allowing hospitals around the world to better utilize

their resources to provide better health-care to millions.

„‟We also reduce the „door-to-balloon‟ time or the time duration of the procedure that is

critical in order to significantly reduce the mortality rate due to acute MI (Myocardial

Infarction).‟‟

The above diagram is the general breakup of a cardiac cath-lab staff. The (red) marks the

optional staff required for the invasive procedure while the (blue) marks the necessary staff

required for a procedure.

Typically, the people employed at a cardiac cath lab are exposed to radiation during the

course of a procedure. Despite common preventive measures, several leading cardiologists in

India have been diagnosed with cancer. Cardiac care staff in the cardiac cath-lab are sufficiently

susceptible to radiation as well, even more so because they generally have longer shifts in

comparison to the doctors.1

Our implementation using KINECT reduces the exposure risk for all three categories of

staff in the cardiac cath lab helping devote greater resources to analysis of the CT obtained,

while maintaining the health of the cath-lab staff.

Cardiac Cath-lab

RN(Registered Nurse)

Gives anesthesia and monitors its dosage

RT(Radiologist Technician)

Controls C-Arm movements as well as

Patient Bed movements

CVT(Cardiovascular Technologist)

Controls and monitors video feed at directions from

RCIS

Another CVT/RN

Documentation of Diagnosis and

Treatment

Optional Rookie

Generally guided by the RCIS for training

CCU attending +ICU Resident

Optionally required in CC(Critical Care

Cases)

RCIS(Registered Cardiovascular

Invasive Specialist)

Performs Required invasive procedures

Common scenarios that occur when a RCIS needs to analyze a particular frame are:

(i) He requests the CVT (Cardiovascular Technologist) to zoom the „image‟ frame. The CVT

moves into the console room (wasting valuable door-to-balloon time) and zooms the image

followed by panning the zoomed image to the required part.

(ii) The technician has a delay in hearing/listening to the request of the RCIS, responds late, and

skips a few frames ahead. This results in wastage of time since the technician has to trace-back

the sequence frame-by-frame.

(iii) Due to non-central position of the technician in comparison to the „wireless sensor‟

available, the „remote‟ signal emitted does not sufficiently reach the „wireless sensor‟ making

navigation through the CT feed significantly more difficult.

a) Large scale (Benefits for the common man) : Reduce patient mortality rates by

reducing the „door-to-balloon‟ time by improving the user interface of the C-Arm and the video

control.

b) Medium Scale: Reduce radiation exposure for cardiac cath-lab staff, while improving their

utilization in diagnosis and treatment of diseases. More importantly, if the cath-lab staff isn‟t

required to control the video-feeds or the C-Arm movements in the cardiac cath-lab, then they

could possibly aid in greater amounts towards the diagnosis and treatment of the patient or in

other parts of the hospital.

c) Small scale: It would also benefit doctors by reducing their exposure time per invasive

procedure. [1] http://www.theheart.org/article/1361685.do

“Coronary heart disease caused 1 of every 6 deaths in the United States in 2007. Coronary heart

disease mortality in 2007 was 406,351. Each year, an estimated 785 000 Americans will have a

new coronary attack, and 470,000 will have a recurrent attack. It is estimated that an additional

195,000 silent first myocardial infarctions occur each year. Approximately every 25 seconds, an

American will have a coronary event, and approximately every minute, someone will die of

one.”

A statistic obtained from ‘http://circ.ahajournals.org/content/123/4/e18.full.pdf’

The Create Study shows 61% of patients are admitted with STEMI in India.About 9.5 million

deaths, which is about one in six deaths worldwide, occur in the country every year. 2.37 million

people die of cardiovascular disease in India

A statistic obtained from Treatment and outcomes of acute coronary syndromes in

India (CREATE): a prospective analysis of registry data.

„The D2B Alliance advocates six key evidence-based strategies and one optional strategy to help

reduce door-to-balloon times:

1. Cath lab team is available within 20–30 minutes (which will not be required as much

since the only people who need to be available after implementation are the RCIS and

the RN.)‟

A list of important procedures required to reduce door-to-balloon times from:

http://en.wikipedia.org/wiki/Door-to-balloon#Improving_door-to-balloon_times

2.) Name and Description of Application: Our application is named: „Project PAMI‟. This stands for „Primary Acute Myocardial

Infarction‟. Primary Acute Myocardial Infarction accounts for the greatest percentage of deaths

among cardiac-disease related deaths and can be significantly reduced by reducing the „door-to-

balloon time‟.

Application Description:

Our application deals with improvement in two particular areas of the cath-lab system,

replacing the normal Remote-Based User Interface with NUI and therefore, significantly

reducing the „door-to-balloon‟ time.

a) The User Interface for Video Analysis: The current video-analysis tools in a modern

cardiac cath-lab are significantly disorganized and „time-consuming‟. Several factors attribute to

this:

(i) The zoom in and scroll image tools are not readily available at the remote. For an analysis, an

additional person is required to work from the console, which is again generally in a separate

room. This often results in waste-age of „door-to-balloon‟ time. We plan to improve this by

providing two gesture recognize-able functions:

When both the hands are brought closer it‟s a zoom in and when they are taken away it‟s a

zoom out gesture. This way the physician can zoom in/out a particular frame in the scan to

observe the defects more carefully.

In addition, a cursor is provided after the zoom in gesture to scroll around the zoomed

segment. This cursor can be used for scrolling by tracking the movement of the using a „point‟

gesture. The direction point will cause the scrolling to occur at a pre-determined speed which can

be changed by the physician if required.

(ii) Standard controls that are provided on the remote are not sufficiently robust. The time delay

between the „button-press‟ and the „acknowledgement‟ on screen is significant. This causes

further increase in „door-to-balloon‟ time for a procedure, especially if the desired frame is

missed during the course of the procedure.

We plan to make it more natural by providing a push gesture, point gesture and a swipe-left,

swipe-right gesture in order to navigate through the frames obtained.

(b) User interface for C-Arm movement: C-Arm movement is in general done by a

radiologist technician. Most of the C-Arm movement is highly repetitive for a common

procedure. There are 5 common „C-Arm Angiographic views‟. We plan to give activate voice

recognition on basis of a particular gesture in order to navigate to a particular view.

For more clarification: Perform Gesture A Voice Recognition is activated „Request „RAO

20 Caud 20‟ is a sample procedure someone using our gadget would have to do.

The five common angiographic views are:4

i) „RAO 20,Caud 20‟

ii) „PA 0, Caud 30‟

iii) „LAO 50, Caud 30‟

iv) „LAO 50, Cran 30‟

v) „PA 0, Cran 40‟

Each of these views will be enumerated with a voice command, and Voice Recognition will be

activated using a gesture command.

[4] http://www.askdrwiki.com/mediawiki/index.php?title=Coronary_Angiography

3.) Originality and Innovation: The C-Arm movement based on Voice Recognition while controlling the Voice

Recognition via simple gestures is original for this field.

This is possibly the first time the CT-scan and catheterization lab have been influenced

significantly by the use of Kinect‟s revolutionary NUI interface.

4.) Pre-existing Source Code: A brief overview of the control toolkit for Philips system is as follows:

Application area: The User Interface Layer: This layer contains the software for the GUI

(the software interface on the data monitor and the Xper module) and NGUI (Geometry and

Review modules, view pads, etc). It translates user actions to elements (commands) of the

interface provided by the application layer. Furthermore, it provides information to the user

about the system state, based on state information provided on the interface of the application

layer. The UI layer is notified about changes in the application state by means of events

generated by the application layer. The UI layer is where a majority of programming tools and

interfaces are readily available for those who want to develop applications for Philips C-Arm.

This layer‟s access is sufficient for our requirements and its documentation is readily available in

two forms as open-source.

Source: http://docweb.khk.be/Patrick%20Colleman/ARM7/lpc-ARM-book_srn.pdf

Development toolkit is provided by Philips to work on an imaging module for its C-Arm. This

forms a part of interacting with the Philips microprocessor.

Source : 4522 981 37032 CSIP level 1 2-21 of 26DMR100835, Rev:01

Forms the software-hardware interaction module for Philips C-Arm, from „technician‟

booklet.(Freely available)

System ControlPhysical Interface Blocks

•Geometry block

•X-Ray Generation Block

• Image Detection Block

• Image Processing Block

• Image Display Block

• Image Storage Block

Software Architecture

•User Interface Layer

•Application Layer

•Technical Layer

•Embedded Software Layer

5.) Unique NUI Features:

We intend to design a multimodal system (speech + gesture) using the following NUI features in

our application.

i) Skeleton tracking:

We use the skeleton tracker of Microsoft Kinect SDK to track few vital skeletal joints of the

doctor performing the CT scan. We plan to track right hand to use it as a mouse control. The

position of the right hand is shown on the screen and the doctor can move his hand to select

predefined control buttons that are displayed on the screen.

ii) Push:

When the right hand is moved slightly in the direction of depth (i.e,z axis) a push is detected.

When a push action is performed on any one of the predefined buttons displayed on the screen,

the action is taken accordingly. For example, there are two buttons displayed on the screen: 1)

Start and 2) Stop. The doctor can move his hand to focus the mouse pointer on the start/stop

button and perform a push action. This will start/stop recording the CT Scan.

iii) Swipe right/Swipe left:

When the hand is moved swiftly to left/right a swipe left/right is detected. This gesture can be

used to navigate through the frames back and forth.

iv) Zoom-In and Zoom-Out:

When both the hands are brought closer it‟s a zoom out and when they are taken away it‟s a

zoom in gesture. This way the physician can zoom in/out a particular frame in the scan to

observe the defects more carefully. In addition, a cursor is provided after the zoom in gesture to

scroll around the zoomed segment.

v) Audio recognition:

This is the secondary mode of interaction, gesture being the first. The doctor will be able to

perform various actions on the CT recording just by calling out the appropriate word/sentence

like for ex “delete from start to this” will delete all the frames from the beginning to the current

one.

6. Usability The application should be very intuitive and easy to use, since, we want to minimize the

„door-to-balloon‟ time. All of the features mentioned in the NUI section, are simple to use and

very intuitive.

On consultation with doctors at Krishna Institute of Medical Sciences, while proposing

this gadget, their thoughts were that it would greatly simplify things. The gestures are simple to

use and would be recognizing a single person only, by taking advantage of skeletal tracking

provided by Kinect. The C-Arm movement would be easy to perform with Voice Recognition

stepping in only when actually required.

A one week training period might be required however, to let the physicians get

accustomed to the changes in the interface.

7.) Presentation and Polish: Some of the things which we‟d like to stress upon for our presentation would be:

a) Case studies showing the importance of door-to-balloon time and their influence on patient

mortality rates.

(i) „Using a multivariate logistic regression model, the adjusted odds of in-hospital mortality did

not increase significantly with increasing delay from MI symptom onset to first balloon inflation.

However, for door-to-balloon time (median time 1 hour 56 minutes), the adjusted odds of

mortality were significantly increased by 41% to 62% for patients with door-to-balloon times

longer than 2 hours.‟

Source: http://jama.ama-assn.org/content/283/22/2941.full

b) Possible improvements in door-to-balloon time because of „our gadget‟.

(i) In multivariate analysis, six strategies were significantly associated with a faster door-to-

balloon time. These strategies included having emergency medicine physicians activate the

catheterization laboratory (mean reduction in door-to-balloon time, 8.2 minutes), having a single

call to a central page operator activate the laboratory (13.8 minutes), having the emergency

department activate the catheterization laboratory while the patient is en route to the hospital

(15.4 minutes), expecting staff to arrive in the catheterization laboratory within 20 minutes after

being paged (vs. >30 minutes) (19.3 minutes), having an attending cardiologist always on site

(14.6 minutes), and having staff in the emergency department and the catheterization laboratory

use real-time data feedback (8.6 minutes). Despite the effectiveness of these strategies, only a

minority of hospitals surveyed were using them.

Source: http://www.nejm.org/doi/full/10.1056/NEJMsa063117

One of these methods namely, „expecting staff to arrive in the catheterization laboratory 20

minutes after being paged‟ will be made much easier, since with lesser requirement of staff,

lesser mean delay in arrival of staff can be expected. More detailed case studies dealing with the

same would be appropriate as well.

c) Case studies showing the effects of radiation in the catheterization lab in hospitals.

„Cancer risk from professional exposure in staff working in cardiac catheterization laboratory:

Insights from the National Research Council's Biological Effects of Ionizing Radiation VII

Report.‟

„Conclusions: Cumulative professional radiological exposure is associated with a non-negligible

Lifetime attributable risk of cancer for the most exposed contemporary cardiac catheterization

laboratory staff.‟

Source: http://wrp-usa.com/images/Radiation_cardiac_catheterisation.pdf