cw_comp1108_520_hh0764s_20150520_141707_1415.pdf (1.23mb) - upfile.vn.pdf

Ha Trung Hieu - EXTRACTING HEART BEAT THROUGH HEAD MOTION 331 - FPT UNIVERSITY, HANOI, VIETNAM | [email protected] BSC (HONS) COMPUTING

Final Year Project EXTRACTING HEART BEAT THROUGH HEAD MOTION

Student: Ha Trung Hieu

Banner ID: 000865979

Supervisor: MSc Pham Van Vung

May 2015

Words count: 8400

HA TRUNG HIEU - EXTRACTING HEART BEAT THROUGH HEAD MOTION 1

1. ABSTRACT

One of the indicators of health are of primary concern to evaluate and determine the health

status of the person that is the heartbeat. There are many ways to measure the heart rate of people

rely on the achievements of science and technology, they are applied on a heart rate monitor such

as ECG (Electrocardiography). Image processing technology applied to the recognition of the change

of capillaries on the fingers, the color change on the forehead which is also to measure heart rate.

Recently, scientists at MIT have devised a study can get your heart rate recorded through head

movement from the video was recorded by phone, camera, etc. This project will develop technology

based on the results of research and scientific reports of the scientists at MIT. The MITs paper is

Detecting Pulse from Head Motions in Video [1]

Technical extract from the video is the heart rate is based on measuring the motion of the

head due to the impact of the flow of blood to be pumped at each beat. The movement of the head

and the flow of blood to comply with Newton's laws. Method of scientists at MIT are monitoring

points and doing principal component analysis (PCA) to degrade their trajectory into a set of moving

parts. Then select the best components corresponding to the heart rate based on its frequency.

This document will briefly discuss the approach and conduct research based on the research

of scientists at MIT. This project has been studied and implemented in the python script and using

graphics processors OpenCV library. It will then detail the steps that were studied during the project

implementation and the role of research applications. Finally, a conclusion will be drawn as the pros

and cons of using this technology in the measurement of the heart rate.


2. ACKNOWLEDGEMENTS

I want to thank to all those who help and support their enthusiasm in this project:

My supervisor Pham Van Vung with invaluable lessons, knowledge, the enthusiastic guidance and

feedback quickly. My best friend Vu Cong Thanh who always feedback during the analysis, design,

development and testing stages of this project.

Thanks to my grandmother, who brought ideas and goals to implement this Project. I choose a

project related to health, after completion, it will be a tool to help me and the family can monitor

her health and everyone in the family a simple and intuitive way possible.

Thank the teachers at universities FPT (FPT Greenwich) who have given us many lessons and

experiences. Thank the friends, parents, brothers and sisters in the family spent time caring and

enabling me to fulfill all these projects. They are not afraid of difficulties and hard to help me. I

sincerely thank you!


3. TABLE OF CONTENTS 1. ABSTRACT ................................................................................................................................................................. 1

2. ACKNOWLEDGEMENTS ............................................................................................................................................ 2

3. TABLE OF CONTENTS ................................................................................................................................................ 3

4. INTRODUCTION ........................................................................................................................................................ 4

5. LITERATURE REVIEW ................................................................................................................................................ 6

5.1 Background ....................................................................................................................................................... 6

5.1.1 Head Movement ........................................................................................................................................... 6

5.1.2 Beat-to-Beat Variability. ................................................................................................................................ 8

5.2 Method Technology. ......................................................................................................................................... 9

5.3 Face Detecting With Viola Jones Algorithm And Region Selection. ................................................................ 10

5.4 Selection and Tracking Feature Points ............................................................................................................ 14

5.4.1 Feature Points Selection ............................................................................................................................. 14

5.4.2 Feature Points Tracking Using Lucas Kanade Optical Flow Algorithm ........................................................ 18

5.5 Applying Cubic Spline Interpolation Algorithm to Increase Sampling Rate. ................................................... 22

5.6 Temporal Filtering Using 5th Order Butterworth Filter. .................................................................................. 24

5.7 Retaining Most Stable Features Using Confidence Intervals of Distribution. ................................................. 25

5.8 PCA Decomposition ......................................................................................................................................... 27

5.9 Peak Detection ................................................................................................................................................ 30

6. PROBLEM AREA ...................................................................................................................................................... 31

7. EVALUATION ........................................................................................................................................................... 31

8. CONCLUSION .......................................................................................................................................................... 34

9. TABLE OF FIGURES .................................................................................................................................................. 35

10. REFERENCES ........................................................................................................................................................ 36


4. INTRODUCTION

Heart rate is an important indicator of medical diagnosis. This date with the development of

society, there is a lot of heart disease, blood pressure seriously affects the health and longevity of

people in the world. With the development of science and technology, more and more interest in

the dissection of the heart rate information and do not need direct contact with objects such as

other ECG traditional gauges. The application of these techniques do not measure the impact on the

body will be very useful when doctors performed on infants, elderly people who are very difficult to

work with them like other adults, or those with vulnerable skin by traditional measurement

methods. There are many ways to measure heart rate and track new changes in the capillaries on

the forehead or the fingertips, combined with image processing technology. Moreover, as the

population is aging quickly, the regular monitoring of the equipment outside the hospital can give

physicians timely information patterns but also easily identify trends and statistical analysis. The

techniques in the future will reduce the cost of transportation, procurement of equipment for

hospitals and health centers, which will be easily accessible to the poor residential areas, and lack

of facilities.

In this project, we exploit fluctuations in a very "delicate" cycle of head to extract information

about the activities of the heart from a video was filmed by

telephone or a device average normal recording. It's a very

interesting study from scientists at MIT in the past year. In

addition to providing an unobtrusive way to measure heart rate,

this method can also be used to extract useful clinical

information about activity of the heart, such as the change in

length each heart rate, which is related to the health of the

autonomic nervous system.

In addition to moving a deliberately controlled by the

brain or the external impact, our head there is an unintentional

movement. These motions cyclical of blood pumped from the

heart to the head via the abdominal aorta and carotid arteries.

FIGURE 1 BLOOD WAS FLOWING FROM THE HEART PUMPING AND FLOW TO THE HEAD

THROUGH TWO ARTERIES ON EITHER SIDE [2]


(Figure 1). The position of relatively deep artery, at the wrist only, live legs, temples, and both sides

of the neck relatively shallow. At the site can see circuit basis, which the doctors can know the

situation of the arteries. Far the largest artery through the heart valve. The blood vessels that due

to multiple muscle elasticity constitutes, can stretch or bend. In arterial blood is bright red under

pulsed flow. Therefore, we can easily self-heartbeat measured through the comments on both sides

of the front of the neck just below the angle of the jaw. To limit the physical impact on the body, the

measurement of the change of the head position will help us to measure heart rate. The algorithm

is applied to detect the pulse signal from the time blood is pumped from the heart to the arteries.

The approach of the original algorithm that uses an algorithm to determine the Viola Jones

faces, since faces are recognizable place and occupied most of the area on head. Then, use Lucas

Kanade algorithm to figure out the best features in the handle that so that tracking the variability

through each frame in a video image. After the data has been the characteristic of a person, the next

step of the algorithm is the velocity filter of a band, techniques used in this section is Temporal

Filtering using 5th order butter worth filter. Finally the algorithm will use the principal component

analysis (PCA) to find a best periodic signal caused by the pulse in the arteries. Then apply the

algorithm to find the highest point of the pulse, which will have an average value of heart rate.

Before implement the project, the techniques will be described in the following section it will

help to analyze the pulse rate based on a series of rhythm from the recording sequence of head

movements. The results compare with other devices is relatively accurate.


5. LITERATURE REVIEW 5.1 Background

In this background section will provide brief information on some key topics related to the

study. First, the key elements of a cardiac cycle, what they are, and how they work, and thanks

to the elements that make the body and head movements. Finally, some clinical significance of

heart rate variability are discussed.

5.1.1 Head Movement

As stated above in the introduction, when a person is at rest, for example, sit on a chair. The

first part of them will move, and the movement includes two movements are conscious

movement, and the movement has no consciousness (other ingredients that act on the motion).

The conscious movement is deliberate change of body state as shaking his head, leaned forward,

backwards, left or right, this is the big moves easily see with the naked eye. The no consciousness

movement is the movement by acting through the heart pump blood from passing through the

abdominal aorta and carotid arteries in the neck and on the head. This motion to change the very

small, very hard to detect with the naked eye. When the movement that happens, the body that

tend to gravitate towards the front or the rear, with the command of the brain controls the

muscles for balance. From then make a series of moves in a certain period without interruption.

According to the biological knowledge about the human body, a healthy heart is a hollow, it's

just a bit bigger than the fist. It pumps blood continuously through the circulatory system through

the body by repeating the contraction in a certain cycle. The circulatory system is a network of

elastic blood vessels that carry blood through the body. It includes the heart, lungs, aorta, artery

(small aorta) and capillaries (tiny blood vessels). The aorta carries blood to turn the upper body,

such as head, neck through the common carotid artery, carotid artery from which the blood will

continue to migrate to other regions of the head.

Research discovers the vertical direction is the best shaft to determine the motion of the head

on the body caused by the pumping process. While the horizontal axis is the change of state

swaying. To see the details of the move, we made some simple calculations to apply Newton's

second law and III, ignoring many details about fluid dynamics and physiology. We have:


With the heart of a healthy body, in the process pump may be 13ml (0.013 kg) blood in speed

is 0.02 m/s in 0.1s, and the acceleration is 0.9 m/s2 [5]. Assuming both carotid arteries are the

same, and the heads weight is 5kg. In III Newton's law indicates that the individual does not

appear forces that appear in pairs-jet dynamics. In other words, the force appears only when

there is an interaction between two or more things together. Pair this force, the law more clear

III, the driving force for the pair. We have the same magnitude but opposite directions. In the

interaction between two objects Head and B, if A exert a force to B, then B also causes a

force to A and

=

Apply II and III Newtons law we have.

|| = ||

| | = | |

| 5 | = |0.9

2 0.013

|

|| = | (0.9 * 0.013) / 5) |

|| = 0.00234 (

2)

Since the acceleration of the movement of the head, we determine the distance traveled by

the head of the body. By transporting blood from the heart to the arteries is a modified linear

motion are, we consider the trajectory of the blood flow from the heart to the head is straight

and instantaneous velocity of blood or raise them or is declined steadily over time. From there

we have the formula to calculate the distance change were:

=1

2 2

=1

2 0.00234 0.12*1000

s = 0.0117 millimeters

We can see, the movement is very small.


5.1.2 Beat-to-Beat Variability.

The heart rate variability or HRV for short, is a stage of critical health check possible you've

never experienced. HRV involves a series risk such as stress, depression, diabetes, cardiovascular

disease, sleep disorders and abdominal fat. For some people, it acts as a catalyst for major HRV

will be the cause of many problems and diseases is a red alert.

Moreover, according to the cardiologist, ideal HRV indices associated with frequent orgasms!

Speaking more understandable, HRV is some change in a certain time period of the heartbeat.

This can mean anything? Imagine your heart rate, for examples, the number of times your heart

beats per minute, the same rhythm to a song, to extend and keep pace. HRV notice more

complex rhythms, enabling acceleration or deceleration flexibility by changing the elapsed time

between each pulse.

To understand how reduced HRV increases, the first need to have an autonomic nervous

system (ANS). ANS would strongly affect the main parts of the body, including the heart, there

are divided into two branches. Affiliates parasympathetic slows the heart rate and sympathetic

branch helps raise your heart rate. The two leaders discussed the tasks depending on whether

you're tense, relax or battling illness.

If you are in a state of

stress or prolonged sick, the

sympathetic branch will

govern, putting you in the

mood amidst fighting and

give up, will reduce your

HRV. Low HRV is a sign

warning you may develop

other symptoms. Patients with decreased HRV increased risk of arrhythmia, since it is very easy

to kill.

FIGURE 2 AN EXAMPLE OF THE ECG SIGNAL FROM THE MACHINE. THE DIFFERENCE BETWEEN THE R PEAK IN THE PULSE IS USED TO CALCULATE HEART RATE VARIABILITY OR HRV [3]


5.2 Method Technology.

This method requires the input

request is a recorded video changes a

person's head and then the result

returned is a heart rate as well as a series

of point beating of the heart rate. Since it

can be used to analyze the beat-to-beat

variability. The first step of the method is

to identify the face to be treated, then

proceed to take the facial features based

on an algorithm. Then proceed to monitor

the change of position x and y coordinates

of each frame that point until the end of

the length of the input video above. After

having been gathered variation of the

feature points, the next step is to increase

the density (smoothing) which gathers

variation by interpolation method.

Because, above input video file, the video

recorded with handheld devices such as

travel cameras, phones, most of them

have very low speed, only about 25-30

frames/s. After proceedings interpolated

smoothing the data, the data set that is

passed through the interference filters,

filter change point in an unusual way. Then proceed PCA algorithm to select the set of variations

in the most appropriate location. The Figure 3 show the overview of the method. From the data

after running the PCA algorithm, data is transformed into the frequency spectrum, from which

to select a clear frequency spectrum. The average heart rate is determined using these

frequencies.

FIGURE 3 OVERVIEW OF THE TECHNIQUE. [4]


5.3 Face Detecting With Viola Jones Algorithm And Region Selection.

The problem of determining a human face (Face detection) is a computer technique for

determining the positions and the size of the face in any photo (digital image). This technique

identifies the featured faces and ignore other things such as buildings, trees, body, etc.

Some applications of the problem is determined faces: system interaction between humans

and machines (Control computer through facial movements), identification system (enables

security agencies managed humans), observing system monitoring and management system of

the entry for agencies and companies, the system checks the driver is asleep or not, the system

analysis of facial emotions, and facial recognition system for digital cameras, etc.

There are so many methods of facial recognition, which is based on the nature of the method,

can be divided into four main approaches are as follows: The approach is based on knowledge:

knowledge of coding people in all kinds of faces and create sets of rules to determine the face.

The approach is based on the book description does not change: the goal of finding algorithm

described structure featured a human face (the book represents no change to the position,

location of capture device, brightness to change, etc.). The approach is based on the matched

sample: Using the standard sample or the faces of people featured. The approach is based on

appearance: methodological training from a set pattern to identify human faces.

Program to locate human faces in this article using approaches based on appearance, using

strong AdaBoost classifier is a combination of the weak classifier based on Haar-like poses

challenge to determine face. Method development objects and Paul Viola Michael Jones has

suggested in their article "Rapid Object Detection using a Boosted Cascade of Simple Features"

in 2001. It is a machine learning method based on numerous data sets positive train (there

appear face), negative (no facial appearance).

To algorithm works well, we need a large amount of training data set includes positive and

negative images. Then we conducted dissection of features from them. With this algorithm, Viola

and Join announced an algorithm to detect the characteristic is known under the name Haar-

like. There are four basic characteristics to identify human faces. Each Haar-like feature is a

combination of two or three rectangular "white" or "black" as shown below:


FIGURE 4 FOUR BASIC CHARACTERISTICS TO IDENTIFY HUMAN FACES

To use this characteristic to identify human faces, 4 featured Haar-like base is expanding, and is

divided into 3 features set as follows:

FIGURE 5 3 FEATURES SET OF HAAR-LIKE

Each feature is a single value is calculated by subtracting the total number of pixels of the

black rectangle and the total number of pixels of white rectangle. Thus we can see that, for

calculating the value of Haar-like features, one must calculate the sum of the pixels in the image

area. But to calculate the value of the Haar-like feature for all positions on the image requires

considerable computing costs, not met for applications that require run-time. Therefore Viola

and Jones launched a concept called Integral Image, that is a 2-dimensional array with a size

equal to the size of the image to calculate the Haar-like features, with each element of the array

is calculated by aggregating upper pixel (row-1) and left (column-1) of it. Starting position the

upper left to the lower position, right image, this calculation solely on simple addition integer,

so the execution speed is very fast.


FIGURE 6 INTEGRAL IMAGE

Having calculated the Integral Image, the sum of the gray values of a particular area on any

image made very simple way:

Suppose we need to calculate the total value of the gray level D as in Figure 7, can be

calculated as follows:

D = A + B + C + D - (A + B) - (A + C) + A

With A + B + C + D is the value at the point P4 on Integral Image, likewise A + B is the value at

point P2, A + C is the value at point P3, and A is the value at point P1. So we can rewrite the

above expression D calculated as follows:

FIGURE 7EXAMPLES OF FAST CALCULATION OF THE GRAYSCALE VALUES OF POINT D IN THE IMAGE

Next, to choose the Haar-like features for setting thresholds, Viola and Jones uses a machine

learning method AdaBoost. AdaBoost will incorporate the weak classifier to form a strong

classifier.


The weakness of this algorithm is a very long process of learning machine, but strong again shows

that enjoyment is the face detection in images very quickly, in line with the current needs of the

Project. OpenCV has hosted a lot of money training classifications for the identification and

classification of face, eyes, smile, etc. The XML file data is stored in training OpenCV. We need to

do is load the XML classification, then transmit an image input gray.

After you have identified the face, in the course of running the algorithm, there may be errors

in a frame as there appear two faces have different sizes (In fact, the only one), we will proceed

to remove the face with a smaller area, just take the face with the largest area. Because we use

cameras placed close to the subject's face to record the faces of that object.

We will choose 50% of the width of the rectangle

from the midpoint of edge width and 90% height of

the upper edge from the first term to ensure that the

whole rectangle lies in the facial area. To avoid

creating in additional interference signals from the

unwanted Movements like the blink of an eye, the

movement of the pupil. We proceed to remove the

rectangle containing the left and right eyes. To do

this, we remove the rectangle with width sizes as

above, but the length of the start from 20% to 55%

compared with the original length. In the Figure 8. FIGURE 8 SELECTION AREA


The red area is a removing region, the rectangle green border is a selection region. We will

extract features points of this region.

FIGURE 9 REGION FACE SELECTION

5.4 Selection and Tracking Feature Points

5.4.1 Feature Points Selection

As we have calculated above in section 5.1.1, when blood is pumped from the heart to, the

head will be moved compared to original position a distance of about 0.0117 millimeters. This

gap is very small to be seen with the naked eye.

After identifying the faces and the data processing algorithm is presented in section 5.3

above. We have been measuring the movement of the head through the video by searching out

new features in the face processing and tracking their changes through frame by frame. In the

process of understanding, being aware of the feature points to get the point using a recognition


algorithm which is Shi-Tomasi Corner Detector. This algorithm based on Harris Corner Detector

which are some small changes make a better result.

Mathematical model of Harris Corner Detector algorithm is presented summarized as follows:

Suppose we have a grayscale image I, with each point (u, v) and displacement (x, y) we can

calculate the deviation when a window move from (u, v) to (u + x, y + v) as follows:

(1)

In particular, S(x, y) Total value deviation squared, w(u, v) is a window in (u, v), I(u, v) and

I(u + x, v + y) is the light intensity value of the pixel at the position (u, v) and I (u + x, v + y).

I(u + x, v + y) may develop Taylor as follows:

(2)

Ix, Iy is the derivative components of x, y. Since then, (1) can be rewritten as follows:

(3)

If expressed as a matrix, then S (x, y) as follows:

(4)

In particular, A is a tensor structure as follows:

1 and 2 are called the eigenvalues of A. Then, the expression:

(5)

The value Mc will decide if the window w containing corner or not. In late 1994, J.Shi and

C.Tomashi has researched and produced a small change in the formula (5) above:

= min (1, 2)


If this ratio is greater than a threshold value, then it is considered a corner. To demonstrate the

algorithm in a more figurative way. Suppose we have a picture, see the movement of the

windows in the following figure:

FIGURE 10 ILLUSTRATES USE SHI-TOMASI ALGORITHM DETECTOR CORNER IN PICTURES OF HO CHI MINH MAUSOLEUM [7]

The green window corresponding displacement amounts (x, y) from the red window. On the

picture we see no significant differences between the two windows. But consider the following:

FIGURE 11 ILLUSTRATES USE SHI-TOMASI ALGORITHM DETECTOR CORNER IN PICTURES OF HO CHI MINH MAUSOLEUM [7]

Meanwhile with any small movement, S (x, y) will be changed quite large. The purpose of a series

of equations (2), (3), (4), (5) is to find out the change S (x, y) the largest (corresponding to the

appearance of the corner). From equation (5), if Mc larger than a certain value, the window will

contain the corner.


The algorithm Shi-Tomasi Corner Detector has been implemented in the OpenCV library (the

cv2.goodFeaturesToTrack() function). As usual, this function will require the input is a grayscale

image, then you determine N number of strongest corners in that gray picture. Then continue

you determine the level of quality, the minimum Euclidean distance between the detection

corners. Level of quality is a value from 0 to 1, the value used to determine a window there is a

corner or not. If the value of Mc in formula (5) is greater than this value, it will be a corner. Even

if smaller, it will be removed.

In implementation of this project, the number of corners should consider 1500 in face

processing areas. Level of quality is 0.05 (this coefficient is the empirical coefficient). The

Euclidean distance is 0.5.

FIGURE 12 EXAMPLE OF THE RESULT OF APPLYING SHI-TOMASI CORNER DETECTOR


The feature points (red points) will be used to monitor changes in the next frame position

compared to their original position through each frame of video. From then shows the variation

of the point position, it is also regarded as the movement of the head.

5.4.2 Feature Points Tracking Using Lucas Kanade Optical Flow Algorithm

From the feature points (the red point) in the results of the above algorithm, we conduct such

tracking points in the video frame by frame. In Frame 1 we obtain the feature points (e.g. the

feature points in the results above), we conducted tracking them to see the change from frame

t=2...T. (T is the total frame in a video). In each frame t, we have N of the feature points, each

point n we feature our coordinates were ( Xn(t), Yn(t) ). The tracking is done by algorithm Lucas

Kanade Optical Flow.

Optical flow is a concept that refers to the relative motion of the points on the surface of an

object, the object that caused, in terms of a landmark observation (eye, camera, etc.). The

movement of the object (which may actually be considered as the movement of the points on

the surface of that object) in 3-dimensional space, they are projected onto a 2D plane observers

called motion field. Generally, the purpose of the optical flow Estimation method is to determine

(approximately) motion field from a set of frame image changes over time, they are widely used

in object segmentation problems, motion detection, tracking, etc.

The key idea of the method of calculation of optical flow based on the following assumptions

gradient: Appearance of objects without much change (in light intensity - intensity) when

considered from frame t to frame t+1. There are a lot of method determine motion field. Lucas-

Kanade method is one effective method to calculate the motion. The main content method is

summarized as follows:

Some conditions of the algorithm:

1. Light intensity of pixel constant t,

2. Slightest movement

3. Neighboring pixels and moving at the same speed.

About conditional offer of algorithms, data, we fully met, the facial movements are small, and

people sit in front of the camera is guaranteed to light intensity, without any significant changes.


At time t pixel (x, y) color value is I (x, y, t), after time t of pixel position is (x + x, y + y) and I

(x, y, t) = I (x + x, y + y, t + t) (for conditions 1).

That I(x + x, y + y, t + t) I (x, y, t) +

+

+

Therefore, we have

+

+

= 0

Or

+

+

= 0

Put =

; =

We have

+

+

= 0 (*)

The equation (*) is a quadratic equation in two unknowns least Vx, Vy. The 3rd condition

provided, we can use the pixel values adjacent to the above equation. Then we have m equations

system with m>= 2

Solve simultaneous equations so determined by the minimum squares method (Least-Quares)

we get the velocity vector of a moving object.


FIGURE 13 FLOW CHART OF IMPLEMENT LUCAS KANADE ALGORITHM [6]

Algorithms have been deployed strong support of OpenCV library, using a function called

calcOpticalFlowPyrLK() in OpenCV library. The parameter passed to the algorithm includes an

initial gray image, the feature points of image points that gray (The collection of feature points

have been processed from step 5.4.1), and a next frame gray image. The algorithm will return a

list of the status of the feature points in the new gray photo frame. The status of the feature

point includes 0 and 1, which is the first state means the feature point which has appeared in the

next frame, 0 means lost. We iteratively pass these next points as previous points in next step.

Below is a graph of statistical variation first three feature points throughout the course of

handling video. Y-axis coordinate of the Y, X axis coordinates of video time.


FIGURE 14 TRACKING POINT NUMBER 1



The algorithm uses a cell window size 40x40 pixels around each of the feature points to follow

the movement of their location. The size of this window is a rectangular area 1/125 faces found

in section 5.3. It is large enough to get the point feature in the area around the nose and mouth.

Not all the features are appearing in point the next frame, the majority of them will be lost, and

the loss of the feature point does not change too much the quality of the calculations in this

project. The cause of the loss of information of the feature points were:

1. A feature point falls outside of the image.

2. An image patch around the tracked point varies too much between images


A video of us have about 10 seconds equivalent length 300 frames (30 frames / s), after applying

the tracking algorithm, we obtained the following results tracking (the majority of the feature

points will decrease)

Phase 3: Processing Tracked Points -- Tracked 1014 Feature Point in frame 0/300 -- Tracked 1014 Feature Point in frame 1/300 -- Tracked 1014 Feature Point in frame 2/300 -- Tracked 1009 Feature Point in frame 3/300 -- Tracked 1003 Feature Point in frame 4/300 -- Tracked 1003 Feature Point in frame 5/300 -- Tracked 1003 Feature Point in frame 6/300 -- Tracked 1003 Feature Point in frame 7/300 -- Tracked 1003 Feature Point in frame 8/300

5.5 Applying Cubic Spline Interpolation Algorithm to Increase Sampling Rate.

With camera phones now, most of the technology only supports video recording with up to

30 frames per second (about 30Hz). With the average number of frames like this, there can be

fully recorded information on heartbeat. Looking at Figure 12, 13, and 14. We can see, the data

is very fragmented and varies a great way. With data like this, the application of the technique

will face limitations on data quality, can occur many errors or even not to get the desired results.

While the ECG machine, they operate approximately 250Hz range to capture information

about heart rate variability. To get much more data sample, the study of the document indicates

that it applies an interpolation technique to increase the data sample. The technique is Cubic

Spline Interpolation in this study.

In the method using a straight line

connecting two points can be considered as

a first-order polynomial (first derivative is a

straight line). If instead of using the most

we use higher order, the most common is

tertiary, it can force the third derivative of

this first derivative at the junction equal.

This method is commonly known under the

name "cubic spline", which is very popular FIGURE 17 DIFFERENCE BETWEEN LINEAR AND CUBIC SPLINE INTERPOLATION


in engineering way. The graph above shows the difference between linear and cubic spline

interpolation.

This algorithm has been deployed in scipy library using python script. The interp1d class in

scipy.interpolate is a convenient method to create a function based on fixed data points which

can be evaluated anywhere within the domain defined by the given data using linear

interpolation. An instance of this class is created by passing the 1-d vectors comprising the data.

The instance of this class defines a __call__ method and can therefore by treated like a function

which interpolates between known data values to obtain unknown values. Behavior at the

boundary can be specified at instantiation time. The following example demonstrates its use, for

cubic spline interpolation.

FIGURE 18 IMPLEMENT CUBIC SPLINE INTERPOLATION USING SCIPY PYTHON

The code above is deployed in this project, with the aim of increasing the amount of data samples

obtained from the processing of video. For example here, the video was filmed in about 10

seconds with a frame rate of 30 frame / s (equivalent to 10 * 30 = 300 frames). But meanwhile,

the amount of data we need here is about 250 frames / s (equivalent to 10 * 250 = 2500 frames).

Therefore, to make the data more finely, we need to split the density axis of time (t) up. The

density was calculated using the formula 30/250 = 0.12. The three charts below are the results

after Cubic Spline Interpolation algorithms run to increase the sample data of three feature

points are taken from Section 5.4.2.


FIGURE 19 FEATURE POINT NUMBER 1 APPLIED CUBIC SPLINE INTERPOLATION



5.6 Temporal Filtering Using 5th Order Butterworth Filter.

Not all the frequencies of the change of position is useful to detect the pulse. In a normal adult

in a resting state, the average heart rate usually falls in the range [0.75, 2] Hz, equivalent to [45,

120] beats / minute. According to research by MIT scientists, the changes in the lower frequency

0.75Hz usually very adversely affect the results of the program returns. The cause of this motion

because respiration and movement of the head should not have been said in section 5.1.1. But

conversely, the higher frequency 2Hz to provide a lot of useful information about the heart rate,

which in order to find the peak. Because we only consider the location coordinates y (t) of each

feature point. Should we proceed through the techniques 5th Order Butterworth Filter to conduct

data filter frequency range is [0.75, 5] Hz. Another study showed that, for infants, they can beat

faster than adults, so bands applied to the filter to be changed to [1.25, 5] Hz. This algorithm has

been developed in scipy.signal.butter library python, we will use this library to create a band pass


filter Butterworth. To generate the filter coefficients for a band pass filter, we using butter()

function the filter order, the cutoff frequencies Wn=[low, high] (expressed as the fraction of the

Nyquist frequency, which is half the sampling frequency) and the band type btype="band". The

low frequency has been set 0.75, and high has been set 5 Hz. The chart below gives us the results

after applying the filter method of 1 feature points, better data and better quality.

FIGURE 22 DATA OF A FEATURE POINT AFTER APPLYING 5TH BUTTER WORD BANDPASS FILTER

5.7 Retaining Most Stable Features Using Confidence Intervals of

Distribution.

Many feature points have moved unstable variable, points to retain the most stable feature,

we use a method that is relatively classic distribution based on Gausses. After calculating the


maximum distance traveled between successive frames of the feature points. We just took the

feature point with a distance exceeding the 75th percentile.

A popular normal distribution problem involves finding percentiles for X. That is, you are given

the percentage or statistical probability of being at or below a certain x-value, and you have to

find the x-value that corresponds to it. A percentile isn't a percent. A percent is a number

between 0 and 100; a percentile is a value of X (maximum distance).

This process is carried out as follows:

1. Find out the list of the greatest distance of the points under the frame.

2. Then look mean and sigma modeled distributions Gause.

3. Calculate the confidence interval based on value index and mean sigma which has been

found above.

4. Get the feature points have been validated through normal distribution.

Probability distribution classes are located in scipy.stats. The standard deviation is the square

root of the average of the squared deviations from the mean, i.e., std = sqrt(mean(abs(x -

x.mean())**2)). The average squared deviation is normally calculated as x.sum() / N, where N =

len(x). Compute the standard deviation along the specified axis using numpy.std function. This

function will be returned the standard deviation, a measure of the spread of a distribution, of the

array elements. The standard deviation is computed for the flattened array by default, otherwise

over the specified axis. Compute the arithmetic mean along the specified axis using numpy.mean()

function. This function will be returned the average of the array elements. The average is taken over

the flattened array by default, otherwise over the specified axis.

When we perform the analysis on a sample and average values of the results obtained, we

estimate the real value of the sample. Although this average value represents the best estimate of

the true value but it is still only an estimate. We can calculate confidence intervals for these analyzes

to demonstrate the accuracy of the analysis. The confidence interval is determined by limiting the

upper and lower confidence levels. The chart below represent the distribution of a distance of the

feature points, and determine confidence intervals.


FIGURE 23 RETAINING MOST STABLE FEATURES USING CONFIDENCE INTERVALS OF DISTRIBUTION.

5.8 PCA Decomposition

(Principal Component Analysis - PCA) is one method of multivariate data analysis simple. In

statistics, the usual need to "study" the data before building the model inference based on that

data. But sometimes high dimensional data, cannot visualize in 2 or 3-dimensional space, so it

needs to find ways to put the data space of smaller dimension.

PCA is one such method, but more than that, PCA was like a Swiss knife with many good

features:


1. Help reduce the dimensionality of the data.

2. Instead of retaining the axes of the old space, PCA build a new low-dimensional space,

but it also is capable of performing data equivalent good old space, means ensuring

variation (variability) of data on each new dimension.

3. The axes of the new space is a linear combination of the old space, so in terms of

semantics, PCA build new features based on the features observed. Point or are still

performing well this feature original data.

4. In the new space, the potential link data can be discovered, which if put in the old space,

it is difficult to detect, or the link thus not reflected.

Briefly, the objective of PCA is to find a new space (with smaller dimensional space of the

former). The axes of the new space is built so that on each axis, the variance of the data on it is

the largest possible. The objective of the PCA is: maximize the variability.

FIGURE 24 ILLUSTRATIONS PCA: PROJECTION ON DIFFERENT AXES CAN GIVE VERY DIFFERENT VIEWS OF THE SAME DATA.

A classic example is the picture of the camel. Together as a camel but if viewed from

the side, we have enough information, while viewed from the front, it's hard to say it

is camel. A more convincing example is illustrated in the following figure.


FIGURE 25 ILLUSTRATIONS PCA: FIND NEW COORDINATE DATA SO THAT THE HIGHEST VARIABILITY

Assuming the initial data set (file green points) was observed in 3-dimensional space

(black shaft) as shown on the left. Clearly, this 3-axis is not performing the best level of

data variability. PCA will therefore find new coordinate system (the red axes in the picture

left). After finding a new space, the data will be moved into this space to be represented

as shown on the right. Apparently the right image just 2 axes but better representation of

data variability compared with the original 3-D axes. A very nice point of PCA are more

quadrants in the new space ensures double orthogonal to each other, although in the original

space, the axels may be not orthogonal.

The basic signal source is the motion of the head, which are caused by the cardiovascular

pulse. The trajectory of the feature points is a collection of many of the movement change as

respiratory, expression changes in the face, or other deliberate action. We have to decompose

the mix moves into a smaller collection to isolate the signal pulse. To do this we consider the

multidimensional position of the head at each frame as a separate data point and use PCA to find

a set of main dimensions along which the position varies. We then select a dimension on which

to project the position time-series to obtain the pulse signal.

Formally, given N feature points, we represent the N dimensional position of the head at

frame t as m t = [y 1 (t), y 2 (t), , y N (t)]. The mean and the covariance matrix of the positions

are:


=1

||

=1

=1

( )( )

=1

PCA finds the principal axes of variation of the position as the eigenvectors of the covariance

matrix:

=

Where m denotes a diagonal matrix of the eigenvalues 1, 2 N corresponding to the

eigenvectors in the columns of m, 1, 2, , N. Each eigenvector represents the N-

dimensional direction and magnitude of movement for the feature points. The eigenvectors

differ for each subject. We obtain the 1-D position signal si (t) by projecting the position time-

series onto i:

We picked out five objects s1, s2, s3, and s4, s5 after the PCA, to proceed with identifying

peaks and find the heart rate. The maximum frequency of the chosen signal as an approximate

the pulse rate as 60 / fpulse beats per minute.

5.9 Peak Detection

A heartbeat is not lonely, to fully evaluate a complete cardiovascular system, doctors often

consider the index beat-to-beat (presented in section 5.1.2). To do so, we need to get the signal

peak on the newly selected PCA. We now have almost enough data because after applying a

Cubic Spline algorithm Interpolation to increase value from 30Hz up to 250Hz sample (As

according to the ECG machine). The peaks are close to 1

fpulse seconds apart with some variability

due to the natural variability of heartbeats, variations of the head motion, and noise. We label

each sample in the signal as a peak if it is the largest value in a window centered at the sample.


We set the length of the window (in samples) to be round (250

fplulse). The figure below is an example

of beat-to-beat data from PCA chose.

FIGURE 26 EXAMPLES OF MOTION SIGNALS OUTPUTTED BY OUR METHOD. WHITE CIRCLES ARE PEAKS.

6. PROBLEM AREA The biggest difficulty when implementing this project is the difficulty in determining the

topics and self-study techniques that the scientists at MIT have made. With a huge amount of

knowledge to really catch fast in such a short time: The treatment of facial recognition and

detection of feature points based on the tracking algorithm, polynomial interpolation, PCA, etc.

Second, based on the ideas of inventors at MIT, there is no precedent or applications outside

of life, should experience performing fledgling. The papers and documents relating limited and

unclear information.

Finally the time limit for implementing the scheme, in short time, cannot avoid making

mistakes.

7. EVALUATION In modern society, the application of advanced techniques for monitoring the health of the

people is indispensable. With the rise of science and technology as well as the popularity of smart

phones. The implementation of this project to the smart device is feasible and in the near future.

The heart rate monitors or monitor current health equipped primarily in large hospitals, they

have cost the expensive and bulky hard move, depends very much on the electrical equipment.

It is these weaknesses, the health problems in remote areas, far not been adequately care.

In addition, the use of a heart rate monitor to present a direct impact on people, such as

attaching the sensor to the people, this will cause difficulty and monitor health care for children

and adults old.


By analyzing video just to get the heart rate information, which will promise to bring a new

era of health monitoring, detection of crime, etc. Doctors do not need to directly measure the

heart rate of patients, they just sit at home and watch the patient via online video.

There's also a number of other advantages when conducting manufacture or use of heart rate

measurement technology compact and convenient:

1. Walk, exercise: Not only athletes needed new peak heart rates were measured by the normal

exercise can also make use of training to increase efficiency. By maintaining optimal heart

rate range for the reduction of fat, you can optimize training time to burn calories more.

2. Jogging: Measuring heart rate helps you maintain the intensity at the highest level in the days

weight training or aerobic hold mode set lighter these days. Some types also have more

advanced warning function dehydration or hypoglycemia.

3. Bicycles: With heart rate measuring device, you can keep track of the physical situation in the

different training regime type as downhill racing, sprint, on different terrains as asphalt,

terrain, roads Pass. The dedicated type also provides many other parameters when

connected to a bicycle speed meter.

4. Mountaineering, skiing, adventure: For hiking, using cardio equipment help deliverables

reasonable effort to reach the top. For skiers, it helps to check the excitement and thrill

overcome dangerous obstacles

5. Weight loss: Measuring heart rate improves daily exercise and set reasonable diet. Most of

the devices are functional display calorie released when the collective wage and help

determine light intensity consistent weight training to achieve the best weight loss effect.

6. Recovering from injury: With the ability to display information instantly, a heart rate monitor

is useful for physicians to help patients recover from injury or illness, particularly concerning

cases of heart. Thereby can guarantee health recovery safely and stable.

Within the scope of this project, we only provide a prototype version. See this prototype was

developed on the platform python language, and the OpenCV library. Some features are

presented in this report, has not been completed in the prototype as Signal Selection, Peak

Detection function still works less well.


In the near future, after the end of this project, the entire source code of the program will be

shared free of charge to the community under the GNU GPL license. These missing features or

activities are not well developed and perfected. With the desire to create an application that

serves a medical purpose to the community, to help track diseases and health problems of the

places and the poor.


8. CONCLUSION Based on research by computer scientists from MIT, they have discovered a link

between the heart's rhythm and bean variation of head and body. From this infinitesimal

movement, has helped us to obtain information about the heartbeat and the health-related

information.

Through monitoring the movement of the feature point by a motion tracking algorithm, then

perform analysis principal component (PCA), to identify significant shifts and most close to

your heart rate. Thereby heartbeat selected based on the frequency spectrum of its time.

Through assessment and testing on 10 subjects, the results of this project is nearly identical

to the heart rate is measured by the ECG machine. From useful medical information, the

doctors can get information on clinical heart rate variability. Thereby making preliminary

conclusions about heart disease.


9. TABLE OF FIGURES FIGURE 1 BLOOD WAS FLOWING FROM THE HEART PUMPING AND FLOW TO THE HEAD THROUGH TWO ARTERIES ON EITHER SIDE [2] .............................. 4 FIGURE 2 AN EXAMPLE OF THE ECG SIGNAL FROM THE MACHINE. THE DIFFERENCE BETWEEN THE R PEAK IN THE PULSE IS USED TO CALCULATE HEART RATE

VARIABILITY OR HRV [3] .................................................................................................................................................................. 8 FIGURE 3 OVERVIEW OF THE TECHNIQUE. [4] ................................................................................................................................................ 9 FIGURE 4 FOUR BASIC CHARACTERISTICS TO IDENTIFY HUMAN FACES ................................................................................................................ 11 FIGURE 5 3 FEATURES SET OF HAAR-LIKE .................................................................................................................................................... 11 FIGURE 6 INTEGRAL IMAGE ...................................................................................................................................................................... 12 FIGURE 7EXAMPLES OF FAST CALCULATION OF THE GRAYSCALE VALUES OF POINT D IN THE IMAGE ......................................................................... 12 FIGURE 8 SELECTION AREA ...................................................................................................................................................................... 13 FIGURE 9 REGION FACE SELECTION ............................................................................................................................................................ 14 FIGURE 10 ILLUSTRATES USE SHI-TOMASI ALGORITHM DETECTOR CORNER IN PICTURES OF HO CHI MINH MAUSOLEUM [7] ...................................... 16 FIGURE 11 ILLUSTRATES USE SHI-TOMASI ALGORITHM DETECTOR CORNER IN PICTURES OF HO CHI MINH MAUSOLEUM [7] ...................................... 16 FIGURE 12 EXAMPLE OF THE RESULT OF APPLYING SHI-TOMASI CORNER DETECTOR ............................................................................................ 17 FIGURE 13 FLOW CHART OF IMPLEMENT LUCAS KANADE ALGORITHM [6] ........................................................................................................ 20 FIGURE 14 TRACKING POINT NUMBER 1 ..................................................................................................................................................... 21 FIGURE 15 TRACKING POINT NUMBER 2 ..................................................................................................................................................... 21 FIGURE 16 TRACKING POINT NUMBER 3 ..................................................................................................................................................... 21 FIGURE 17 DIFFERENCE BETWEEN LINEAR AND CUBIC SPLINE INTERPOLATION ..................................................................................................... 22 FIGURE 18 IMPLEMENT CUBIC SPLINE INTERPOLATION USING SCIPY PYTHON ...................................................................................................... 23 FIGURE 19 FEATURE POINT NUMBER 1 APPLIED CUBIC SPLINE INTERPOLATION .................................................................................................. 24 FIGURE 20 FEATURE POINT NUMBER 2 APPLIED CUBIC SPLINE INTERPOLATION .................................................................................................. 24 FIGURE 21 FEATURE POINT NUMBER 3 APPLIED CUBIC SPLINE INTERPOLATION .................................................................................................. 24 FIGURE 22 DATA OF A FEATURE POINT AFTER APPLYING 5TH BUTTER WORD BANDPASS FILTER .......................................................................... 25 FIGURE 23 RETAINING MOST STABLE FEATURES USING CONFIDENCE INTERVALS OF DISTRIBUTION. ...................................................................... 27 FIGURE 24 ILLUSTRATIONS PCA: PROJECTION ON DIFFERENT AXES CAN GIVE VERY DIFFERENT VIEWS OF THE SAME DATA. ........................................... 28 FIGURE 25 ILLUSTRATIONS PCA: FIND NEW COORDINATE DATA SO THAT THE HIGHEST VARIABILITY ........................................................................ 29 FIGURE 26 EXAMPLES OF MOTION SIGNALS OUTPUTTED BY OUR METHOD. WHITE CIRCLES ARE PEAKS. ................................................................... 31


10. REFERENCES

[1] Balakrishnan, G., Durand, F. and Guttag, J. (2013). Detecting Pulse from Head Motions in Video. 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[2] Institute, M. (2015). Angiogram. [online] Mayfieldclinic.com. Available at: http://www.mayfieldclinic.com/PE-ANGIO.htm#.VVwKEdLtmko [Accessed 20 May 2015].

[3] Ni.com, (2015). Using LabVIEW for Heart Rate Variability Analysis - National Instruments. [online] Available at: http://www.ni.com/example/30832/en/ [Accessed 20 May 2015].

[4] Balakrishnan, G., Durand, F. and Guttag, J. (2013). Detecting Pulse from Head Motions in Video. 2013 IEEE Conference on Computer Vision and Pattern Recognition, Figure 2 - page 3.

[5] HN. Sabbat et al. Noninvasive evaluation of left ventricular performance based on peak aortic blood acceleration measured with a continuous-wave Doppler velocity meter. Circulation, 74, 1986

[6] Kanade - Lucas - Tomasi (KLT) Feature Tracker - Computer Vision (EEE6503) Fall 2009, Yonsei Univ.

[7] Wikimapia.org, (2015). Lng Ch tch H Ch Minh (Lng Bc) - Wikimapia. [online] Available at: http://wikimapia.org/14596244/vi/L%C4%83ng-Ch%E1%BB%A7-t%E1%BB%8Bch-H%E1%BB%93-Ch%C3%AD-Minh-L%C4%83ng-B%C3%A1c [Accessed 20 May 2015].

cw_comp1108_520_hh0764s_20150520_141707_1415.pdf (1.23mb) - upfile.vn.pdf

Documents