Drowsiness Detection System Using Heartbeat Rate in Android-
based Handheld Devices
Advisor : Dr. Kai-Wei KePresenter : D. Jayasakthi
Department of Electrical Engineering and Computer Science
6/24/2013 3
Introduction• Driver drowsiness is a major cause of traffic crashes.
• Drowsy driving is a serious issue in our society not only because it affects those who are driving while drowsy, but because it puts all other road users in danger.
• Therefore, the use of assisting systems that monitor a driver’s level of vigilance is important to prevent road accidents.
• These systems should then alert the driver in the case of drowsiness or inattention
6/24/2013 4
Motivation
• A common activity in most people’s life is driving; therefore, making driving safe is an important issue in everyday life.
• Even though the driver’s safety is improving in road and vehicle design, the total number of serious crashes is still increasing.
• Most of these crashes result from impairments of the driver’s attention.
6/24/2013 5
Motivation• Drowsiness detection can be done in various ways based on
the results of different researchers.
• The most accurate technique towards driver fatigue detection is dependent on physiological phenomena like brain waves, heart rate etc.
• Also different techniques based on the behaviors can be used, which are natural and non-intrusive.
• These techniques focus on observable visual behaviors from changes in a human’s facial features like eyes, head and face.
6/24/2013 6
Objective
• The aim of the thesis is develop a prototype for drowsiness detection system.
• The application is developed using the android SDK and it will detect the heart beat signals from the i_Mami-HRM2 heart rate monitoring device.
• ECG signal obtained from the sensor is analyzed in time domain and frequency domain.
6/24/2013 7
Objective• In frequency domain, the power spectral density (PSD)
is found.
• From the PSD the Low Frequency(LF) to High Frequency(HF) ratio is estimated.
• It is found that the LF/HF ratio decreases as the person becomes sleepy.
• As a result the drowsiness of a person can be detected from this power ratio.
6/24/2013 8
How it Works
• Autonomic Nervous System (ANS) activity presents alterations during stress, extreme fatigue and drowsiness.
• Wakefulness states are characterized by an increase of sympathetic activity and/or a decrease of parasympathetic activity.
• Extreme relaxation states are characterized by an increase of parasympathetic activity and/or a decrease of sympathetic activity.
6/24/2013 9
How it Works• The ANS activity can be measured non-invasively from the
Heart Rate Variability (HRV) signal obtained from ECG.
• Power on low frequency (LF) band (0.04-0.15Hz) is considered as a measure of sympathetic activity.
• Power on high frequency (HF) band (0.15-0.4 Hz) is considered of parasympathetic origin in classical HRV analysis.
• Balance between sympathetic and parasympathetic systems is measured by the LF/HF ratio.
6/24/2013 10
Methodology• Various methods that has been implemented are:• Bluetooth module• ECG• Measuring Heart beat• Heart Rate Variability
• Various Signal Processing Methods applied to the ECG signals are:• Decimation• Hamming Window• Fast Fourier Transform• Calculate the low to high frequency ratio
6/24/2013 11
I-Mami HRM2 and Android Phone
I-Mami HRM2 sensor from Microtime Computer Inc.
Garmin Asus A50
6/24/2013 12
Pairing the Sensor with the Mobile
• First the device discovery is done in order to connect the sensor with the mobile.
• If a device is discoverable, it will respond to the discovery request by sharing some information, such as the device name and its unique MAC address.
• Once a connection is made with a remote device for the first time, a pairing request will be automatically presented to the user.
• The user must enter a 4 digit pin number for the device to be paired.
6/24/2013 14
Bluetooth Module
A
A
No
Yes
Initialize Bluetooth Socket
Perform a lookup on the remote device in order to match the UUID
UUID - Universally Unique Identifier
6/24/2013 15
1. Main Screen with all modules
2.Bluetooth Module
3. List of paired device
4. Sensor Connected to the mobile 5. Device not connected
6/24/2013 16
Display ECG signals• As a result of the electrical stimulation a change in potential
of the order of 1mV can be measured during the cardiac cycle.
• This signal is known as the electrocardiogram (ECG).
• The ECG detector works mostly by detecting and amplifying the tiny electrical changes on the skin that are caused during each heartbeat.
• The I-Mami HRM2 heart rate monitoring device is used to fetch the heart rate of a person and it is displayed in the android mobile with the help of programmable application, developed by using android SDK.
6/24/2013 17
2. Select Sensor from menu
3. Displays the paired devices 4. Displays the ECG
signals
1. ECG Module Main Screen
6/24/2013 18
Displaying the Heart Rate• The heart rate is the number of heart beats per minute.
• Normal heart rate of a human being depends on the age. For example, children will have higher heart rates comparing with the adults.
• This measurement can be done in various ways with respect to time.• 60 seconds (no calculation needed) - most accurate• 15 seconds (multiply by 4)• 10 seconds (multiply by 6)• Less than 10 seconds = less precise
6/24/2013 19
1.Heart Rate MeasurementModule
2. Select a device from menu
3. Lists the paired device
4. Displays the heart rate and other values.
6/24/2013 20
Heart Rate Variability• Heart rate variability (HRV), known as the variation of the period between consecutive
heartbeats over time.
• HRV refers to the variations in the beat intervals or correspondingly the instantaneous HR.
• In time domain analysis, based on beat to beat or NN intervals some variables are analyzed. They are
• SDNN: Standard Deviation of all normal to normal intervals index. Often calculated over a 24-hour period.
• SDANN, the standard deviation of the average NN intervals calculated over short periods, usually 5 minutes. SDANN is therefore a measure of changes in heart rate due to cycles longer than 5 minutes.
• NN50: Number of pairs of adjacent NN intervals differing by more than 50 ms in the entire recording
• pNN50: The proportion of NN50 divided by total number of NNs.
• AVNN: Average of all NN intervals.
6/24/2013 21
1. Heart Rate Variability
2. Select a device from menu
3. Select the sensor
4.Displays the HRV
6/24/2013 22
Method to Detect DrowsinessDrowsine
ss Detection
Obtain ECG
signal from
sensorReduce
the sampling rate to 50
Hz
Apply Hamming Window
Apply FFT
Calculate LF/HF ratio
Is Ratio Decreasing
Person Becomes Drowsy
Person is not drowsy
No
Yes
6/24/2013 23
Decimation• Consider a band-limited discrete-time signal x(m) with a base-band spectrum
X(f).
• The sampling rate can be decreased by a factor of L through discarding of L–1 samples for every L samples of x(m).
• Decimation by a factor of L can be achieved through a two-stage process of:
(a) Low-pass filtering of the zero-inserted signal by a filter with a cutoff frequency of Fs/2L, where Fs is the sampling rate.
(b) Discarding of L–1 samples for every L samples
• The decimation factor is simply the ratio of the input rate to the output rate. It is usually symbolized by "M", so input rate / output rate=M.
6/24/2013 24
Decimation
• The sampling frequency of the sensor was 250 Hz which means 250 samples per second.
• It was very high to process the ECG signals.
• So the sampling frequency was reduced by 50 Hz which means 250/50 = 5 samples per second .
• The decimation was done using a low pass filter technique.
6/24/2013 25
Hamming Window Technique• Windowing functions, enhances the ability of an FFT to extract spectral
data from signals.
• Windowing functions act on raw data to reduce the effects of the leakage that occurs during an FFT of the data.
• There are many window functions available.
• For an ECG signal the appropriate window function is the Hamming Window.
• The formula for Hamming window is w(n)=0.54−0.46cos(2πn/N−1).
• If x(n) is the signal ,then we get the windowed signal by multiplying x(n) with the w(n) .
6/24/2013 26
Fast Fourier Transform(FFT)• The FFT is a highly elegant and efficient algorithm, which is still one of the
most used algorithms in speech processing, communications, frequency estimation, etc
• Basic radix-2 algorithm is used which requires N to be a power of 2.
• FFT is applied to the windowed ECG signal.
• By applying FFT , the power spectrum was found .
• LF/HF ratio is calculated every 1 minute .
• If this ratio decreases then the person in becoming drowsy.
6/24/2013 29
Conclusion• A non-obstructive, real-time, continuous monitoring method
for determining the drowsiness of the driver has been described .
• From the results it is clear that the LF/HF ratio decreases when the person is sleeping.
• Since ECG is one of the most easy to use physiological signals, a definite relation between drowsiness and HRV may lead to safer driving.
• By applying FFT , the computational complexity is reduced.
6/24/2013 30
Reference• S. Hu and R. Bowlds, "Pulse wave sensor for non-intrusive driver's
drowsiness detection," in Engineering in Medicine and Biology Society, 2009. EMBC 2009. Annual International Conference of the IEEE, Minneapolis, MN, 2009.
• G. Furman, A. Baharav, C. Cahan and S. Akselrod, "Early detection of falling asleep at the wheel: A Heart Rate Variability approach," Computers in Cardiology, pp. 1109-1112, 2008.
• S. Elsenbruch, M. Harnish, and W. C. Orr, “Heart rate variability during waking and sleep in healthy males and females,” Sleep, vol. 22, pp.1067-1071, 1999.